haikuwebkit/Source/JavaScriptCore/ftl/FTLSlowPathCall.cpp

170 lines
6.5 KiB
C++
Raw Permalink Normal View History

FTL should be able to do some simple inline caches using LLVM patchpoints https://bugs.webkit.org/show_bug.cgi?id=123164 Source/JavaScriptCore: Reviewed by Mark Hahnenberg. This implements GetById inline caches in the FTL using llvm.webkit.patchpoint. The idea is that we ask LLVM for a nop slide the size of a GetById inline cache and then fill in the code after LLVM compilation is complete. For now, we just use the system calling convention for the arguments and return. We also still make some assumptions about registers that aren't correct. But, most of the scaffolding is there and this will successfully patch an inline cache. * JavaScriptCore.xcodeproj/project.pbxproj: * assembler/AbstractMacroAssembler.h: * assembler/LinkBuffer.cpp: (JSC::LinkBuffer::finalizeCodeWithoutDisassembly): (JSC::LinkBuffer::linkCode): (JSC::LinkBuffer::allocate): * assembler/LinkBuffer.h: (JSC::LinkBuffer::LinkBuffer): (JSC::LinkBuffer::link): * ftl/FTLAbbreviations.h: (JSC::FTL::constNull): (JSC::FTL::buildCall): * ftl/FTLCapabilities.cpp: (JSC::FTL::canCompile): * ftl/FTLCompile.cpp: (JSC::FTL::fixFunctionBasedOnStackMaps): * ftl/FTLInlineCacheDescriptor.h: Added. (JSC::FTL::InlineCacheDescriptor::InlineCacheDescriptor): (JSC::FTL::GetByIdDescriptor::GetByIdDescriptor): (JSC::FTL::GetByIdDescriptor::stackmapID): (JSC::FTL::GetByIdDescriptor::codeOrigin): (JSC::FTL::GetByIdDescriptor::uid): * ftl/FTLInlineCacheSize.cpp: Added. (JSC::FTL::sizeOfGetById): (JSC::FTL::sizeOfPutById): * ftl/FTLInlineCacheSize.h: Added. * ftl/FTLIntrinsicRepository.h: * ftl/FTLJITFinalizer.cpp: (JSC::FTL::JITFinalizer::finalizeFunction): * ftl/FTLJITFinalizer.h: * ftl/FTLLocation.cpp: (JSC::FTL::Location::directGPR): * ftl/FTLLocation.h: * ftl/FTLLowerDFGToLLVM.cpp: (JSC::FTL::LowerDFGToLLVM::compileGetById): * ftl/FTLOutput.h: (JSC::FTL::Output::call): * ftl/FTLSlowPathCall.cpp: Added. (JSC::FTL::callOperation): * ftl/FTLSlowPathCall.h: Added. (JSC::FTL::SlowPathCall::SlowPathCall): (JSC::FTL::SlowPathCall::call): (JSC::FTL::SlowPathCall::key): * ftl/FTLSlowPathCallKey.cpp: Added. (JSC::FTL::SlowPathCallKey::dump): * ftl/FTLSlowPathCallKey.h: Added. (JSC::FTL::SlowPathCallKey::SlowPathCallKey): (JSC::FTL::SlowPathCallKey::usedRegisters): (JSC::FTL::SlowPathCallKey::callTarget): (JSC::FTL::SlowPathCallKey::offset): (JSC::FTL::SlowPathCallKey::isEmptyValue): (JSC::FTL::SlowPathCallKey::isDeletedValue): (JSC::FTL::SlowPathCallKey::operator==): (JSC::FTL::SlowPathCallKey::hash): (JSC::FTL::SlowPathCallKeyHash::hash): (JSC::FTL::SlowPathCallKeyHash::equal): * ftl/FTLStackMaps.cpp: (JSC::FTL::StackMaps::Location::directGPR): * ftl/FTLStackMaps.h: * ftl/FTLState.h: * ftl/FTLThunks.cpp: (JSC::FTL::slowPathCallThunkGenerator): * ftl/FTLThunks.h: (JSC::FTL::Thunks::getSlowPathCallThunk): * jit/CCallHelpers.h: (JSC::CCallHelpers::setupArguments): * jit/GPRInfo.h: * jit/JITInlineCacheGenerator.cpp: (JSC::garbageStubInfo): (JSC::JITInlineCacheGenerator::JITInlineCacheGenerator): (JSC::JITByIdGenerator::finalize): * jit/JITInlineCacheGenerator.h: (JSC::JITByIdGenerator::slowPathBegin): * jit/RegisterSet.cpp: (JSC::RegisterSet::stackRegisters): (JSC::RegisterSet::specialRegisters): (JSC::RegisterSet::calleeSaveRegisters): (JSC::RegisterSet::allGPRs): (JSC::RegisterSet::allFPRs): (JSC::RegisterSet::allRegisters): (JSC::RegisterSet::dump): * jit/RegisterSet.h: (JSC::RegisterSet::exclude): (JSC::RegisterSet::numberOfSetRegisters): (JSC::RegisterSet::RegisterSet): (JSC::RegisterSet::isEmptyValue): (JSC::RegisterSet::isDeletedValue): (JSC::RegisterSet::operator==): (JSC::RegisterSet::hash): (JSC::RegisterSetHash::hash): (JSC::RegisterSetHash::equal): * runtime/Options.h: Source/WTF: Reviewed by Mark Hahnenberg. This needed some better bitvector support, like merging (|=), excluding (&=~), hashing, and bit counting. * wtf/BitVector.cpp: (WTF::BitVector::setSlow): (WTF::BitVector::excludeSlow): (WTF::BitVector::bitCountSlow): (WTF::BitVector::equalsSlowCase): (WTF::BitVector::hashSlowCase): (WTF::BitVector::dump): * wtf/BitVector.h: (WTF::BitVector::merge): (WTF::BitVector::exclude): (WTF::BitVector::bitCount): (WTF::BitVector::BitVector): (WTF::BitVector::isEmptyValue): (WTF::BitVector::isDeletedValue): (WTF::BitVector::isEmptyOrDeletedValue): (WTF::BitVector::operator==): (WTF::BitVector::hash): (WTF::BitVectorHash::hash): (WTF::BitVectorHash::equal): * wtf/HashTraits.h: (WTF::CustomHashTraits::constructDeletedValue): (WTF::CustomHashTraits::isDeletedValue): (WTF::CustomHashTraits::emptyValue): (WTF::CustomHashTraits::isEmptyValue): * wtf/StdLibExtras.h: (WTF::bitCount): Canonical link: https://commits.webkit.org/141315@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@157872 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2013-10-23 18:22:09 +00:00
/*
Use constexpr instead of const in symbol definitions that are obviously constexpr. https://bugs.webkit.org/show_bug.cgi?id=201879 Rubber-stamped by Joseph Pecoraro. Source/bmalloc: * bmalloc/AvailableMemory.cpp: * bmalloc/IsoTLS.h: * bmalloc/Map.h: * bmalloc/Mutex.cpp: (bmalloc::Mutex::lockSlowCase): * bmalloc/PerThread.h: * bmalloc/Vector.h: * bmalloc/Zone.h: Source/JavaScriptCore: const may require external storage (at the compiler's whim) though these currently do not. constexpr makes it clear that the value is a literal constant that can be inlined. In most cases in the code, when we say static const, we actually mean static constexpr. I'm changing the code to reflect this. * API/JSAPIValueWrapper.h: * API/JSCallbackConstructor.h: * API/JSCallbackObject.h: * API/JSContextRef.cpp: * API/JSWrapperMap.mm: * API/tests/CompareAndSwapTest.cpp: * API/tests/TypedArrayCTest.cpp: * API/tests/testapi.mm: (testObjectiveCAPIMain): * KeywordLookupGenerator.py: (Trie.printAsC): * assembler/ARMv7Assembler.h: * assembler/AssemblerBuffer.h: * assembler/AssemblerCommon.h: * assembler/MacroAssembler.h: * assembler/MacroAssemblerARM64.h: * assembler/MacroAssemblerARM64E.h: * assembler/MacroAssemblerARMv7.h: * assembler/MacroAssemblerCodeRef.h: * assembler/MacroAssemblerMIPS.h: * assembler/MacroAssemblerX86.h: * assembler/MacroAssemblerX86Common.h: (JSC::MacroAssemblerX86Common::absDouble): (JSC::MacroAssemblerX86Common::negateDouble): * assembler/MacroAssemblerX86_64.h: * assembler/X86Assembler.h: * b3/B3Bank.h: * b3/B3CheckSpecial.h: * b3/B3DuplicateTails.cpp: * b3/B3EliminateCommonSubexpressions.cpp: * b3/B3FixSSA.cpp: * b3/B3FoldPathConstants.cpp: * b3/B3InferSwitches.cpp: * b3/B3Kind.h: * b3/B3LowerToAir.cpp: * b3/B3NativeTraits.h: * b3/B3ReduceDoubleToFloat.cpp: * b3/B3ReduceLoopStrength.cpp: * b3/B3ReduceStrength.cpp: * b3/B3ValueKey.h: * b3/air/AirAllocateRegistersByGraphColoring.cpp: * b3/air/AirAllocateStackByGraphColoring.cpp: * b3/air/AirArg.h: * b3/air/AirCCallSpecial.h: * b3/air/AirEmitShuffle.cpp: * b3/air/AirFixObviousSpills.cpp: * b3/air/AirFormTable.h: * b3/air/AirLowerAfterRegAlloc.cpp: * b3/air/AirPrintSpecial.h: * b3/air/AirStackAllocation.cpp: * b3/air/AirTmp.h: * b3/testb3_6.cpp: (testInterpreter): * bytecode/AccessCase.cpp: * bytecode/CallLinkStatus.cpp: * bytecode/CallVariant.h: * bytecode/CodeBlock.h: * bytecode/CodeOrigin.h: * bytecode/DFGExitProfile.h: * bytecode/DirectEvalCodeCache.h: * bytecode/ExecutableToCodeBlockEdge.h: * bytecode/GetterSetterAccessCase.cpp: * bytecode/LazyOperandValueProfile.h: * bytecode/ObjectPropertyCondition.h: * bytecode/ObjectPropertyConditionSet.cpp: * bytecode/PolymorphicAccess.cpp: * bytecode/PropertyCondition.h: * bytecode/SpeculatedType.h: * bytecode/StructureStubInfo.cpp: * bytecode/UnlinkedCodeBlock.cpp: (JSC::UnlinkedCodeBlock::typeProfilerExpressionInfoForBytecodeOffset): * bytecode/UnlinkedCodeBlock.h: * bytecode/UnlinkedEvalCodeBlock.h: * bytecode/UnlinkedFunctionCodeBlock.h: * bytecode/UnlinkedFunctionExecutable.h: * bytecode/UnlinkedModuleProgramCodeBlock.h: * bytecode/UnlinkedProgramCodeBlock.h: * bytecode/ValueProfile.h: * bytecode/VirtualRegister.h: * bytecode/Watchpoint.h: * bytecompiler/BytecodeGenerator.h: * bytecompiler/Label.h: * bytecompiler/NodesCodegen.cpp: (JSC::ThisNode::emitBytecode): * bytecompiler/RegisterID.h: * debugger/Breakpoint.h: * debugger/DebuggerParseData.cpp: * debugger/DebuggerPrimitives.h: * debugger/DebuggerScope.h: * dfg/DFGAbstractHeap.h: * dfg/DFGAbstractValue.h: * dfg/DFGArgumentsEliminationPhase.cpp: * dfg/DFGByteCodeParser.cpp: * dfg/DFGCSEPhase.cpp: * dfg/DFGCommon.h: * dfg/DFGCompilationKey.h: * dfg/DFGDesiredGlobalProperty.h: * dfg/DFGEdgeDominates.h: * dfg/DFGEpoch.h: * dfg/DFGForAllKills.h: (JSC::DFG::forAllKilledNodesAtNodeIndex): * dfg/DFGGraph.cpp: (JSC::DFG::Graph::isLiveInBytecode): * dfg/DFGHeapLocation.h: * dfg/DFGInPlaceAbstractState.cpp: * dfg/DFGIntegerCheckCombiningPhase.cpp: * dfg/DFGIntegerRangeOptimizationPhase.cpp: * dfg/DFGInvalidationPointInjectionPhase.cpp: * dfg/DFGLICMPhase.cpp: * dfg/DFGLazyNode.h: * dfg/DFGMinifiedID.h: * dfg/DFGMovHintRemovalPhase.cpp: * dfg/DFGNodeFlowProjection.h: * dfg/DFGNodeType.h: * dfg/DFGObjectAllocationSinkingPhase.cpp: * dfg/DFGPhantomInsertionPhase.cpp: * dfg/DFGPromotedHeapLocation.h: * dfg/DFGPropertyTypeKey.h: * dfg/DFGPureValue.h: * dfg/DFGPutStackSinkingPhase.cpp: * dfg/DFGRegisterBank.h: * dfg/DFGSSAConversionPhase.cpp: * dfg/DFGSSALoweringPhase.cpp: * dfg/DFGSpeculativeJIT.cpp: (JSC::DFG::SpeculativeJIT::compileDoubleRep): (JSC::DFG::compileClampDoubleToByte): (JSC::DFG::SpeculativeJIT::compileArithRounding): (JSC::DFG::compileArithPowIntegerFastPath): (JSC::DFG::SpeculativeJIT::compileArithPow): (JSC::DFG::SpeculativeJIT::emitBinarySwitchStringRecurse): * dfg/DFGStackLayoutPhase.cpp: * dfg/DFGStoreBarrierInsertionPhase.cpp: * dfg/DFGStrengthReductionPhase.cpp: * dfg/DFGStructureAbstractValue.h: * dfg/DFGVarargsForwardingPhase.cpp: * dfg/DFGVariableEventStream.cpp: (JSC::DFG::VariableEventStream::reconstruct const): * dfg/DFGWatchpointCollectionPhase.cpp: * disassembler/ARM64/A64DOpcode.h: * ftl/FTLLocation.h: * ftl/FTLLowerDFGToB3.cpp: (JSC::FTL::DFG::LowerDFGToB3::compileArithRandom): * ftl/FTLSlowPathCall.cpp: * ftl/FTLSlowPathCallKey.h: * heap/CellContainer.h: * heap/CellState.h: * heap/ConservativeRoots.h: * heap/GCSegmentedArray.h: * heap/HandleBlock.h: * heap/Heap.cpp: (JSC::Heap::updateAllocationLimits): * heap/Heap.h: * heap/HeapSnapshot.h: * heap/HeapUtil.h: (JSC::HeapUtil::findGCObjectPointersForMarking): * heap/IncrementalSweeper.cpp: * heap/LargeAllocation.h: * heap/MarkedBlock.cpp: * heap/Strong.h: * heap/VisitRaceKey.h: * heap/Weak.h: * heap/WeakBlock.h: * inspector/JSInjectedScriptHost.h: * inspector/JSInjectedScriptHostPrototype.h: * inspector/JSJavaScriptCallFrame.h: * inspector/JSJavaScriptCallFramePrototype.h: * inspector/agents/InspectorConsoleAgent.cpp: * inspector/agents/InspectorRuntimeAgent.cpp: (Inspector::InspectorRuntimeAgent::getRuntimeTypesForVariablesAtOffsets): * inspector/scripts/codegen/generate_cpp_protocol_types_header.py: (CppProtocolTypesHeaderGenerator._generate_versions): * inspector/scripts/tests/generic/expected/version.json-result: * interpreter/Interpreter.h: * interpreter/ShadowChicken.cpp: * jit/BinarySwitch.cpp: * jit/CallFrameShuffler.h: * jit/ExecutableAllocator.h: * jit/FPRInfo.h: * jit/GPRInfo.h: * jit/ICStats.h: * jit/JITThunks.h: * jit/Reg.h: * jit/RegisterSet.h: * jit/TempRegisterSet.h: * jsc.cpp: * parser/ASTBuilder.h: * parser/Nodes.h: * parser/SourceCodeKey.h: * parser/SyntaxChecker.h: * parser/VariableEnvironment.h: * profiler/ProfilerOrigin.h: * profiler/ProfilerOriginStack.h: * profiler/ProfilerUID.h: * runtime/AbstractModuleRecord.cpp: * runtime/ArrayBufferNeuteringWatchpointSet.h: * runtime/ArrayConstructor.h: * runtime/ArrayConventions.h: * runtime/ArrayIteratorPrototype.h: * runtime/ArrayPrototype.cpp: (JSC::setLength): * runtime/AsyncFromSyncIteratorPrototype.h: * runtime/AsyncGeneratorFunctionPrototype.h: * runtime/AsyncGeneratorPrototype.h: * runtime/AsyncIteratorPrototype.h: * runtime/AtomicsObject.cpp: * runtime/BigIntConstructor.h: * runtime/BigIntPrototype.h: * runtime/BooleanPrototype.h: * runtime/ClonedArguments.h: * runtime/CodeCache.h: * runtime/ControlFlowProfiler.h: * runtime/CustomGetterSetter.h: * runtime/DateConstructor.h: * runtime/DatePrototype.h: * runtime/DefinePropertyAttributes.h: * runtime/ErrorPrototype.h: * runtime/EvalExecutable.h: * runtime/Exception.h: * runtime/ExceptionHelpers.cpp: (JSC::invalidParameterInSourceAppender): (JSC::invalidParameterInstanceofSourceAppender): * runtime/ExceptionHelpers.h: * runtime/ExecutableBase.h: * runtime/FunctionExecutable.h: * runtime/FunctionRareData.h: * runtime/GeneratorPrototype.h: * runtime/GenericArguments.h: * runtime/GenericOffset.h: * runtime/GetPutInfo.h: * runtime/GetterSetter.h: * runtime/GlobalExecutable.h: * runtime/Identifier.h: * runtime/InspectorInstrumentationObject.h: * runtime/InternalFunction.h: * runtime/IntlCollatorConstructor.h: * runtime/IntlCollatorPrototype.h: * runtime/IntlDateTimeFormatConstructor.h: * runtime/IntlDateTimeFormatPrototype.h: * runtime/IntlNumberFormatConstructor.h: * runtime/IntlNumberFormatPrototype.h: * runtime/IntlObject.h: * runtime/IntlPluralRulesConstructor.h: * runtime/IntlPluralRulesPrototype.h: * runtime/IteratorPrototype.h: * runtime/JSArray.cpp: (JSC::JSArray::tryCreateUninitializedRestricted): * runtime/JSArray.h: * runtime/JSArrayBuffer.h: * runtime/JSArrayBufferView.h: * runtime/JSBigInt.h: * runtime/JSCJSValue.h: * runtime/JSCell.h: * runtime/JSCustomGetterSetterFunction.h: * runtime/JSDataView.h: * runtime/JSDataViewPrototype.h: * runtime/JSDestructibleObject.h: * runtime/JSFixedArray.h: * runtime/JSGenericTypedArrayView.h: * runtime/JSGlobalLexicalEnvironment.h: * runtime/JSGlobalObject.h: * runtime/JSImmutableButterfly.h: * runtime/JSInternalPromiseConstructor.h: * runtime/JSInternalPromiseDeferred.h: * runtime/JSInternalPromisePrototype.h: * runtime/JSLexicalEnvironment.h: * runtime/JSModuleEnvironment.h: * runtime/JSModuleLoader.h: * runtime/JSModuleNamespaceObject.h: * runtime/JSNonDestructibleProxy.h: * runtime/JSONObject.cpp: * runtime/JSONObject.h: * runtime/JSObject.h: * runtime/JSPromiseConstructor.h: * runtime/JSPromiseDeferred.h: * runtime/JSPromisePrototype.h: * runtime/JSPropertyNameEnumerator.h: * runtime/JSProxy.h: * runtime/JSScope.h: * runtime/JSScriptFetchParameters.h: * runtime/JSScriptFetcher.h: * runtime/JSSegmentedVariableObject.h: * runtime/JSSourceCode.h: * runtime/JSString.cpp: * runtime/JSString.h: * runtime/JSSymbolTableObject.h: * runtime/JSTemplateObjectDescriptor.h: * runtime/JSTypeInfo.h: * runtime/MapPrototype.h: * runtime/MinimumReservedZoneSize.h: * runtime/ModuleProgramExecutable.h: * runtime/NativeExecutable.h: * runtime/NativeFunction.h: * runtime/NativeStdFunctionCell.h: * runtime/NumberConstructor.h: * runtime/NumberPrototype.h: * runtime/ObjectConstructor.h: * runtime/ObjectPrototype.h: * runtime/ProgramExecutable.h: * runtime/PromiseDeferredTimer.cpp: * runtime/PropertyMapHashTable.h: * runtime/PropertyNameArray.h: (JSC::PropertyNameArray::add): * runtime/PrototypeKey.h: * runtime/ProxyConstructor.h: * runtime/ProxyObject.cpp: (JSC::ProxyObject::performGetOwnPropertyNames): * runtime/ProxyRevoke.h: * runtime/ReflectObject.h: * runtime/RegExp.h: * runtime/RegExpCache.h: * runtime/RegExpConstructor.h: * runtime/RegExpKey.h: * runtime/RegExpObject.h: * runtime/RegExpPrototype.h: * runtime/RegExpStringIteratorPrototype.h: * runtime/SamplingProfiler.cpp: * runtime/ScopedArgumentsTable.h: * runtime/ScriptExecutable.h: * runtime/SetPrototype.h: * runtime/SmallStrings.h: * runtime/SparseArrayValueMap.h: * runtime/StringConstructor.h: * runtime/StringIteratorPrototype.h: * runtime/StringObject.h: * runtime/StringPrototype.h: * runtime/Structure.h: * runtime/StructureChain.h: * runtime/StructureRareData.h: * runtime/StructureTransitionTable.h: * runtime/Symbol.h: * runtime/SymbolConstructor.h: * runtime/SymbolPrototype.h: * runtime/SymbolTable.h: * runtime/TemplateObjectDescriptor.h: * runtime/TypeProfiler.cpp: * runtime/TypeProfiler.h: * runtime/TypeProfilerLog.cpp: * runtime/VarOffset.h: * testRegExp.cpp: * tools/HeapVerifier.cpp: (JSC::HeapVerifier::checkIfRecorded): * tools/JSDollarVM.cpp: * wasm/WasmB3IRGenerator.cpp: * wasm/WasmBBQPlan.cpp: * wasm/WasmFaultSignalHandler.cpp: * wasm/WasmFunctionParser.h: * wasm/WasmOMGForOSREntryPlan.cpp: * wasm/WasmOMGPlan.cpp: * wasm/WasmPlan.cpp: * wasm/WasmSignature.cpp: * wasm/WasmSignature.h: * wasm/WasmWorklist.cpp: * wasm/js/JSWebAssembly.h: * wasm/js/JSWebAssemblyCodeBlock.h: * wasm/js/WebAssemblyCompileErrorConstructor.h: * wasm/js/WebAssemblyCompileErrorPrototype.h: * wasm/js/WebAssemblyFunction.h: * wasm/js/WebAssemblyInstanceConstructor.h: * wasm/js/WebAssemblyInstancePrototype.h: * wasm/js/WebAssemblyLinkErrorConstructor.h: * wasm/js/WebAssemblyLinkErrorPrototype.h: * wasm/js/WebAssemblyMemoryConstructor.h: * wasm/js/WebAssemblyMemoryPrototype.h: * wasm/js/WebAssemblyModuleConstructor.h: * wasm/js/WebAssemblyModulePrototype.h: * wasm/js/WebAssemblyRuntimeErrorConstructor.h: * wasm/js/WebAssemblyRuntimeErrorPrototype.h: * wasm/js/WebAssemblyTableConstructor.h: * wasm/js/WebAssemblyTablePrototype.h: * wasm/js/WebAssemblyToJSCallee.h: * yarr/Yarr.h: * yarr/YarrParser.h: * yarr/generateYarrCanonicalizeUnicode: Source/WebCore: No new tests. Covered by existing tests. * bindings/js/JSDOMConstructorBase.h: * bindings/js/JSDOMWindowProperties.h: * bindings/scripts/CodeGeneratorJS.pm: (GenerateHeader): (GeneratePrototypeDeclaration): * bindings/scripts/test/JS/JSTestActiveDOMObject.h: * bindings/scripts/test/JS/JSTestEnabledBySetting.h: * bindings/scripts/test/JS/JSTestEnabledForContext.h: * bindings/scripts/test/JS/JSTestEventTarget.h: * bindings/scripts/test/JS/JSTestGlobalObject.h: * bindings/scripts/test/JS/JSTestIndexedSetterNoIdentifier.h: * bindings/scripts/test/JS/JSTestIndexedSetterThrowingException.h: * bindings/scripts/test/JS/JSTestIndexedSetterWithIdentifier.h: * bindings/scripts/test/JS/JSTestNamedAndIndexedSetterNoIdentifier.h: * bindings/scripts/test/JS/JSTestNamedAndIndexedSetterThrowingException.h: * bindings/scripts/test/JS/JSTestNamedAndIndexedSetterWithIdentifier.h: * bindings/scripts/test/JS/JSTestNamedDeleterNoIdentifier.h: * bindings/scripts/test/JS/JSTestNamedDeleterThrowingException.h: * bindings/scripts/test/JS/JSTestNamedDeleterWithIdentifier.h: * bindings/scripts/test/JS/JSTestNamedDeleterWithIndexedGetter.h: * bindings/scripts/test/JS/JSTestNamedGetterCallWith.h: * bindings/scripts/test/JS/JSTestNamedGetterNoIdentifier.h: * bindings/scripts/test/JS/JSTestNamedGetterWithIdentifier.h: * bindings/scripts/test/JS/JSTestNamedSetterNoIdentifier.h: * bindings/scripts/test/JS/JSTestNamedSetterThrowingException.h: * bindings/scripts/test/JS/JSTestNamedSetterWithIdentifier.h: * bindings/scripts/test/JS/JSTestNamedSetterWithIndexedGetter.h: * bindings/scripts/test/JS/JSTestNamedSetterWithIndexedGetterAndSetter.h: * bindings/scripts/test/JS/JSTestNamedSetterWithOverrideBuiltins.h: * bindings/scripts/test/JS/JSTestNamedSetterWithUnforgableProperties.h: * bindings/scripts/test/JS/JSTestNamedSetterWithUnforgablePropertiesAndOverrideBuiltins.h: * bindings/scripts/test/JS/JSTestObj.h: * bindings/scripts/test/JS/JSTestOverrideBuiltins.h: * bindings/scripts/test/JS/JSTestPluginInterface.h: * bindings/scripts/test/JS/JSTestTypedefs.h: * bridge/objc/objc_runtime.h: * bridge/runtime_array.h: * bridge/runtime_method.h: * bridge/runtime_object.h: Source/WebKit: * WebProcess/Plugins/Netscape/JSNPObject.h: Source/WTF: * wtf/Assertions.cpp: * wtf/AutomaticThread.cpp: * wtf/BitVector.h: * wtf/Bitmap.h: * wtf/BloomFilter.h: * wtf/Brigand.h: * wtf/CheckedArithmetic.h: * wtf/CrossThreadCopier.h: * wtf/CurrentTime.cpp: * wtf/DataLog.cpp: * wtf/DateMath.cpp: (WTF::daysFrom1970ToYear): * wtf/DeferrableRefCounted.h: * wtf/GetPtr.h: * wtf/HashFunctions.h: * wtf/HashMap.h: * wtf/HashTable.h: * wtf/HashTraits.h: * wtf/JSONValues.cpp: * wtf/JSONValues.h: * wtf/ListHashSet.h: * wtf/Lock.h: * wtf/LockAlgorithm.h: * wtf/LockAlgorithmInlines.h: (WTF::Hooks>::lockSlow): * wtf/Logger.h: * wtf/LoggerHelper.h: (WTF::LoggerHelper::childLogIdentifier const): * wtf/MainThread.cpp: * wtf/MetaAllocatorPtr.h: * wtf/MonotonicTime.h: * wtf/NaturalLoops.h: (WTF::NaturalLoops::NaturalLoops): * wtf/ObjectIdentifier.h: * wtf/RAMSize.cpp: * wtf/Ref.h: * wtf/RefPtr.h: * wtf/RetainPtr.h: * wtf/SchedulePair.h: * wtf/StackShot.h: * wtf/StdLibExtras.h: * wtf/TinyPtrSet.h: * wtf/URL.cpp: * wtf/URLHash.h: * wtf/URLParser.cpp: (WTF::URLParser::defaultPortForProtocol): * wtf/Vector.h: * wtf/VectorTraits.h: * wtf/WallTime.h: * wtf/WeakHashSet.h: * wtf/WordLock.h: * wtf/cocoa/CPUTimeCocoa.cpp: * wtf/cocoa/MemoryPressureHandlerCocoa.mm: * wtf/persistence/PersistentDecoder.h: * wtf/persistence/PersistentEncoder.h: * wtf/text/AtomStringHash.h: * wtf/text/CString.h: * wtf/text/StringBuilder.cpp: (WTF::expandedCapacity): * wtf/text/StringHash.h: * wtf/text/StringImpl.h: * wtf/text/StringToIntegerConversion.h: (WTF::toIntegralType): * wtf/text/SymbolRegistry.h: * wtf/text/TextStream.cpp: (WTF::hasFractions): * wtf/text/WTFString.h: * wtf/text/cocoa/TextBreakIteratorInternalICUCocoa.cpp: Canonical link: https://commits.webkit.org/215538@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@250005 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2019-09-18 00:36:19 +00:00
* Copyright (C) 2013-2019 Apple Inc. All rights reserved.
FTL should be able to do some simple inline caches using LLVM patchpoints https://bugs.webkit.org/show_bug.cgi?id=123164 Source/JavaScriptCore: Reviewed by Mark Hahnenberg. This implements GetById inline caches in the FTL using llvm.webkit.patchpoint. The idea is that we ask LLVM for a nop slide the size of a GetById inline cache and then fill in the code after LLVM compilation is complete. For now, we just use the system calling convention for the arguments and return. We also still make some assumptions about registers that aren't correct. But, most of the scaffolding is there and this will successfully patch an inline cache. * JavaScriptCore.xcodeproj/project.pbxproj: * assembler/AbstractMacroAssembler.h: * assembler/LinkBuffer.cpp: (JSC::LinkBuffer::finalizeCodeWithoutDisassembly): (JSC::LinkBuffer::linkCode): (JSC::LinkBuffer::allocate): * assembler/LinkBuffer.h: (JSC::LinkBuffer::LinkBuffer): (JSC::LinkBuffer::link): * ftl/FTLAbbreviations.h: (JSC::FTL::constNull): (JSC::FTL::buildCall): * ftl/FTLCapabilities.cpp: (JSC::FTL::canCompile): * ftl/FTLCompile.cpp: (JSC::FTL::fixFunctionBasedOnStackMaps): * ftl/FTLInlineCacheDescriptor.h: Added. (JSC::FTL::InlineCacheDescriptor::InlineCacheDescriptor): (JSC::FTL::GetByIdDescriptor::GetByIdDescriptor): (JSC::FTL::GetByIdDescriptor::stackmapID): (JSC::FTL::GetByIdDescriptor::codeOrigin): (JSC::FTL::GetByIdDescriptor::uid): * ftl/FTLInlineCacheSize.cpp: Added. (JSC::FTL::sizeOfGetById): (JSC::FTL::sizeOfPutById): * ftl/FTLInlineCacheSize.h: Added. * ftl/FTLIntrinsicRepository.h: * ftl/FTLJITFinalizer.cpp: (JSC::FTL::JITFinalizer::finalizeFunction): * ftl/FTLJITFinalizer.h: * ftl/FTLLocation.cpp: (JSC::FTL::Location::directGPR): * ftl/FTLLocation.h: * ftl/FTLLowerDFGToLLVM.cpp: (JSC::FTL::LowerDFGToLLVM::compileGetById): * ftl/FTLOutput.h: (JSC::FTL::Output::call): * ftl/FTLSlowPathCall.cpp: Added. (JSC::FTL::callOperation): * ftl/FTLSlowPathCall.h: Added. (JSC::FTL::SlowPathCall::SlowPathCall): (JSC::FTL::SlowPathCall::call): (JSC::FTL::SlowPathCall::key): * ftl/FTLSlowPathCallKey.cpp: Added. (JSC::FTL::SlowPathCallKey::dump): * ftl/FTLSlowPathCallKey.h: Added. (JSC::FTL::SlowPathCallKey::SlowPathCallKey): (JSC::FTL::SlowPathCallKey::usedRegisters): (JSC::FTL::SlowPathCallKey::callTarget): (JSC::FTL::SlowPathCallKey::offset): (JSC::FTL::SlowPathCallKey::isEmptyValue): (JSC::FTL::SlowPathCallKey::isDeletedValue): (JSC::FTL::SlowPathCallKey::operator==): (JSC::FTL::SlowPathCallKey::hash): (JSC::FTL::SlowPathCallKeyHash::hash): (JSC::FTL::SlowPathCallKeyHash::equal): * ftl/FTLStackMaps.cpp: (JSC::FTL::StackMaps::Location::directGPR): * ftl/FTLStackMaps.h: * ftl/FTLState.h: * ftl/FTLThunks.cpp: (JSC::FTL::slowPathCallThunkGenerator): * ftl/FTLThunks.h: (JSC::FTL::Thunks::getSlowPathCallThunk): * jit/CCallHelpers.h: (JSC::CCallHelpers::setupArguments): * jit/GPRInfo.h: * jit/JITInlineCacheGenerator.cpp: (JSC::garbageStubInfo): (JSC::JITInlineCacheGenerator::JITInlineCacheGenerator): (JSC::JITByIdGenerator::finalize): * jit/JITInlineCacheGenerator.h: (JSC::JITByIdGenerator::slowPathBegin): * jit/RegisterSet.cpp: (JSC::RegisterSet::stackRegisters): (JSC::RegisterSet::specialRegisters): (JSC::RegisterSet::calleeSaveRegisters): (JSC::RegisterSet::allGPRs): (JSC::RegisterSet::allFPRs): (JSC::RegisterSet::allRegisters): (JSC::RegisterSet::dump): * jit/RegisterSet.h: (JSC::RegisterSet::exclude): (JSC::RegisterSet::numberOfSetRegisters): (JSC::RegisterSet::RegisterSet): (JSC::RegisterSet::isEmptyValue): (JSC::RegisterSet::isDeletedValue): (JSC::RegisterSet::operator==): (JSC::RegisterSet::hash): (JSC::RegisterSetHash::hash): (JSC::RegisterSetHash::equal): * runtime/Options.h: Source/WTF: Reviewed by Mark Hahnenberg. This needed some better bitvector support, like merging (|=), excluding (&=~), hashing, and bit counting. * wtf/BitVector.cpp: (WTF::BitVector::setSlow): (WTF::BitVector::excludeSlow): (WTF::BitVector::bitCountSlow): (WTF::BitVector::equalsSlowCase): (WTF::BitVector::hashSlowCase): (WTF::BitVector::dump): * wtf/BitVector.h: (WTF::BitVector::merge): (WTF::BitVector::exclude): (WTF::BitVector::bitCount): (WTF::BitVector::BitVector): (WTF::BitVector::isEmptyValue): (WTF::BitVector::isDeletedValue): (WTF::BitVector::isEmptyOrDeletedValue): (WTF::BitVector::operator==): (WTF::BitVector::hash): (WTF::BitVectorHash::hash): (WTF::BitVectorHash::equal): * wtf/HashTraits.h: (WTF::CustomHashTraits::constructDeletedValue): (WTF::CustomHashTraits::isDeletedValue): (WTF::CustomHashTraits::emptyValue): (WTF::CustomHashTraits::isEmptyValue): * wtf/StdLibExtras.h: (WTF::bitCount): Canonical link: https://commits.webkit.org/141315@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@157872 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2013-10-23 18:22:09 +00:00
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
* are met:
* 1. Redistributions of source code must retain the above copyright
* notice, this list of conditions and the following disclaimer.
* 2. Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
*
* THIS SOFTWARE IS PROVIDED BY APPLE INC. ``AS IS'' AND ANY
* EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
* PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR
* CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
* EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
* PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
* PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
* OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
* (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
* OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
#include "config.h"
#include "FTLSlowPathCall.h"
#if ENABLE(FTL_JIT)
#include "CCallHelpers.h"
#include "FTLState.h"
FTL should generate code to call slow paths lazily https://bugs.webkit.org/show_bug.cgi?id=149936 Reviewed by Saam Barati. Source/JavaScriptCore: We often have complex slow paths in FTL-generated code. Those slow paths may never run. Even if they do run, they don't need stellar performance. So, it doesn't make sense to have LLVM worry about compiling such slow path code. This patch enables us to use our own MacroAssembler for compiling the slow path inside FTL code. It does this by using a crazy lambda thingy (see FTLLowerDFGToLLVM.cpp's lazySlowPath() and its documentation). The result is quite natural to use. Even for straight slow path calls via something like vmCall(), the lazySlowPath offers the benefit that the call marshalling and the exception checking are not expressed using LLVM IR and do not require LLVM to think about it. It also has the benefit that we never generate the code if it never runs. That's great, since function calls usually involve ~10 instructions total (move arguments to argument registers, make the call, check exception, etc.). This patch adds the lazy slow path abstraction and uses it for some slow paths in the FTL. The code we generate with lazy slow paths is worse than the code that LLVM would have generated. Therefore, a lazy slow path only makes sense when we have strong evidence that the slow path will execute infrequently relative to the fast path. This completely precludes the use of lazy slow paths for out-of-line Nodes that unconditionally call a C++ function. It also precludes their use for the GetByVal out-of-bounds handler, since when we generate a GetByVal with an out-of-bounds handler it means that we only know that the out-of-bounds case executed at least once. So, for all we know, it may actually be the common case. So, this patch just deployed the lazy slow path for GC slow paths and masquerades-as-undefined slow paths. It makes sense for GC slow paths because those have a statistical guarantee of slow path frequency - probably bounded at less than 1/10. It makes sense for masquerades-as- undefined because we can say quite confidently that this is an uncommon scenario on the modern Web. Something that's always been challenging about abstractions involving the MacroAssembler is that linking is a separate phase, and there is no way for someone who is just given access to the MacroAssembler& to emit code that requires linking, since linking happens once we have emitted all code and we are creating the LinkBuffer. Moreover, the FTL requires that the final parts of linking happen on the main thread. This patch ran into this issue, and solved it comprehensively, by introducing MacroAssembler::addLinkTask(). This takes a lambda and runs it at the bitter end of linking - when performFinalization() is called. This ensure that the task added by addLinkTask() runs on the main thread. This patch doesn't replace all of the previously existing idioms for dealing with this issue; we can do that later. This shows small speed-ups on a bunch of things. No big win on any benchmark aggregate. But mainly this is done for https://bugs.webkit.org/show_bug.cgi?id=149852, where we found that outlining the slow path in this way was a significant speed boost. * CMakeLists.txt: * JavaScriptCore.vcxproj/JavaScriptCore.vcxproj: * JavaScriptCore.xcodeproj/project.pbxproj: * assembler/AbstractMacroAssembler.h: (JSC::AbstractMacroAssembler::replaceWithAddressComputation): (JSC::AbstractMacroAssembler::addLinkTask): (JSC::AbstractMacroAssembler::AbstractMacroAssembler): * assembler/LinkBuffer.cpp: (JSC::LinkBuffer::linkCode): (JSC::LinkBuffer::allocate): (JSC::LinkBuffer::performFinalization): * assembler/LinkBuffer.h: (JSC::LinkBuffer::wasAlreadyDisassembled): (JSC::LinkBuffer::didAlreadyDisassemble): (JSC::LinkBuffer::vm): (JSC::LinkBuffer::executableOffsetFor): * bytecode/CodeOrigin.h: (JSC::CodeOrigin::CodeOrigin): (JSC::CodeOrigin::isSet): (JSC::CodeOrigin::operator bool): (JSC::CodeOrigin::isHashTableDeletedValue): (JSC::CodeOrigin::operator!): Deleted. * ftl/FTLCompile.cpp: (JSC::FTL::mmAllocateDataSection): * ftl/FTLInlineCacheDescriptor.h: (JSC::FTL::InlineCacheDescriptor::InlineCacheDescriptor): (JSC::FTL::CheckInDescriptor::CheckInDescriptor): (JSC::FTL::LazySlowPathDescriptor::LazySlowPathDescriptor): * ftl/FTLJITCode.h: * ftl/FTLJITFinalizer.cpp: (JSC::FTL::JITFinalizer::finalizeFunction): * ftl/FTLJITFinalizer.h: * ftl/FTLLazySlowPath.cpp: Added. (JSC::FTL::LazySlowPath::LazySlowPath): (JSC::FTL::LazySlowPath::~LazySlowPath): (JSC::FTL::LazySlowPath::generate): * ftl/FTLLazySlowPath.h: Added. (JSC::FTL::LazySlowPath::createGenerator): (JSC::FTL::LazySlowPath::patchpoint): (JSC::FTL::LazySlowPath::usedRegisters): (JSC::FTL::LazySlowPath::callSiteIndex): (JSC::FTL::LazySlowPath::stub): * ftl/FTLLazySlowPathCall.h: Added. (JSC::FTL::createLazyCallGenerator): * ftl/FTLLowerDFGToLLVM.cpp: (JSC::FTL::DFG::LowerDFGToLLVM::compileCreateActivation): (JSC::FTL::DFG::LowerDFGToLLVM::compileNewFunction): (JSC::FTL::DFG::LowerDFGToLLVM::compileCreateDirectArguments): (JSC::FTL::DFG::LowerDFGToLLVM::compileNewArrayWithSize): (JSC::FTL::DFG::LowerDFGToLLVM::compileMakeRope): (JSC::FTL::DFG::LowerDFGToLLVM::compileNotifyWrite): (JSC::FTL::DFG::LowerDFGToLLVM::compileIsObjectOrNull): (JSC::FTL::DFG::LowerDFGToLLVM::compileIsFunction): (JSC::FTL::DFG::LowerDFGToLLVM::compileIn): (JSC::FTL::DFG::LowerDFGToLLVM::compileMaterializeNewObject): (JSC::FTL::DFG::LowerDFGToLLVM::compileMaterializeCreateActivation): (JSC::FTL::DFG::LowerDFGToLLVM::compileCheckWatchdogTimer): (JSC::FTL::DFG::LowerDFGToLLVM::allocatePropertyStorageWithSizeImpl): (JSC::FTL::DFG::LowerDFGToLLVM::allocateObject): (JSC::FTL::DFG::LowerDFGToLLVM::allocateJSArray): (JSC::FTL::DFG::LowerDFGToLLVM::buildTypeOf): (JSC::FTL::DFG::LowerDFGToLLVM::sensibleDoubleToInt32): (JSC::FTL::DFG::LowerDFGToLLVM::lazySlowPath): (JSC::FTL::DFG::LowerDFGToLLVM::speculate): (JSC::FTL::DFG::LowerDFGToLLVM::emitStoreBarrier): * ftl/FTLOperations.cpp: (JSC::FTL::operationMaterializeObjectInOSR): (JSC::FTL::compileFTLLazySlowPath): * ftl/FTLOperations.h: * ftl/FTLSlowPathCall.cpp: (JSC::FTL::SlowPathCallContext::SlowPathCallContext): (JSC::FTL::SlowPathCallContext::~SlowPathCallContext): (JSC::FTL::SlowPathCallContext::keyWithTarget): (JSC::FTL::SlowPathCallContext::makeCall): (JSC::FTL::callSiteIndexForCodeOrigin): (JSC::FTL::storeCodeOrigin): Deleted. (JSC::FTL::callOperation): Deleted. * ftl/FTLSlowPathCall.h: (JSC::FTL::callOperation): * ftl/FTLState.h: * ftl/FTLThunks.cpp: (JSC::FTL::genericGenerationThunkGenerator): (JSC::FTL::osrExitGenerationThunkGenerator): (JSC::FTL::lazySlowPathGenerationThunkGenerator): (JSC::FTL::registerClobberCheck): * ftl/FTLThunks.h: * interpreter/CallFrame.h: (JSC::CallSiteIndex::CallSiteIndex): (JSC::CallSiteIndex::operator bool): (JSC::CallSiteIndex::bits): * jit/CCallHelpers.h: (JSC::CCallHelpers::setupArgument): (JSC::CCallHelpers::setupArgumentsWithExecState): * jit/JITOperations.cpp: Source/WTF: Enables SharedTask to handle any function type, not just void(). It's probably better to use SharedTask instead of std::function in performance-sensitive code. std::function uses the system malloc and has copy semantics. SharedTask uses FastMalloc and has aliasing semantics. So, you can just trust that it will have sensible performance characteristics. * wtf/ParallelHelperPool.cpp: (WTF::ParallelHelperClient::~ParallelHelperClient): (WTF::ParallelHelperClient::setTask): (WTF::ParallelHelperClient::doSomeHelping): (WTF::ParallelHelperClient::runTaskInParallel): (WTF::ParallelHelperClient::finish): (WTF::ParallelHelperClient::claimTask): (WTF::ParallelHelperClient::runTask): (WTF::ParallelHelperPool::doSomeHelping): (WTF::ParallelHelperPool::helperThreadBody): * wtf/ParallelHelperPool.h: (WTF::ParallelHelperClient::setFunction): (WTF::ParallelHelperClient::runFunctionInParallel): (WTF::ParallelHelperClient::pool): * wtf/SharedTask.h: (WTF::createSharedTask): (WTF::SharedTask::SharedTask): Deleted. (WTF::SharedTask::~SharedTask): Deleted. (WTF::SharedTaskFunctor::SharedTaskFunctor): Deleted. Canonical link: https://commits.webkit.org/168154@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@190860 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2015-10-12 17:56:26 +00:00
#include "FTLThunks.h"
FTL should be able to do some simple inline caches using LLVM patchpoints https://bugs.webkit.org/show_bug.cgi?id=123164 Source/JavaScriptCore: Reviewed by Mark Hahnenberg. This implements GetById inline caches in the FTL using llvm.webkit.patchpoint. The idea is that we ask LLVM for a nop slide the size of a GetById inline cache and then fill in the code after LLVM compilation is complete. For now, we just use the system calling convention for the arguments and return. We also still make some assumptions about registers that aren't correct. But, most of the scaffolding is there and this will successfully patch an inline cache. * JavaScriptCore.xcodeproj/project.pbxproj: * assembler/AbstractMacroAssembler.h: * assembler/LinkBuffer.cpp: (JSC::LinkBuffer::finalizeCodeWithoutDisassembly): (JSC::LinkBuffer::linkCode): (JSC::LinkBuffer::allocate): * assembler/LinkBuffer.h: (JSC::LinkBuffer::LinkBuffer): (JSC::LinkBuffer::link): * ftl/FTLAbbreviations.h: (JSC::FTL::constNull): (JSC::FTL::buildCall): * ftl/FTLCapabilities.cpp: (JSC::FTL::canCompile): * ftl/FTLCompile.cpp: (JSC::FTL::fixFunctionBasedOnStackMaps): * ftl/FTLInlineCacheDescriptor.h: Added. (JSC::FTL::InlineCacheDescriptor::InlineCacheDescriptor): (JSC::FTL::GetByIdDescriptor::GetByIdDescriptor): (JSC::FTL::GetByIdDescriptor::stackmapID): (JSC::FTL::GetByIdDescriptor::codeOrigin): (JSC::FTL::GetByIdDescriptor::uid): * ftl/FTLInlineCacheSize.cpp: Added. (JSC::FTL::sizeOfGetById): (JSC::FTL::sizeOfPutById): * ftl/FTLInlineCacheSize.h: Added. * ftl/FTLIntrinsicRepository.h: * ftl/FTLJITFinalizer.cpp: (JSC::FTL::JITFinalizer::finalizeFunction): * ftl/FTLJITFinalizer.h: * ftl/FTLLocation.cpp: (JSC::FTL::Location::directGPR): * ftl/FTLLocation.h: * ftl/FTLLowerDFGToLLVM.cpp: (JSC::FTL::LowerDFGToLLVM::compileGetById): * ftl/FTLOutput.h: (JSC::FTL::Output::call): * ftl/FTLSlowPathCall.cpp: Added. (JSC::FTL::callOperation): * ftl/FTLSlowPathCall.h: Added. (JSC::FTL::SlowPathCall::SlowPathCall): (JSC::FTL::SlowPathCall::call): (JSC::FTL::SlowPathCall::key): * ftl/FTLSlowPathCallKey.cpp: Added. (JSC::FTL::SlowPathCallKey::dump): * ftl/FTLSlowPathCallKey.h: Added. (JSC::FTL::SlowPathCallKey::SlowPathCallKey): (JSC::FTL::SlowPathCallKey::usedRegisters): (JSC::FTL::SlowPathCallKey::callTarget): (JSC::FTL::SlowPathCallKey::offset): (JSC::FTL::SlowPathCallKey::isEmptyValue): (JSC::FTL::SlowPathCallKey::isDeletedValue): (JSC::FTL::SlowPathCallKey::operator==): (JSC::FTL::SlowPathCallKey::hash): (JSC::FTL::SlowPathCallKeyHash::hash): (JSC::FTL::SlowPathCallKeyHash::equal): * ftl/FTLStackMaps.cpp: (JSC::FTL::StackMaps::Location::directGPR): * ftl/FTLStackMaps.h: * ftl/FTLState.h: * ftl/FTLThunks.cpp: (JSC::FTL::slowPathCallThunkGenerator): * ftl/FTLThunks.h: (JSC::FTL::Thunks::getSlowPathCallThunk): * jit/CCallHelpers.h: (JSC::CCallHelpers::setupArguments): * jit/GPRInfo.h: * jit/JITInlineCacheGenerator.cpp: (JSC::garbageStubInfo): (JSC::JITInlineCacheGenerator::JITInlineCacheGenerator): (JSC::JITByIdGenerator::finalize): * jit/JITInlineCacheGenerator.h: (JSC::JITByIdGenerator::slowPathBegin): * jit/RegisterSet.cpp: (JSC::RegisterSet::stackRegisters): (JSC::RegisterSet::specialRegisters): (JSC::RegisterSet::calleeSaveRegisters): (JSC::RegisterSet::allGPRs): (JSC::RegisterSet::allFPRs): (JSC::RegisterSet::allRegisters): (JSC::RegisterSet::dump): * jit/RegisterSet.h: (JSC::RegisterSet::exclude): (JSC::RegisterSet::numberOfSetRegisters): (JSC::RegisterSet::RegisterSet): (JSC::RegisterSet::isEmptyValue): (JSC::RegisterSet::isDeletedValue): (JSC::RegisterSet::operator==): (JSC::RegisterSet::hash): (JSC::RegisterSetHash::hash): (JSC::RegisterSetHash::equal): * runtime/Options.h: Source/WTF: Reviewed by Mark Hahnenberg. This needed some better bitvector support, like merging (|=), excluding (&=~), hashing, and bit counting. * wtf/BitVector.cpp: (WTF::BitVector::setSlow): (WTF::BitVector::excludeSlow): (WTF::BitVector::bitCountSlow): (WTF::BitVector::equalsSlowCase): (WTF::BitVector::hashSlowCase): (WTF::BitVector::dump): * wtf/BitVector.h: (WTF::BitVector::merge): (WTF::BitVector::exclude): (WTF::BitVector::bitCount): (WTF::BitVector::BitVector): (WTF::BitVector::isEmptyValue): (WTF::BitVector::isDeletedValue): (WTF::BitVector::isEmptyOrDeletedValue): (WTF::BitVector::operator==): (WTF::BitVector::hash): (WTF::BitVectorHash::hash): (WTF::BitVectorHash::equal): * wtf/HashTraits.h: (WTF::CustomHashTraits::constructDeletedValue): (WTF::CustomHashTraits::isDeletedValue): (WTF::CustomHashTraits::emptyValue): (WTF::CustomHashTraits::isEmptyValue): * wtf/StdLibExtras.h: (WTF::bitCount): Canonical link: https://commits.webkit.org/141315@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@157872 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2013-10-23 18:22:09 +00:00
#include "GPRInfo.h"
namespace JSC { namespace FTL {
// This code relies on us being 64-bit. FTL is currently always 64-bit.
Use constexpr instead of const in symbol definitions that are obviously constexpr. https://bugs.webkit.org/show_bug.cgi?id=201879 Rubber-stamped by Joseph Pecoraro. Source/bmalloc: * bmalloc/AvailableMemory.cpp: * bmalloc/IsoTLS.h: * bmalloc/Map.h: * bmalloc/Mutex.cpp: (bmalloc::Mutex::lockSlowCase): * bmalloc/PerThread.h: * bmalloc/Vector.h: * bmalloc/Zone.h: Source/JavaScriptCore: const may require external storage (at the compiler's whim) though these currently do not. constexpr makes it clear that the value is a literal constant that can be inlined. In most cases in the code, when we say static const, we actually mean static constexpr. I'm changing the code to reflect this. * API/JSAPIValueWrapper.h: * API/JSCallbackConstructor.h: * API/JSCallbackObject.h: * API/JSContextRef.cpp: * API/JSWrapperMap.mm: * API/tests/CompareAndSwapTest.cpp: * API/tests/TypedArrayCTest.cpp: * API/tests/testapi.mm: (testObjectiveCAPIMain): * KeywordLookupGenerator.py: (Trie.printAsC): * assembler/ARMv7Assembler.h: * assembler/AssemblerBuffer.h: * assembler/AssemblerCommon.h: * assembler/MacroAssembler.h: * assembler/MacroAssemblerARM64.h: * assembler/MacroAssemblerARM64E.h: * assembler/MacroAssemblerARMv7.h: * assembler/MacroAssemblerCodeRef.h: * assembler/MacroAssemblerMIPS.h: * assembler/MacroAssemblerX86.h: * assembler/MacroAssemblerX86Common.h: (JSC::MacroAssemblerX86Common::absDouble): (JSC::MacroAssemblerX86Common::negateDouble): * assembler/MacroAssemblerX86_64.h: * assembler/X86Assembler.h: * b3/B3Bank.h: * b3/B3CheckSpecial.h: * b3/B3DuplicateTails.cpp: * b3/B3EliminateCommonSubexpressions.cpp: * b3/B3FixSSA.cpp: * b3/B3FoldPathConstants.cpp: * b3/B3InferSwitches.cpp: * b3/B3Kind.h: * b3/B3LowerToAir.cpp: * b3/B3NativeTraits.h: * b3/B3ReduceDoubleToFloat.cpp: * b3/B3ReduceLoopStrength.cpp: * b3/B3ReduceStrength.cpp: * b3/B3ValueKey.h: * b3/air/AirAllocateRegistersByGraphColoring.cpp: * b3/air/AirAllocateStackByGraphColoring.cpp: * b3/air/AirArg.h: * b3/air/AirCCallSpecial.h: * b3/air/AirEmitShuffle.cpp: * b3/air/AirFixObviousSpills.cpp: * b3/air/AirFormTable.h: * b3/air/AirLowerAfterRegAlloc.cpp: * b3/air/AirPrintSpecial.h: * b3/air/AirStackAllocation.cpp: * b3/air/AirTmp.h: * b3/testb3_6.cpp: (testInterpreter): * bytecode/AccessCase.cpp: * bytecode/CallLinkStatus.cpp: * bytecode/CallVariant.h: * bytecode/CodeBlock.h: * bytecode/CodeOrigin.h: * bytecode/DFGExitProfile.h: * bytecode/DirectEvalCodeCache.h: * bytecode/ExecutableToCodeBlockEdge.h: * bytecode/GetterSetterAccessCase.cpp: * bytecode/LazyOperandValueProfile.h: * bytecode/ObjectPropertyCondition.h: * bytecode/ObjectPropertyConditionSet.cpp: * bytecode/PolymorphicAccess.cpp: * bytecode/PropertyCondition.h: * bytecode/SpeculatedType.h: * bytecode/StructureStubInfo.cpp: * bytecode/UnlinkedCodeBlock.cpp: (JSC::UnlinkedCodeBlock::typeProfilerExpressionInfoForBytecodeOffset): * bytecode/UnlinkedCodeBlock.h: * bytecode/UnlinkedEvalCodeBlock.h: * bytecode/UnlinkedFunctionCodeBlock.h: * bytecode/UnlinkedFunctionExecutable.h: * bytecode/UnlinkedModuleProgramCodeBlock.h: * bytecode/UnlinkedProgramCodeBlock.h: * bytecode/ValueProfile.h: * bytecode/VirtualRegister.h: * bytecode/Watchpoint.h: * bytecompiler/BytecodeGenerator.h: * bytecompiler/Label.h: * bytecompiler/NodesCodegen.cpp: (JSC::ThisNode::emitBytecode): * bytecompiler/RegisterID.h: * debugger/Breakpoint.h: * debugger/DebuggerParseData.cpp: * debugger/DebuggerPrimitives.h: * debugger/DebuggerScope.h: * dfg/DFGAbstractHeap.h: * dfg/DFGAbstractValue.h: * dfg/DFGArgumentsEliminationPhase.cpp: * dfg/DFGByteCodeParser.cpp: * dfg/DFGCSEPhase.cpp: * dfg/DFGCommon.h: * dfg/DFGCompilationKey.h: * dfg/DFGDesiredGlobalProperty.h: * dfg/DFGEdgeDominates.h: * dfg/DFGEpoch.h: * dfg/DFGForAllKills.h: (JSC::DFG::forAllKilledNodesAtNodeIndex): * dfg/DFGGraph.cpp: (JSC::DFG::Graph::isLiveInBytecode): * dfg/DFGHeapLocation.h: * dfg/DFGInPlaceAbstractState.cpp: * dfg/DFGIntegerCheckCombiningPhase.cpp: * dfg/DFGIntegerRangeOptimizationPhase.cpp: * dfg/DFGInvalidationPointInjectionPhase.cpp: * dfg/DFGLICMPhase.cpp: * dfg/DFGLazyNode.h: * dfg/DFGMinifiedID.h: * dfg/DFGMovHintRemovalPhase.cpp: * dfg/DFGNodeFlowProjection.h: * dfg/DFGNodeType.h: * dfg/DFGObjectAllocationSinkingPhase.cpp: * dfg/DFGPhantomInsertionPhase.cpp: * dfg/DFGPromotedHeapLocation.h: * dfg/DFGPropertyTypeKey.h: * dfg/DFGPureValue.h: * dfg/DFGPutStackSinkingPhase.cpp: * dfg/DFGRegisterBank.h: * dfg/DFGSSAConversionPhase.cpp: * dfg/DFGSSALoweringPhase.cpp: * dfg/DFGSpeculativeJIT.cpp: (JSC::DFG::SpeculativeJIT::compileDoubleRep): (JSC::DFG::compileClampDoubleToByte): (JSC::DFG::SpeculativeJIT::compileArithRounding): (JSC::DFG::compileArithPowIntegerFastPath): (JSC::DFG::SpeculativeJIT::compileArithPow): (JSC::DFG::SpeculativeJIT::emitBinarySwitchStringRecurse): * dfg/DFGStackLayoutPhase.cpp: * dfg/DFGStoreBarrierInsertionPhase.cpp: * dfg/DFGStrengthReductionPhase.cpp: * dfg/DFGStructureAbstractValue.h: * dfg/DFGVarargsForwardingPhase.cpp: * dfg/DFGVariableEventStream.cpp: (JSC::DFG::VariableEventStream::reconstruct const): * dfg/DFGWatchpointCollectionPhase.cpp: * disassembler/ARM64/A64DOpcode.h: * ftl/FTLLocation.h: * ftl/FTLLowerDFGToB3.cpp: (JSC::FTL::DFG::LowerDFGToB3::compileArithRandom): * ftl/FTLSlowPathCall.cpp: * ftl/FTLSlowPathCallKey.h: * heap/CellContainer.h: * heap/CellState.h: * heap/ConservativeRoots.h: * heap/GCSegmentedArray.h: * heap/HandleBlock.h: * heap/Heap.cpp: (JSC::Heap::updateAllocationLimits): * heap/Heap.h: * heap/HeapSnapshot.h: * heap/HeapUtil.h: (JSC::HeapUtil::findGCObjectPointersForMarking): * heap/IncrementalSweeper.cpp: * heap/LargeAllocation.h: * heap/MarkedBlock.cpp: * heap/Strong.h: * heap/VisitRaceKey.h: * heap/Weak.h: * heap/WeakBlock.h: * inspector/JSInjectedScriptHost.h: * inspector/JSInjectedScriptHostPrototype.h: * inspector/JSJavaScriptCallFrame.h: * inspector/JSJavaScriptCallFramePrototype.h: * inspector/agents/InspectorConsoleAgent.cpp: * inspector/agents/InspectorRuntimeAgent.cpp: (Inspector::InspectorRuntimeAgent::getRuntimeTypesForVariablesAtOffsets): * inspector/scripts/codegen/generate_cpp_protocol_types_header.py: (CppProtocolTypesHeaderGenerator._generate_versions): * inspector/scripts/tests/generic/expected/version.json-result: * interpreter/Interpreter.h: * interpreter/ShadowChicken.cpp: * jit/BinarySwitch.cpp: * jit/CallFrameShuffler.h: * jit/ExecutableAllocator.h: * jit/FPRInfo.h: * jit/GPRInfo.h: * jit/ICStats.h: * jit/JITThunks.h: * jit/Reg.h: * jit/RegisterSet.h: * jit/TempRegisterSet.h: * jsc.cpp: * parser/ASTBuilder.h: * parser/Nodes.h: * parser/SourceCodeKey.h: * parser/SyntaxChecker.h: * parser/VariableEnvironment.h: * profiler/ProfilerOrigin.h: * profiler/ProfilerOriginStack.h: * profiler/ProfilerUID.h: * runtime/AbstractModuleRecord.cpp: * runtime/ArrayBufferNeuteringWatchpointSet.h: * runtime/ArrayConstructor.h: * runtime/ArrayConventions.h: * runtime/ArrayIteratorPrototype.h: * runtime/ArrayPrototype.cpp: (JSC::setLength): * runtime/AsyncFromSyncIteratorPrototype.h: * runtime/AsyncGeneratorFunctionPrototype.h: * runtime/AsyncGeneratorPrototype.h: * runtime/AsyncIteratorPrototype.h: * runtime/AtomicsObject.cpp: * runtime/BigIntConstructor.h: * runtime/BigIntPrototype.h: * runtime/BooleanPrototype.h: * runtime/ClonedArguments.h: * runtime/CodeCache.h: * runtime/ControlFlowProfiler.h: * runtime/CustomGetterSetter.h: * runtime/DateConstructor.h: * runtime/DatePrototype.h: * runtime/DefinePropertyAttributes.h: * runtime/ErrorPrototype.h: * runtime/EvalExecutable.h: * runtime/Exception.h: * runtime/ExceptionHelpers.cpp: (JSC::invalidParameterInSourceAppender): (JSC::invalidParameterInstanceofSourceAppender): * runtime/ExceptionHelpers.h: * runtime/ExecutableBase.h: * runtime/FunctionExecutable.h: * runtime/FunctionRareData.h: * runtime/GeneratorPrototype.h: * runtime/GenericArguments.h: * runtime/GenericOffset.h: * runtime/GetPutInfo.h: * runtime/GetterSetter.h: * runtime/GlobalExecutable.h: * runtime/Identifier.h: * runtime/InspectorInstrumentationObject.h: * runtime/InternalFunction.h: * runtime/IntlCollatorConstructor.h: * runtime/IntlCollatorPrototype.h: * runtime/IntlDateTimeFormatConstructor.h: * runtime/IntlDateTimeFormatPrototype.h: * runtime/IntlNumberFormatConstructor.h: * runtime/IntlNumberFormatPrototype.h: * runtime/IntlObject.h: * runtime/IntlPluralRulesConstructor.h: * runtime/IntlPluralRulesPrototype.h: * runtime/IteratorPrototype.h: * runtime/JSArray.cpp: (JSC::JSArray::tryCreateUninitializedRestricted): * runtime/JSArray.h: * runtime/JSArrayBuffer.h: * runtime/JSArrayBufferView.h: * runtime/JSBigInt.h: * runtime/JSCJSValue.h: * runtime/JSCell.h: * runtime/JSCustomGetterSetterFunction.h: * runtime/JSDataView.h: * runtime/JSDataViewPrototype.h: * runtime/JSDestructibleObject.h: * runtime/JSFixedArray.h: * runtime/JSGenericTypedArrayView.h: * runtime/JSGlobalLexicalEnvironment.h: * runtime/JSGlobalObject.h: * runtime/JSImmutableButterfly.h: * runtime/JSInternalPromiseConstructor.h: * runtime/JSInternalPromiseDeferred.h: * runtime/JSInternalPromisePrototype.h: * runtime/JSLexicalEnvironment.h: * runtime/JSModuleEnvironment.h: * runtime/JSModuleLoader.h: * runtime/JSModuleNamespaceObject.h: * runtime/JSNonDestructibleProxy.h: * runtime/JSONObject.cpp: * runtime/JSONObject.h: * runtime/JSObject.h: * runtime/JSPromiseConstructor.h: * runtime/JSPromiseDeferred.h: * runtime/JSPromisePrototype.h: * runtime/JSPropertyNameEnumerator.h: * runtime/JSProxy.h: * runtime/JSScope.h: * runtime/JSScriptFetchParameters.h: * runtime/JSScriptFetcher.h: * runtime/JSSegmentedVariableObject.h: * runtime/JSSourceCode.h: * runtime/JSString.cpp: * runtime/JSString.h: * runtime/JSSymbolTableObject.h: * runtime/JSTemplateObjectDescriptor.h: * runtime/JSTypeInfo.h: * runtime/MapPrototype.h: * runtime/MinimumReservedZoneSize.h: * runtime/ModuleProgramExecutable.h: * runtime/NativeExecutable.h: * runtime/NativeFunction.h: * runtime/NativeStdFunctionCell.h: * runtime/NumberConstructor.h: * runtime/NumberPrototype.h: * runtime/ObjectConstructor.h: * runtime/ObjectPrototype.h: * runtime/ProgramExecutable.h: * runtime/PromiseDeferredTimer.cpp: * runtime/PropertyMapHashTable.h: * runtime/PropertyNameArray.h: (JSC::PropertyNameArray::add): * runtime/PrototypeKey.h: * runtime/ProxyConstructor.h: * runtime/ProxyObject.cpp: (JSC::ProxyObject::performGetOwnPropertyNames): * runtime/ProxyRevoke.h: * runtime/ReflectObject.h: * runtime/RegExp.h: * runtime/RegExpCache.h: * runtime/RegExpConstructor.h: * runtime/RegExpKey.h: * runtime/RegExpObject.h: * runtime/RegExpPrototype.h: * runtime/RegExpStringIteratorPrototype.h: * runtime/SamplingProfiler.cpp: * runtime/ScopedArgumentsTable.h: * runtime/ScriptExecutable.h: * runtime/SetPrototype.h: * runtime/SmallStrings.h: * runtime/SparseArrayValueMap.h: * runtime/StringConstructor.h: * runtime/StringIteratorPrototype.h: * runtime/StringObject.h: * runtime/StringPrototype.h: * runtime/Structure.h: * runtime/StructureChain.h: * runtime/StructureRareData.h: * runtime/StructureTransitionTable.h: * runtime/Symbol.h: * runtime/SymbolConstructor.h: * runtime/SymbolPrototype.h: * runtime/SymbolTable.h: * runtime/TemplateObjectDescriptor.h: * runtime/TypeProfiler.cpp: * runtime/TypeProfiler.h: * runtime/TypeProfilerLog.cpp: * runtime/VarOffset.h: * testRegExp.cpp: * tools/HeapVerifier.cpp: (JSC::HeapVerifier::checkIfRecorded): * tools/JSDollarVM.cpp: * wasm/WasmB3IRGenerator.cpp: * wasm/WasmBBQPlan.cpp: * wasm/WasmFaultSignalHandler.cpp: * wasm/WasmFunctionParser.h: * wasm/WasmOMGForOSREntryPlan.cpp: * wasm/WasmOMGPlan.cpp: * wasm/WasmPlan.cpp: * wasm/WasmSignature.cpp: * wasm/WasmSignature.h: * wasm/WasmWorklist.cpp: * wasm/js/JSWebAssembly.h: * wasm/js/JSWebAssemblyCodeBlock.h: * wasm/js/WebAssemblyCompileErrorConstructor.h: * wasm/js/WebAssemblyCompileErrorPrototype.h: * wasm/js/WebAssemblyFunction.h: * wasm/js/WebAssemblyInstanceConstructor.h: * wasm/js/WebAssemblyInstancePrototype.h: * wasm/js/WebAssemblyLinkErrorConstructor.h: * wasm/js/WebAssemblyLinkErrorPrototype.h: * wasm/js/WebAssemblyMemoryConstructor.h: * wasm/js/WebAssemblyMemoryPrototype.h: * wasm/js/WebAssemblyModuleConstructor.h: * wasm/js/WebAssemblyModulePrototype.h: * wasm/js/WebAssemblyRuntimeErrorConstructor.h: * wasm/js/WebAssemblyRuntimeErrorPrototype.h: * wasm/js/WebAssemblyTableConstructor.h: * wasm/js/WebAssemblyTablePrototype.h: * wasm/js/WebAssemblyToJSCallee.h: * yarr/Yarr.h: * yarr/YarrParser.h: * yarr/generateYarrCanonicalizeUnicode: Source/WebCore: No new tests. Covered by existing tests. * bindings/js/JSDOMConstructorBase.h: * bindings/js/JSDOMWindowProperties.h: * bindings/scripts/CodeGeneratorJS.pm: (GenerateHeader): (GeneratePrototypeDeclaration): * bindings/scripts/test/JS/JSTestActiveDOMObject.h: * bindings/scripts/test/JS/JSTestEnabledBySetting.h: * bindings/scripts/test/JS/JSTestEnabledForContext.h: * bindings/scripts/test/JS/JSTestEventTarget.h: * bindings/scripts/test/JS/JSTestGlobalObject.h: * bindings/scripts/test/JS/JSTestIndexedSetterNoIdentifier.h: * bindings/scripts/test/JS/JSTestIndexedSetterThrowingException.h: * bindings/scripts/test/JS/JSTestIndexedSetterWithIdentifier.h: * bindings/scripts/test/JS/JSTestNamedAndIndexedSetterNoIdentifier.h: * bindings/scripts/test/JS/JSTestNamedAndIndexedSetterThrowingException.h: * bindings/scripts/test/JS/JSTestNamedAndIndexedSetterWithIdentifier.h: * bindings/scripts/test/JS/JSTestNamedDeleterNoIdentifier.h: * bindings/scripts/test/JS/JSTestNamedDeleterThrowingException.h: * bindings/scripts/test/JS/JSTestNamedDeleterWithIdentifier.h: * bindings/scripts/test/JS/JSTestNamedDeleterWithIndexedGetter.h: * bindings/scripts/test/JS/JSTestNamedGetterCallWith.h: * bindings/scripts/test/JS/JSTestNamedGetterNoIdentifier.h: * bindings/scripts/test/JS/JSTestNamedGetterWithIdentifier.h: * bindings/scripts/test/JS/JSTestNamedSetterNoIdentifier.h: * bindings/scripts/test/JS/JSTestNamedSetterThrowingException.h: * bindings/scripts/test/JS/JSTestNamedSetterWithIdentifier.h: * bindings/scripts/test/JS/JSTestNamedSetterWithIndexedGetter.h: * bindings/scripts/test/JS/JSTestNamedSetterWithIndexedGetterAndSetter.h: * bindings/scripts/test/JS/JSTestNamedSetterWithOverrideBuiltins.h: * bindings/scripts/test/JS/JSTestNamedSetterWithUnforgableProperties.h: * bindings/scripts/test/JS/JSTestNamedSetterWithUnforgablePropertiesAndOverrideBuiltins.h: * bindings/scripts/test/JS/JSTestObj.h: * bindings/scripts/test/JS/JSTestOverrideBuiltins.h: * bindings/scripts/test/JS/JSTestPluginInterface.h: * bindings/scripts/test/JS/JSTestTypedefs.h: * bridge/objc/objc_runtime.h: * bridge/runtime_array.h: * bridge/runtime_method.h: * bridge/runtime_object.h: Source/WebKit: * WebProcess/Plugins/Netscape/JSNPObject.h: Source/WTF: * wtf/Assertions.cpp: * wtf/AutomaticThread.cpp: * wtf/BitVector.h: * wtf/Bitmap.h: * wtf/BloomFilter.h: * wtf/Brigand.h: * wtf/CheckedArithmetic.h: * wtf/CrossThreadCopier.h: * wtf/CurrentTime.cpp: * wtf/DataLog.cpp: * wtf/DateMath.cpp: (WTF::daysFrom1970ToYear): * wtf/DeferrableRefCounted.h: * wtf/GetPtr.h: * wtf/HashFunctions.h: * wtf/HashMap.h: * wtf/HashTable.h: * wtf/HashTraits.h: * wtf/JSONValues.cpp: * wtf/JSONValues.h: * wtf/ListHashSet.h: * wtf/Lock.h: * wtf/LockAlgorithm.h: * wtf/LockAlgorithmInlines.h: (WTF::Hooks>::lockSlow): * wtf/Logger.h: * wtf/LoggerHelper.h: (WTF::LoggerHelper::childLogIdentifier const): * wtf/MainThread.cpp: * wtf/MetaAllocatorPtr.h: * wtf/MonotonicTime.h: * wtf/NaturalLoops.h: (WTF::NaturalLoops::NaturalLoops): * wtf/ObjectIdentifier.h: * wtf/RAMSize.cpp: * wtf/Ref.h: * wtf/RefPtr.h: * wtf/RetainPtr.h: * wtf/SchedulePair.h: * wtf/StackShot.h: * wtf/StdLibExtras.h: * wtf/TinyPtrSet.h: * wtf/URL.cpp: * wtf/URLHash.h: * wtf/URLParser.cpp: (WTF::URLParser::defaultPortForProtocol): * wtf/Vector.h: * wtf/VectorTraits.h: * wtf/WallTime.h: * wtf/WeakHashSet.h: * wtf/WordLock.h: * wtf/cocoa/CPUTimeCocoa.cpp: * wtf/cocoa/MemoryPressureHandlerCocoa.mm: * wtf/persistence/PersistentDecoder.h: * wtf/persistence/PersistentEncoder.h: * wtf/text/AtomStringHash.h: * wtf/text/CString.h: * wtf/text/StringBuilder.cpp: (WTF::expandedCapacity): * wtf/text/StringHash.h: * wtf/text/StringImpl.h: * wtf/text/StringToIntegerConversion.h: (WTF::toIntegralType): * wtf/text/SymbolRegistry.h: * wtf/text/TextStream.cpp: (WTF::hasFractions): * wtf/text/WTFString.h: * wtf/text/cocoa/TextBreakIteratorInternalICUCocoa.cpp: Canonical link: https://commits.webkit.org/215538@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@250005 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2019-09-18 00:36:19 +00:00
static constexpr size_t wordSize = 8;
FTL should be able to do some simple inline caches using LLVM patchpoints https://bugs.webkit.org/show_bug.cgi?id=123164 Source/JavaScriptCore: Reviewed by Mark Hahnenberg. This implements GetById inline caches in the FTL using llvm.webkit.patchpoint. The idea is that we ask LLVM for a nop slide the size of a GetById inline cache and then fill in the code after LLVM compilation is complete. For now, we just use the system calling convention for the arguments and return. We also still make some assumptions about registers that aren't correct. But, most of the scaffolding is there and this will successfully patch an inline cache. * JavaScriptCore.xcodeproj/project.pbxproj: * assembler/AbstractMacroAssembler.h: * assembler/LinkBuffer.cpp: (JSC::LinkBuffer::finalizeCodeWithoutDisassembly): (JSC::LinkBuffer::linkCode): (JSC::LinkBuffer::allocate): * assembler/LinkBuffer.h: (JSC::LinkBuffer::LinkBuffer): (JSC::LinkBuffer::link): * ftl/FTLAbbreviations.h: (JSC::FTL::constNull): (JSC::FTL::buildCall): * ftl/FTLCapabilities.cpp: (JSC::FTL::canCompile): * ftl/FTLCompile.cpp: (JSC::FTL::fixFunctionBasedOnStackMaps): * ftl/FTLInlineCacheDescriptor.h: Added. (JSC::FTL::InlineCacheDescriptor::InlineCacheDescriptor): (JSC::FTL::GetByIdDescriptor::GetByIdDescriptor): (JSC::FTL::GetByIdDescriptor::stackmapID): (JSC::FTL::GetByIdDescriptor::codeOrigin): (JSC::FTL::GetByIdDescriptor::uid): * ftl/FTLInlineCacheSize.cpp: Added. (JSC::FTL::sizeOfGetById): (JSC::FTL::sizeOfPutById): * ftl/FTLInlineCacheSize.h: Added. * ftl/FTLIntrinsicRepository.h: * ftl/FTLJITFinalizer.cpp: (JSC::FTL::JITFinalizer::finalizeFunction): * ftl/FTLJITFinalizer.h: * ftl/FTLLocation.cpp: (JSC::FTL::Location::directGPR): * ftl/FTLLocation.h: * ftl/FTLLowerDFGToLLVM.cpp: (JSC::FTL::LowerDFGToLLVM::compileGetById): * ftl/FTLOutput.h: (JSC::FTL::Output::call): * ftl/FTLSlowPathCall.cpp: Added. (JSC::FTL::callOperation): * ftl/FTLSlowPathCall.h: Added. (JSC::FTL::SlowPathCall::SlowPathCall): (JSC::FTL::SlowPathCall::call): (JSC::FTL::SlowPathCall::key): * ftl/FTLSlowPathCallKey.cpp: Added. (JSC::FTL::SlowPathCallKey::dump): * ftl/FTLSlowPathCallKey.h: Added. (JSC::FTL::SlowPathCallKey::SlowPathCallKey): (JSC::FTL::SlowPathCallKey::usedRegisters): (JSC::FTL::SlowPathCallKey::callTarget): (JSC::FTL::SlowPathCallKey::offset): (JSC::FTL::SlowPathCallKey::isEmptyValue): (JSC::FTL::SlowPathCallKey::isDeletedValue): (JSC::FTL::SlowPathCallKey::operator==): (JSC::FTL::SlowPathCallKey::hash): (JSC::FTL::SlowPathCallKeyHash::hash): (JSC::FTL::SlowPathCallKeyHash::equal): * ftl/FTLStackMaps.cpp: (JSC::FTL::StackMaps::Location::directGPR): * ftl/FTLStackMaps.h: * ftl/FTLState.h: * ftl/FTLThunks.cpp: (JSC::FTL::slowPathCallThunkGenerator): * ftl/FTLThunks.h: (JSC::FTL::Thunks::getSlowPathCallThunk): * jit/CCallHelpers.h: (JSC::CCallHelpers::setupArguments): * jit/GPRInfo.h: * jit/JITInlineCacheGenerator.cpp: (JSC::garbageStubInfo): (JSC::JITInlineCacheGenerator::JITInlineCacheGenerator): (JSC::JITByIdGenerator::finalize): * jit/JITInlineCacheGenerator.h: (JSC::JITByIdGenerator::slowPathBegin): * jit/RegisterSet.cpp: (JSC::RegisterSet::stackRegisters): (JSC::RegisterSet::specialRegisters): (JSC::RegisterSet::calleeSaveRegisters): (JSC::RegisterSet::allGPRs): (JSC::RegisterSet::allFPRs): (JSC::RegisterSet::allRegisters): (JSC::RegisterSet::dump): * jit/RegisterSet.h: (JSC::RegisterSet::exclude): (JSC::RegisterSet::numberOfSetRegisters): (JSC::RegisterSet::RegisterSet): (JSC::RegisterSet::isEmptyValue): (JSC::RegisterSet::isDeletedValue): (JSC::RegisterSet::operator==): (JSC::RegisterSet::hash): (JSC::RegisterSetHash::hash): (JSC::RegisterSetHash::equal): * runtime/Options.h: Source/WTF: Reviewed by Mark Hahnenberg. This needed some better bitvector support, like merging (|=), excluding (&=~), hashing, and bit counting. * wtf/BitVector.cpp: (WTF::BitVector::setSlow): (WTF::BitVector::excludeSlow): (WTF::BitVector::bitCountSlow): (WTF::BitVector::equalsSlowCase): (WTF::BitVector::hashSlowCase): (WTF::BitVector::dump): * wtf/BitVector.h: (WTF::BitVector::merge): (WTF::BitVector::exclude): (WTF::BitVector::bitCount): (WTF::BitVector::BitVector): (WTF::BitVector::isEmptyValue): (WTF::BitVector::isDeletedValue): (WTF::BitVector::isEmptyOrDeletedValue): (WTF::BitVector::operator==): (WTF::BitVector::hash): (WTF::BitVectorHash::hash): (WTF::BitVectorHash::equal): * wtf/HashTraits.h: (WTF::CustomHashTraits::constructDeletedValue): (WTF::CustomHashTraits::isDeletedValue): (WTF::CustomHashTraits::emptyValue): (WTF::CustomHashTraits::isEmptyValue): * wtf/StdLibExtras.h: (WTF::bitCount): Canonical link: https://commits.webkit.org/141315@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@157872 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2013-10-23 18:22:09 +00:00
FTL should generate code to call slow paths lazily https://bugs.webkit.org/show_bug.cgi?id=149936 Reviewed by Saam Barati. Source/JavaScriptCore: We often have complex slow paths in FTL-generated code. Those slow paths may never run. Even if they do run, they don't need stellar performance. So, it doesn't make sense to have LLVM worry about compiling such slow path code. This patch enables us to use our own MacroAssembler for compiling the slow path inside FTL code. It does this by using a crazy lambda thingy (see FTLLowerDFGToLLVM.cpp's lazySlowPath() and its documentation). The result is quite natural to use. Even for straight slow path calls via something like vmCall(), the lazySlowPath offers the benefit that the call marshalling and the exception checking are not expressed using LLVM IR and do not require LLVM to think about it. It also has the benefit that we never generate the code if it never runs. That's great, since function calls usually involve ~10 instructions total (move arguments to argument registers, make the call, check exception, etc.). This patch adds the lazy slow path abstraction and uses it for some slow paths in the FTL. The code we generate with lazy slow paths is worse than the code that LLVM would have generated. Therefore, a lazy slow path only makes sense when we have strong evidence that the slow path will execute infrequently relative to the fast path. This completely precludes the use of lazy slow paths for out-of-line Nodes that unconditionally call a C++ function. It also precludes their use for the GetByVal out-of-bounds handler, since when we generate a GetByVal with an out-of-bounds handler it means that we only know that the out-of-bounds case executed at least once. So, for all we know, it may actually be the common case. So, this patch just deployed the lazy slow path for GC slow paths and masquerades-as-undefined slow paths. It makes sense for GC slow paths because those have a statistical guarantee of slow path frequency - probably bounded at less than 1/10. It makes sense for masquerades-as- undefined because we can say quite confidently that this is an uncommon scenario on the modern Web. Something that's always been challenging about abstractions involving the MacroAssembler is that linking is a separate phase, and there is no way for someone who is just given access to the MacroAssembler& to emit code that requires linking, since linking happens once we have emitted all code and we are creating the LinkBuffer. Moreover, the FTL requires that the final parts of linking happen on the main thread. This patch ran into this issue, and solved it comprehensively, by introducing MacroAssembler::addLinkTask(). This takes a lambda and runs it at the bitter end of linking - when performFinalization() is called. This ensure that the task added by addLinkTask() runs on the main thread. This patch doesn't replace all of the previously existing idioms for dealing with this issue; we can do that later. This shows small speed-ups on a bunch of things. No big win on any benchmark aggregate. But mainly this is done for https://bugs.webkit.org/show_bug.cgi?id=149852, where we found that outlining the slow path in this way was a significant speed boost. * CMakeLists.txt: * JavaScriptCore.vcxproj/JavaScriptCore.vcxproj: * JavaScriptCore.xcodeproj/project.pbxproj: * assembler/AbstractMacroAssembler.h: (JSC::AbstractMacroAssembler::replaceWithAddressComputation): (JSC::AbstractMacroAssembler::addLinkTask): (JSC::AbstractMacroAssembler::AbstractMacroAssembler): * assembler/LinkBuffer.cpp: (JSC::LinkBuffer::linkCode): (JSC::LinkBuffer::allocate): (JSC::LinkBuffer::performFinalization): * assembler/LinkBuffer.h: (JSC::LinkBuffer::wasAlreadyDisassembled): (JSC::LinkBuffer::didAlreadyDisassemble): (JSC::LinkBuffer::vm): (JSC::LinkBuffer::executableOffsetFor): * bytecode/CodeOrigin.h: (JSC::CodeOrigin::CodeOrigin): (JSC::CodeOrigin::isSet): (JSC::CodeOrigin::operator bool): (JSC::CodeOrigin::isHashTableDeletedValue): (JSC::CodeOrigin::operator!): Deleted. * ftl/FTLCompile.cpp: (JSC::FTL::mmAllocateDataSection): * ftl/FTLInlineCacheDescriptor.h: (JSC::FTL::InlineCacheDescriptor::InlineCacheDescriptor): (JSC::FTL::CheckInDescriptor::CheckInDescriptor): (JSC::FTL::LazySlowPathDescriptor::LazySlowPathDescriptor): * ftl/FTLJITCode.h: * ftl/FTLJITFinalizer.cpp: (JSC::FTL::JITFinalizer::finalizeFunction): * ftl/FTLJITFinalizer.h: * ftl/FTLLazySlowPath.cpp: Added. (JSC::FTL::LazySlowPath::LazySlowPath): (JSC::FTL::LazySlowPath::~LazySlowPath): (JSC::FTL::LazySlowPath::generate): * ftl/FTLLazySlowPath.h: Added. (JSC::FTL::LazySlowPath::createGenerator): (JSC::FTL::LazySlowPath::patchpoint): (JSC::FTL::LazySlowPath::usedRegisters): (JSC::FTL::LazySlowPath::callSiteIndex): (JSC::FTL::LazySlowPath::stub): * ftl/FTLLazySlowPathCall.h: Added. (JSC::FTL::createLazyCallGenerator): * ftl/FTLLowerDFGToLLVM.cpp: (JSC::FTL::DFG::LowerDFGToLLVM::compileCreateActivation): (JSC::FTL::DFG::LowerDFGToLLVM::compileNewFunction): (JSC::FTL::DFG::LowerDFGToLLVM::compileCreateDirectArguments): (JSC::FTL::DFG::LowerDFGToLLVM::compileNewArrayWithSize): (JSC::FTL::DFG::LowerDFGToLLVM::compileMakeRope): (JSC::FTL::DFG::LowerDFGToLLVM::compileNotifyWrite): (JSC::FTL::DFG::LowerDFGToLLVM::compileIsObjectOrNull): (JSC::FTL::DFG::LowerDFGToLLVM::compileIsFunction): (JSC::FTL::DFG::LowerDFGToLLVM::compileIn): (JSC::FTL::DFG::LowerDFGToLLVM::compileMaterializeNewObject): (JSC::FTL::DFG::LowerDFGToLLVM::compileMaterializeCreateActivation): (JSC::FTL::DFG::LowerDFGToLLVM::compileCheckWatchdogTimer): (JSC::FTL::DFG::LowerDFGToLLVM::allocatePropertyStorageWithSizeImpl): (JSC::FTL::DFG::LowerDFGToLLVM::allocateObject): (JSC::FTL::DFG::LowerDFGToLLVM::allocateJSArray): (JSC::FTL::DFG::LowerDFGToLLVM::buildTypeOf): (JSC::FTL::DFG::LowerDFGToLLVM::sensibleDoubleToInt32): (JSC::FTL::DFG::LowerDFGToLLVM::lazySlowPath): (JSC::FTL::DFG::LowerDFGToLLVM::speculate): (JSC::FTL::DFG::LowerDFGToLLVM::emitStoreBarrier): * ftl/FTLOperations.cpp: (JSC::FTL::operationMaterializeObjectInOSR): (JSC::FTL::compileFTLLazySlowPath): * ftl/FTLOperations.h: * ftl/FTLSlowPathCall.cpp: (JSC::FTL::SlowPathCallContext::SlowPathCallContext): (JSC::FTL::SlowPathCallContext::~SlowPathCallContext): (JSC::FTL::SlowPathCallContext::keyWithTarget): (JSC::FTL::SlowPathCallContext::makeCall): (JSC::FTL::callSiteIndexForCodeOrigin): (JSC::FTL::storeCodeOrigin): Deleted. (JSC::FTL::callOperation): Deleted. * ftl/FTLSlowPathCall.h: (JSC::FTL::callOperation): * ftl/FTLState.h: * ftl/FTLThunks.cpp: (JSC::FTL::genericGenerationThunkGenerator): (JSC::FTL::osrExitGenerationThunkGenerator): (JSC::FTL::lazySlowPathGenerationThunkGenerator): (JSC::FTL::registerClobberCheck): * ftl/FTLThunks.h: * interpreter/CallFrame.h: (JSC::CallSiteIndex::CallSiteIndex): (JSC::CallSiteIndex::operator bool): (JSC::CallSiteIndex::bits): * jit/CCallHelpers.h: (JSC::CCallHelpers::setupArgument): (JSC::CCallHelpers::setupArgumentsWithExecState): * jit/JITOperations.cpp: Source/WTF: Enables SharedTask to handle any function type, not just void(). It's probably better to use SharedTask instead of std::function in performance-sensitive code. std::function uses the system malloc and has copy semantics. SharedTask uses FastMalloc and has aliasing semantics. So, you can just trust that it will have sensible performance characteristics. * wtf/ParallelHelperPool.cpp: (WTF::ParallelHelperClient::~ParallelHelperClient): (WTF::ParallelHelperClient::setTask): (WTF::ParallelHelperClient::doSomeHelping): (WTF::ParallelHelperClient::runTaskInParallel): (WTF::ParallelHelperClient::finish): (WTF::ParallelHelperClient::claimTask): (WTF::ParallelHelperClient::runTask): (WTF::ParallelHelperPool::doSomeHelping): (WTF::ParallelHelperPool::helperThreadBody): * wtf/ParallelHelperPool.h: (WTF::ParallelHelperClient::setFunction): (WTF::ParallelHelperClient::runFunctionInParallel): (WTF::ParallelHelperClient::pool): * wtf/SharedTask.h: (WTF::createSharedTask): (WTF::SharedTask::SharedTask): Deleted. (WTF::SharedTask::~SharedTask): Deleted. (WTF::SharedTaskFunctor::SharedTaskFunctor): Deleted. Canonical link: https://commits.webkit.org/168154@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@190860 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2015-10-12 17:56:26 +00:00
SlowPathCallContext::SlowPathCallContext(
[JSC] Use DataIC for AccessCase https://bugs.webkit.org/show_bug.cgi?id=226072 Reviewed by Saam Barati and Filip Pizlo. Source/JavaScriptCore: This patch adds non-repatching IC for Baseline JIT in ARM64. This does not work in non-ARM64 architectures (including X64) due to the use of link-register. 1. We add non-repatching IC, which is enabled only in Baseline due to performance reason. We are using the existing IC in DFG and FTL. Non-repatching includes fast-path, and slow-path's operation function. 2. We still keep InlineAccess in all tiers. Removing that causes 0.3 ~ 1.0% regression in Speedometer2. This means that we still need some repatching when we first introduce stubs. 3. We add a mechanism to share generated code stubs in non-repatching IC. Currently, getter / setter / custom accessors are excluded since their code relies on JSGlobalObject, CodeBlock etc. which are not included in AccessCase's data structure. 4. This patch still relies on that CodeBlock will be destroyed synchronously since we need to ensure that sharing-hash-table does not include already-dead JIT code stubs. We can fix it (e.g. annotating epoch to these stubs, bump them in finalizeUnconditionally), but we avoid doing that to prevent from further enlarging this patch. This patch is already significant in its size. 5. Added callOperation(Address) support in CCallHelpers, which can save the target in nonArgGPR0, and call it so that we can use Address including GPR which is also used for arguments. Performance is neutral in JetStream2 and Speedometer2. But it offers the way to remove some code generation in Baseline. * assembler/MacroAssemblerARM64E.h: (JSC::MacroAssemblerARM64E::call): * bytecode/AccessCase.cpp: (JSC::AccessCase::create): (JSC::AccessCase::createTransition): (JSC::AccessCase::createDelete): (JSC::AccessCase::createCheckPrivateBrand): (JSC::AccessCase::createSetPrivateBrand): (JSC::AccessCase::fromStructureStubInfo): (JSC::AccessCase::clone const): (JSC::AccessCase::generateWithGuard): (JSC::AccessCase::generateImpl): (JSC::AccessCase::canBeShared): * bytecode/AccessCase.h: (JSC::AccessCase::hash const): (JSC::AccessCase::AccessCase): (JSC::SharedJITStubSet::Hash::Key::Key): (JSC::SharedJITStubSet::Hash::Key::isHashTableDeletedValue const): (JSC::SharedJITStubSet::Hash::Key::operator==): (JSC::SharedJITStubSet::Hash::hash): (JSC::SharedJITStubSet::Hash::equal): (JSC::SharedJITStubSet::Searcher::Translator::hash): (JSC::SharedJITStubSet::Searcher::Translator::equal): (JSC::SharedJITStubSet::PointerTranslator::hash): (JSC::SharedJITStubSet::PointerTranslator::equal): (JSC::SharedJITStubSet::add): (JSC::SharedJITStubSet::remove): (JSC::SharedJITStubSet::find): * bytecode/ByValInfo.h: (JSC::ByValInfo::setUp): (JSC::ByValInfo::offsetOfSlowOperation): (JSC::ByValInfo::offsetOfNotIndexJumpTarget): (JSC::ByValInfo::offsetOfBadTypeJumpTarget): * bytecode/CallLinkInfo.cpp: (JSC::CallLinkInfo::initializeDirectCall): (JSC::CallLinkInfo::setDirectCallTarget): * bytecode/CodeBlock.h: (JSC::CodeBlock::useDataIC const): * bytecode/GetterSetterAccessCase.cpp: (JSC::GetterSetterAccessCase::create): (JSC::GetterSetterAccessCase::clone const): (JSC::GetterSetterAccessCase::emitDOMJITGetter): * bytecode/GetterSetterAccessCase.h: * bytecode/InlineAccess.cpp: (JSC::getScratchRegister): (JSC::InlineAccess::rewireStubAsJumpInAccessNotUsingInlineAccess): (JSC::InlineAccess::rewireStubAsJumpInAccess): (JSC::InlineAccess::resetStubAsJumpInAccess): (JSC::InlineAccess::resetStubAsJumpInAccessNotUsingInlineAccess): (JSC::InlineAccess::rewireStubAsJump): Deleted. * bytecode/InlineAccess.h: * bytecode/InstanceOfAccessCase.cpp: (JSC::InstanceOfAccessCase::create): (JSC::InstanceOfAccessCase::clone const): * bytecode/InstanceOfAccessCase.h: * bytecode/IntrinsicGetterAccessCase.cpp: (JSC::IntrinsicGetterAccessCase::create): (JSC::IntrinsicGetterAccessCase::clone const): * bytecode/IntrinsicGetterAccessCase.h: * bytecode/ModuleNamespaceAccessCase.cpp: (JSC::ModuleNamespaceAccessCase::create): (JSC::ModuleNamespaceAccessCase::clone const): * bytecode/ModuleNamespaceAccessCase.h: * bytecode/ObjectPropertyConditionSet.h: (JSC::ObjectPropertyConditionSet::hash const): (JSC::ObjectPropertyConditionSet::operator==): (JSC::ObjectPropertyConditionSet::operator!=): * bytecode/PolymorphicAccess.cpp: (JSC::AccessGenerationState::installWatchpoint): (JSC::AccessGenerationState::succeed): (JSC::AccessGenerationState::preserveLiveRegistersToStackForCallWithoutExceptions): (JSC::PolymorphicAccess::addCases): (JSC::PolymorphicAccess::addCase): (JSC::PolymorphicAccess::visitWeak const): (JSC::PolymorphicAccess::regenerate): * bytecode/PolymorphicAccess.h: (JSC::AccessGenerationState::AccessGenerationState): (JSC::AccessGenerationState::preserveLiveRegistersToStackForCallWithoutExceptions): * bytecode/ProxyableAccessCase.cpp: (JSC::ProxyableAccessCase::create): (JSC::ProxyableAccessCase::clone const): * bytecode/ProxyableAccessCase.h: * bytecode/StructureStubInfo.cpp: (JSC::StructureStubInfo::StructureStubInfo): (JSC::StructureStubInfo::addAccessCase): (JSC::StructureStubInfo::reset): * bytecode/StructureStubInfo.h: (JSC::StructureStubInfo::offsetOfCodePtr): (JSC::StructureStubInfo::offsetOfSlowPathStartLocation): (JSC::StructureStubInfo::offsetOfSlowOperation): (JSC::StructureStubInfo::patchableJump): Deleted. * dfg/DFGJITCompiler.h: (JSC::DFG::JITCompiler::appendCall): * dfg/DFGSlowPathGenerator.h: (JSC::DFG::slowPathICCall): * dfg/DFGSpeculativeJIT.cpp: (JSC::DFG::SpeculativeJIT::compileGetById): (JSC::DFG::SpeculativeJIT::compileGetByIdFlush): (JSC::DFG::SpeculativeJIT::compileDeleteById): (JSC::DFG::SpeculativeJIT::compileDeleteByVal): (JSC::DFG::SpeculativeJIT::compileInById): (JSC::DFG::SpeculativeJIT::compileInByVal): (JSC::DFG::SpeculativeJIT::compileGetPrivateNameByVal): (JSC::DFG::SpeculativeJIT::compileGetPrivateNameById): (JSC::DFG::SpeculativeJIT::compilePutPrivateNameById): (JSC::DFG::SpeculativeJIT::compileCheckPrivateBrand): (JSC::DFG::SpeculativeJIT::compileSetPrivateBrand): (JSC::DFG::SpeculativeJIT::compileInstanceOfForCells): (JSC::DFG::SpeculativeJIT::compileInstanceOf): (JSC::DFG::SpeculativeJIT::compilePutByIdFlush): (JSC::DFG::SpeculativeJIT::compilePutById): (JSC::DFG::SpeculativeJIT::compilePutByIdDirect): (JSC::DFG::SpeculativeJIT::cachedPutById): * dfg/DFGSpeculativeJIT.h: (JSC::DFG::SpeculativeJIT::callOperation): (JSC::DFG::SpeculativeJIT::appendCall): (JSC::DFG::SpeculativeJIT::appendCallSetResult): * dfg/DFGSpeculativeJIT32_64.cpp: (JSC::DFG::SpeculativeJIT::cachedGetById): (JSC::DFG::SpeculativeJIT::cachedGetByIdWithThis): (JSC::DFG::SpeculativeJIT::compile): * dfg/DFGSpeculativeJIT64.cpp: (JSC::DFG::SpeculativeJIT::cachedGetById): (JSC::DFG::SpeculativeJIT::cachedGetByIdWithThis): (JSC::DFG::SpeculativeJIT::compile): * ftl/FTLLowerDFGToB3.cpp: (JSC::FTL::DFG::LowerDFGToB3::getPrivateName): (JSC::FTL::DFG::LowerDFGToB3::compilePrivateBrandAccess): (JSC::FTL::DFG::LowerDFGToB3::cachedPutById): (JSC::FTL::DFG::LowerDFGToB3::compileGetByVal): (JSC::FTL::DFG::LowerDFGToB3::compileDelBy): (JSC::FTL::DFG::LowerDFGToB3::compileCompareStrictEq): * ftl/FTLSlowPathCall.cpp: (JSC::FTL::SlowPathCallContext::SlowPathCallContext): (JSC::FTL::SlowPathCallContext::keyWithTarget const): (JSC::FTL::SlowPathCallContext::makeCall): * ftl/FTLSlowPathCall.h: (JSC::FTL::callOperation): * ftl/FTLSlowPathCallKey.cpp: (JSC::FTL::SlowPathCallKey::dump const): * ftl/FTLSlowPathCallKey.h: (JSC::FTL::SlowPathCallKey::SlowPathCallKey): (JSC::FTL::SlowPathCallKey::indirectOffset const): (JSC::FTL::SlowPathCallKey::withCallTarget): (JSC::FTL::SlowPathCallKey::operator== const): (JSC::FTL::SlowPathCallKey::hash const): * ftl/FTLThunks.cpp: (JSC::FTL::genericGenerationThunkGenerator): (JSC::FTL::slowPathCallThunkGenerator): * jit/AssemblyHelpers.cpp: (JSC::AssemblyHelpers::emitVirtualCall): * jit/CCallHelpers.cpp: (JSC::CCallHelpers::emitJITCodeOver): * jit/CCallHelpers.h: (JSC::CCallHelpers::ArgCollection::ArgCollection): (JSC::CCallHelpers::ArgCollection::pushRegArg): (JSC::CCallHelpers::ArgCollection::pushExtraRegArg): (JSC::CCallHelpers::ArgCollection::pushNonArg): (JSC::CCallHelpers::ArgCollection::addGPRArg): (JSC::CCallHelpers::ArgCollection::addGPRExtraArg): (JSC::CCallHelpers::ArgCollection::addStackArg): (JSC::CCallHelpers::ArgCollection::addPoke): (JSC::CCallHelpers::calculatePokeOffset): (JSC::CCallHelpers::pokeForArgument): (JSC::CCallHelpers::stackAligned): (JSC::CCallHelpers::marshallArgumentRegister): (JSC::CCallHelpers::setupArgumentsImpl): (JSC::CCallHelpers::pokeArgumentsAligned): (JSC::CCallHelpers::std::is_integral<CURRENT_ARGUMENT_TYPE>::value): (JSC::CCallHelpers::std::is_pointer<CURRENT_ARGUMENT_TYPE>::value): (JSC::CCallHelpers::setupArgumentsEntryImpl): (JSC::CCallHelpers::setupArguments): (JSC::CCallHelpers::setupArgumentsForIndirectCall): * jit/GCAwareJITStubRoutine.cpp: (JSC::GCAwareJITStubRoutine::GCAwareJITStubRoutine): (JSC::GCAwareJITStubRoutine::makeGCAware): (JSC::GCAwareJITStubRoutine::observeZeroRefCount): (JSC::PolymorphicAccessJITStubRoutine::PolymorphicAccessJITStubRoutine): (JSC::PolymorphicAccessJITStubRoutine::observeZeroRefCount): (JSC::PolymorphicAccessJITStubRoutine::computeHash): (JSC::MarkingGCAwareJITStubRoutine::MarkingGCAwareJITStubRoutine): (JSC::GCAwareJITStubRoutineWithExceptionHandler::GCAwareJITStubRoutineWithExceptionHandler): (JSC::createICJITStubRoutine): (JSC::createJITStubRoutine): Deleted. * jit/GCAwareJITStubRoutine.h: (JSC::GCAwareJITStubRoutine::create): (JSC::PolymorphicAccessJITStubRoutine::cases const): (JSC::PolymorphicAccessJITStubRoutine::weakStructures const): (JSC::PolymorphicAccessJITStubRoutine::hash const): * jit/GPRInfo.h: * jit/JIT.cpp: (JSC::JIT::link): * jit/JIT.h: * jit/JITCall.cpp: (JSC::JIT::emit_op_iterator_open): (JSC::JIT::emitSlow_op_iterator_open): (JSC::JIT::emit_op_iterator_next): (JSC::JIT::emitSlow_op_iterator_next): * jit/JITCall32_64.cpp: (JSC::JIT::emit_op_iterator_open): (JSC::JIT::emit_op_iterator_next): * jit/JITCode.h: (JSC::JITCode::useDataIC): * jit/JITInlineCacheGenerator.cpp: (JSC::JITInlineCacheGenerator::JITInlineCacheGenerator): (JSC::JITInlineCacheGenerator::finalize): (JSC::JITByIdGenerator::JITByIdGenerator): (JSC::JITByIdGenerator::finalize): (JSC::JITByIdGenerator::generateFastCommon): (JSC::JITGetByIdGenerator::JITGetByIdGenerator): (JSC::JITGetByIdWithThisGenerator::JITGetByIdWithThisGenerator): (JSC::JITPutByIdGenerator::JITPutByIdGenerator): (JSC::JITDelByValGenerator::JITDelByValGenerator): (JSC::JITDelByValGenerator::generateFastPath): (JSC::JITDelByValGenerator::finalize): (JSC::JITDelByIdGenerator::JITDelByIdGenerator): (JSC::JITDelByIdGenerator::generateFastPath): (JSC::JITDelByIdGenerator::finalize): (JSC::JITInByValGenerator::JITInByValGenerator): (JSC::JITInByValGenerator::generateFastPath): (JSC::JITInByValGenerator::finalize): (JSC::JITInByIdGenerator::JITInByIdGenerator): (JSC::JITInstanceOfGenerator::JITInstanceOfGenerator): (JSC::JITInstanceOfGenerator::generateFastPath): (JSC::JITInstanceOfGenerator::finalize): (JSC::JITGetByValGenerator::JITGetByValGenerator): (JSC::JITGetByValGenerator::generateFastPath): (JSC::JITGetByValGenerator::finalize): (JSC::JITPrivateBrandAccessGenerator::JITPrivateBrandAccessGenerator): (JSC::JITPrivateBrandAccessGenerator::generateFastPath): (JSC::JITPrivateBrandAccessGenerator::finalize): * jit/JITInlineCacheGenerator.h: (JSC::JITGetByIdGenerator::JITGetByIdGenerator): Deleted. (JSC::JITGetByIdWithThisGenerator::JITGetByIdWithThisGenerator): Deleted. (JSC::JITPutByIdGenerator::JITPutByIdGenerator): Deleted. (JSC::JITDelByValGenerator::JITDelByValGenerator): Deleted. (JSC::JITDelByValGenerator::slowPathJump const): Deleted. (JSC::JITDelByIdGenerator::JITDelByIdGenerator): Deleted. (JSC::JITDelByIdGenerator::slowPathJump const): Deleted. (JSC::JITInByIdGenerator::JITInByIdGenerator): Deleted. (JSC::JITInstanceOfGenerator::JITInstanceOfGenerator): Deleted. (JSC::JITGetByValGenerator::JITGetByValGenerator): Deleted. (JSC::JITGetByValGenerator::slowPathJump const): Deleted. (JSC::JITPrivateBrandAccessGenerator::JITPrivateBrandAccessGenerator): Deleted. (JSC::JITPrivateBrandAccessGenerator::slowPathJump const): Deleted. * jit/JITInlines.h: (JSC::JIT::emitLoadForArrayMode): (JSC::JIT::appendCallWithExceptionCheck): (JSC::JIT::appendCallWithExceptionCheckSetJSValueResult): (JSC::JIT::appendCallWithExceptionCheckSetJSValueResultWithProfile): * jit/JITOpcodes.cpp: (JSC::JIT::emit_op_instanceof): (JSC::JIT::emitSlow_op_instanceof): (JSC::JIT::privateCompileHasIndexedProperty): (JSC::JIT::emit_op_has_enumerable_indexed_property): (JSC::JIT::emitSlow_op_has_enumerable_indexed_property): * jit/JITOpcodes32_64.cpp: (JSC::JIT::emit_op_instanceof): (JSC::JIT::privateCompileHasIndexedProperty): (JSC::JIT::emit_op_has_enumerable_indexed_property): * jit/JITOperations.cpp: (JSC::JSC_DEFINE_JIT_OPERATION): * jit/JITPropertyAccess.cpp: (JSC::JIT::emit_op_get_by_val): (JSC::JIT::emitSlow_op_get_by_val): (JSC::JIT::slow_op_get_by_val_prepareCallGenerator): (JSC::JIT::emit_op_get_private_name): (JSC::JIT::emitSlow_op_get_private_name): (JSC::JIT::slow_op_get_private_name_prepareCallGenerator): (JSC::JIT::emit_op_set_private_brand): (JSC::JIT::emitSlow_op_set_private_brand): (JSC::JIT::emit_op_check_private_brand): (JSC::JIT::emitSlow_op_check_private_brand): (JSC::JIT::emit_op_put_by_val): (JSC::JIT::emitGenericContiguousPutByVal): (JSC::JIT::emitArrayStoragePutByVal): (JSC::JIT::emitPutByValWithCachedId): (JSC::JIT::emitSlow_op_put_by_val): (JSC::JIT::slow_op_put_by_val_prepareCallGenerator): (JSC::JIT::emit_op_put_private_name): (JSC::JIT::emitSlow_op_put_private_name): (JSC::JIT::slow_op_put_private_name_prepareCallGenerator): (JSC::JIT::emit_op_del_by_id): (JSC::JIT::emitSlow_op_del_by_id): (JSC::JIT::slow_op_del_by_id_prepareCallGenerator): (JSC::JIT::emit_op_del_by_val): (JSC::JIT::emitSlow_op_del_by_val): (JSC::JIT::slow_op_del_by_val_prepareCallGenerator): (JSC::JIT::emit_op_try_get_by_id): (JSC::JIT::emitSlow_op_try_get_by_id): (JSC::JIT::emit_op_get_by_id_direct): (JSC::JIT::emitSlow_op_get_by_id_direct): (JSC::JIT::emit_op_get_by_id): (JSC::JIT::emit_op_get_by_id_with_this): (JSC::JIT::emitSlow_op_get_by_id): (JSC::JIT::slow_op_get_by_id_prepareCallGenerator): (JSC::JIT::emitSlow_op_get_by_id_with_this): (JSC::JIT::slow_op_get_by_id_with_this_prepareCallGenerator): (JSC::JIT::emit_op_put_by_id): (JSC::JIT::emitSlow_op_put_by_id): (JSC::JIT::slow_op_put_by_id_prepareCallGenerator): (JSC::JIT::emit_op_in_by_id): (JSC::JIT::emitSlow_op_in_by_id): (JSC::JIT::emit_op_in_by_val): (JSC::JIT::emitSlow_op_in_by_val): (JSC::JIT::privateCompilePutByVal): (JSC::JIT::privateCompilePutPrivateNameWithCachedId): (JSC::JIT::privateCompilePutByValWithCachedId): (JSC::JIT::emitDoubleLoad): (JSC::JIT::emitContiguousLoad): (JSC::JIT::emitArrayStorageLoad): (JSC::JIT::emitIntTypedArrayPutByVal): (JSC::JIT::emitFloatTypedArrayPutByVal): * jit/JITPropertyAccess32_64.cpp: (JSC::JIT::emit_op_del_by_id): (JSC::JIT::emit_op_del_by_val): (JSC::JIT::emit_op_get_by_val): (JSC::JIT::emit_op_get_private_name): (JSC::JIT::emit_op_set_private_brand): (JSC::JIT::emit_op_check_private_brand): (JSC::JIT::emit_op_put_by_val): (JSC::JIT::emitGenericContiguousPutByVal): (JSC::JIT::emitArrayStoragePutByVal): (JSC::JIT::emit_op_try_get_by_id): (JSC::JIT::emit_op_get_by_id_direct): (JSC::JIT::emit_op_get_by_id): (JSC::JIT::emit_op_get_by_id_with_this): (JSC::JIT::emit_op_put_by_id): (JSC::JIT::emit_op_in_by_id): (JSC::JIT::emit_op_in_by_val): * jit/JITStubRoutine.h: * jit/PolymorphicCallStubRoutine.cpp: (JSC::PolymorphicCallStubRoutine::PolymorphicCallStubRoutine): * jit/Repatch.cpp: (JSC::readPutICCallTarget): (JSC::repatchSlowPathCall): (JSC::tryCacheGetBy): (JSC::repatchGetBy): (JSC::tryCacheArrayGetByVal): (JSC::repatchArrayGetByVal): (JSC::tryCachePutByID): (JSC::repatchPutByID): (JSC::tryCacheDeleteBy): (JSC::repatchDeleteBy): (JSC::tryCacheInBy): (JSC::repatchInBy): (JSC::tryCacheCheckPrivateBrand): (JSC::repatchCheckPrivateBrand): (JSC::tryCacheSetPrivateBrand): (JSC::repatchSetPrivateBrand): (JSC::tryCacheInstanceOf): (JSC::repatchInstanceOf): (JSC::linkSlowFor): (JSC::linkVirtualFor): (JSC::resetGetBy): (JSC::resetPutByID): (JSC::resetDelBy): (JSC::resetInBy): (JSC::resetInstanceOf): (JSC::resetCheckPrivateBrand): (JSC::resetSetPrivateBrand): (JSC::resetPatchableJump): Deleted. * jit/Repatch.h: * runtime/Options.cpp: (JSC::Options::recomputeDependentOptions): * runtime/OptionsList.h: * runtime/StructureIDTable.h: * runtime/VM.cpp: (JSC::VM::VM): (JSC::VM::~VM): * runtime/VM.h: Tools: * Scripts/run-jsc-stress-tests: Canonical link: https://commits.webkit.org/238638@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@278656 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-06-09 11:17:36 +00:00
RegisterSet usedRegisters, CCallHelpers& jit, unsigned numArgs, GPRReg returnRegister, GPRReg indirectCallTargetRegister)
FTL should generate code to call slow paths lazily https://bugs.webkit.org/show_bug.cgi?id=149936 Reviewed by Saam Barati. Source/JavaScriptCore: We often have complex slow paths in FTL-generated code. Those slow paths may never run. Even if they do run, they don't need stellar performance. So, it doesn't make sense to have LLVM worry about compiling such slow path code. This patch enables us to use our own MacroAssembler for compiling the slow path inside FTL code. It does this by using a crazy lambda thingy (see FTLLowerDFGToLLVM.cpp's lazySlowPath() and its documentation). The result is quite natural to use. Even for straight slow path calls via something like vmCall(), the lazySlowPath offers the benefit that the call marshalling and the exception checking are not expressed using LLVM IR and do not require LLVM to think about it. It also has the benefit that we never generate the code if it never runs. That's great, since function calls usually involve ~10 instructions total (move arguments to argument registers, make the call, check exception, etc.). This patch adds the lazy slow path abstraction and uses it for some slow paths in the FTL. The code we generate with lazy slow paths is worse than the code that LLVM would have generated. Therefore, a lazy slow path only makes sense when we have strong evidence that the slow path will execute infrequently relative to the fast path. This completely precludes the use of lazy slow paths for out-of-line Nodes that unconditionally call a C++ function. It also precludes their use for the GetByVal out-of-bounds handler, since when we generate a GetByVal with an out-of-bounds handler it means that we only know that the out-of-bounds case executed at least once. So, for all we know, it may actually be the common case. So, this patch just deployed the lazy slow path for GC slow paths and masquerades-as-undefined slow paths. It makes sense for GC slow paths because those have a statistical guarantee of slow path frequency - probably bounded at less than 1/10. It makes sense for masquerades-as- undefined because we can say quite confidently that this is an uncommon scenario on the modern Web. Something that's always been challenging about abstractions involving the MacroAssembler is that linking is a separate phase, and there is no way for someone who is just given access to the MacroAssembler& to emit code that requires linking, since linking happens once we have emitted all code and we are creating the LinkBuffer. Moreover, the FTL requires that the final parts of linking happen on the main thread. This patch ran into this issue, and solved it comprehensively, by introducing MacroAssembler::addLinkTask(). This takes a lambda and runs it at the bitter end of linking - when performFinalization() is called. This ensure that the task added by addLinkTask() runs on the main thread. This patch doesn't replace all of the previously existing idioms for dealing with this issue; we can do that later. This shows small speed-ups on a bunch of things. No big win on any benchmark aggregate. But mainly this is done for https://bugs.webkit.org/show_bug.cgi?id=149852, where we found that outlining the slow path in this way was a significant speed boost. * CMakeLists.txt: * JavaScriptCore.vcxproj/JavaScriptCore.vcxproj: * JavaScriptCore.xcodeproj/project.pbxproj: * assembler/AbstractMacroAssembler.h: (JSC::AbstractMacroAssembler::replaceWithAddressComputation): (JSC::AbstractMacroAssembler::addLinkTask): (JSC::AbstractMacroAssembler::AbstractMacroAssembler): * assembler/LinkBuffer.cpp: (JSC::LinkBuffer::linkCode): (JSC::LinkBuffer::allocate): (JSC::LinkBuffer::performFinalization): * assembler/LinkBuffer.h: (JSC::LinkBuffer::wasAlreadyDisassembled): (JSC::LinkBuffer::didAlreadyDisassemble): (JSC::LinkBuffer::vm): (JSC::LinkBuffer::executableOffsetFor): * bytecode/CodeOrigin.h: (JSC::CodeOrigin::CodeOrigin): (JSC::CodeOrigin::isSet): (JSC::CodeOrigin::operator bool): (JSC::CodeOrigin::isHashTableDeletedValue): (JSC::CodeOrigin::operator!): Deleted. * ftl/FTLCompile.cpp: (JSC::FTL::mmAllocateDataSection): * ftl/FTLInlineCacheDescriptor.h: (JSC::FTL::InlineCacheDescriptor::InlineCacheDescriptor): (JSC::FTL::CheckInDescriptor::CheckInDescriptor): (JSC::FTL::LazySlowPathDescriptor::LazySlowPathDescriptor): * ftl/FTLJITCode.h: * ftl/FTLJITFinalizer.cpp: (JSC::FTL::JITFinalizer::finalizeFunction): * ftl/FTLJITFinalizer.h: * ftl/FTLLazySlowPath.cpp: Added. (JSC::FTL::LazySlowPath::LazySlowPath): (JSC::FTL::LazySlowPath::~LazySlowPath): (JSC::FTL::LazySlowPath::generate): * ftl/FTLLazySlowPath.h: Added. (JSC::FTL::LazySlowPath::createGenerator): (JSC::FTL::LazySlowPath::patchpoint): (JSC::FTL::LazySlowPath::usedRegisters): (JSC::FTL::LazySlowPath::callSiteIndex): (JSC::FTL::LazySlowPath::stub): * ftl/FTLLazySlowPathCall.h: Added. (JSC::FTL::createLazyCallGenerator): * ftl/FTLLowerDFGToLLVM.cpp: (JSC::FTL::DFG::LowerDFGToLLVM::compileCreateActivation): (JSC::FTL::DFG::LowerDFGToLLVM::compileNewFunction): (JSC::FTL::DFG::LowerDFGToLLVM::compileCreateDirectArguments): (JSC::FTL::DFG::LowerDFGToLLVM::compileNewArrayWithSize): (JSC::FTL::DFG::LowerDFGToLLVM::compileMakeRope): (JSC::FTL::DFG::LowerDFGToLLVM::compileNotifyWrite): (JSC::FTL::DFG::LowerDFGToLLVM::compileIsObjectOrNull): (JSC::FTL::DFG::LowerDFGToLLVM::compileIsFunction): (JSC::FTL::DFG::LowerDFGToLLVM::compileIn): (JSC::FTL::DFG::LowerDFGToLLVM::compileMaterializeNewObject): (JSC::FTL::DFG::LowerDFGToLLVM::compileMaterializeCreateActivation): (JSC::FTL::DFG::LowerDFGToLLVM::compileCheckWatchdogTimer): (JSC::FTL::DFG::LowerDFGToLLVM::allocatePropertyStorageWithSizeImpl): (JSC::FTL::DFG::LowerDFGToLLVM::allocateObject): (JSC::FTL::DFG::LowerDFGToLLVM::allocateJSArray): (JSC::FTL::DFG::LowerDFGToLLVM::buildTypeOf): (JSC::FTL::DFG::LowerDFGToLLVM::sensibleDoubleToInt32): (JSC::FTL::DFG::LowerDFGToLLVM::lazySlowPath): (JSC::FTL::DFG::LowerDFGToLLVM::speculate): (JSC::FTL::DFG::LowerDFGToLLVM::emitStoreBarrier): * ftl/FTLOperations.cpp: (JSC::FTL::operationMaterializeObjectInOSR): (JSC::FTL::compileFTLLazySlowPath): * ftl/FTLOperations.h: * ftl/FTLSlowPathCall.cpp: (JSC::FTL::SlowPathCallContext::SlowPathCallContext): (JSC::FTL::SlowPathCallContext::~SlowPathCallContext): (JSC::FTL::SlowPathCallContext::keyWithTarget): (JSC::FTL::SlowPathCallContext::makeCall): (JSC::FTL::callSiteIndexForCodeOrigin): (JSC::FTL::storeCodeOrigin): Deleted. (JSC::FTL::callOperation): Deleted. * ftl/FTLSlowPathCall.h: (JSC::FTL::callOperation): * ftl/FTLState.h: * ftl/FTLThunks.cpp: (JSC::FTL::genericGenerationThunkGenerator): (JSC::FTL::osrExitGenerationThunkGenerator): (JSC::FTL::lazySlowPathGenerationThunkGenerator): (JSC::FTL::registerClobberCheck): * ftl/FTLThunks.h: * interpreter/CallFrame.h: (JSC::CallSiteIndex::CallSiteIndex): (JSC::CallSiteIndex::operator bool): (JSC::CallSiteIndex::bits): * jit/CCallHelpers.h: (JSC::CCallHelpers::setupArgument): (JSC::CCallHelpers::setupArgumentsWithExecState): * jit/JITOperations.cpp: Source/WTF: Enables SharedTask to handle any function type, not just void(). It's probably better to use SharedTask instead of std::function in performance-sensitive code. std::function uses the system malloc and has copy semantics. SharedTask uses FastMalloc and has aliasing semantics. So, you can just trust that it will have sensible performance characteristics. * wtf/ParallelHelperPool.cpp: (WTF::ParallelHelperClient::~ParallelHelperClient): (WTF::ParallelHelperClient::setTask): (WTF::ParallelHelperClient::doSomeHelping): (WTF::ParallelHelperClient::runTaskInParallel): (WTF::ParallelHelperClient::finish): (WTF::ParallelHelperClient::claimTask): (WTF::ParallelHelperClient::runTask): (WTF::ParallelHelperPool::doSomeHelping): (WTF::ParallelHelperPool::helperThreadBody): * wtf/ParallelHelperPool.h: (WTF::ParallelHelperClient::setFunction): (WTF::ParallelHelperClient::runFunctionInParallel): (WTF::ParallelHelperClient::pool): * wtf/SharedTask.h: (WTF::createSharedTask): (WTF::SharedTask::SharedTask): Deleted. (WTF::SharedTask::~SharedTask): Deleted. (WTF::SharedTaskFunctor::SharedTaskFunctor): Deleted. Canonical link: https://commits.webkit.org/168154@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@190860 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2015-10-12 17:56:26 +00:00
: m_jit(jit)
, m_numArgs(numArgs)
, m_returnRegister(returnRegister)
{
// We don't care that you're using callee-save, stack, or hardware registers.
usedRegisters.exclude(RegisterSet::stackRegisters());
usedRegisters.exclude(RegisterSet::reservedHardwareRegisters());
usedRegisters.exclude(RegisterSet::calleeSaveRegisters());
FTL should be able to do some simple inline caches using LLVM patchpoints https://bugs.webkit.org/show_bug.cgi?id=123164 Source/JavaScriptCore: Reviewed by Mark Hahnenberg. This implements GetById inline caches in the FTL using llvm.webkit.patchpoint. The idea is that we ask LLVM for a nop slide the size of a GetById inline cache and then fill in the code after LLVM compilation is complete. For now, we just use the system calling convention for the arguments and return. We also still make some assumptions about registers that aren't correct. But, most of the scaffolding is there and this will successfully patch an inline cache. * JavaScriptCore.xcodeproj/project.pbxproj: * assembler/AbstractMacroAssembler.h: * assembler/LinkBuffer.cpp: (JSC::LinkBuffer::finalizeCodeWithoutDisassembly): (JSC::LinkBuffer::linkCode): (JSC::LinkBuffer::allocate): * assembler/LinkBuffer.h: (JSC::LinkBuffer::LinkBuffer): (JSC::LinkBuffer::link): * ftl/FTLAbbreviations.h: (JSC::FTL::constNull): (JSC::FTL::buildCall): * ftl/FTLCapabilities.cpp: (JSC::FTL::canCompile): * ftl/FTLCompile.cpp: (JSC::FTL::fixFunctionBasedOnStackMaps): * ftl/FTLInlineCacheDescriptor.h: Added. (JSC::FTL::InlineCacheDescriptor::InlineCacheDescriptor): (JSC::FTL::GetByIdDescriptor::GetByIdDescriptor): (JSC::FTL::GetByIdDescriptor::stackmapID): (JSC::FTL::GetByIdDescriptor::codeOrigin): (JSC::FTL::GetByIdDescriptor::uid): * ftl/FTLInlineCacheSize.cpp: Added. (JSC::FTL::sizeOfGetById): (JSC::FTL::sizeOfPutById): * ftl/FTLInlineCacheSize.h: Added. * ftl/FTLIntrinsicRepository.h: * ftl/FTLJITFinalizer.cpp: (JSC::FTL::JITFinalizer::finalizeFunction): * ftl/FTLJITFinalizer.h: * ftl/FTLLocation.cpp: (JSC::FTL::Location::directGPR): * ftl/FTLLocation.h: * ftl/FTLLowerDFGToLLVM.cpp: (JSC::FTL::LowerDFGToLLVM::compileGetById): * ftl/FTLOutput.h: (JSC::FTL::Output::call): * ftl/FTLSlowPathCall.cpp: Added. (JSC::FTL::callOperation): * ftl/FTLSlowPathCall.h: Added. (JSC::FTL::SlowPathCall::SlowPathCall): (JSC::FTL::SlowPathCall::call): (JSC::FTL::SlowPathCall::key): * ftl/FTLSlowPathCallKey.cpp: Added. (JSC::FTL::SlowPathCallKey::dump): * ftl/FTLSlowPathCallKey.h: Added. (JSC::FTL::SlowPathCallKey::SlowPathCallKey): (JSC::FTL::SlowPathCallKey::usedRegisters): (JSC::FTL::SlowPathCallKey::callTarget): (JSC::FTL::SlowPathCallKey::offset): (JSC::FTL::SlowPathCallKey::isEmptyValue): (JSC::FTL::SlowPathCallKey::isDeletedValue): (JSC::FTL::SlowPathCallKey::operator==): (JSC::FTL::SlowPathCallKey::hash): (JSC::FTL::SlowPathCallKeyHash::hash): (JSC::FTL::SlowPathCallKeyHash::equal): * ftl/FTLStackMaps.cpp: (JSC::FTL::StackMaps::Location::directGPR): * ftl/FTLStackMaps.h: * ftl/FTLState.h: * ftl/FTLThunks.cpp: (JSC::FTL::slowPathCallThunkGenerator): * ftl/FTLThunks.h: (JSC::FTL::Thunks::getSlowPathCallThunk): * jit/CCallHelpers.h: (JSC::CCallHelpers::setupArguments): * jit/GPRInfo.h: * jit/JITInlineCacheGenerator.cpp: (JSC::garbageStubInfo): (JSC::JITInlineCacheGenerator::JITInlineCacheGenerator): (JSC::JITByIdGenerator::finalize): * jit/JITInlineCacheGenerator.h: (JSC::JITByIdGenerator::slowPathBegin): * jit/RegisterSet.cpp: (JSC::RegisterSet::stackRegisters): (JSC::RegisterSet::specialRegisters): (JSC::RegisterSet::calleeSaveRegisters): (JSC::RegisterSet::allGPRs): (JSC::RegisterSet::allFPRs): (JSC::RegisterSet::allRegisters): (JSC::RegisterSet::dump): * jit/RegisterSet.h: (JSC::RegisterSet::exclude): (JSC::RegisterSet::numberOfSetRegisters): (JSC::RegisterSet::RegisterSet): (JSC::RegisterSet::isEmptyValue): (JSC::RegisterSet::isDeletedValue): (JSC::RegisterSet::operator==): (JSC::RegisterSet::hash): (JSC::RegisterSetHash::hash): (JSC::RegisterSetHash::equal): * runtime/Options.h: Source/WTF: Reviewed by Mark Hahnenberg. This needed some better bitvector support, like merging (|=), excluding (&=~), hashing, and bit counting. * wtf/BitVector.cpp: (WTF::BitVector::setSlow): (WTF::BitVector::excludeSlow): (WTF::BitVector::bitCountSlow): (WTF::BitVector::equalsSlowCase): (WTF::BitVector::hashSlowCase): (WTF::BitVector::dump): * wtf/BitVector.h: (WTF::BitVector::merge): (WTF::BitVector::exclude): (WTF::BitVector::bitCount): (WTF::BitVector::BitVector): (WTF::BitVector::isEmptyValue): (WTF::BitVector::isDeletedValue): (WTF::BitVector::isEmptyOrDeletedValue): (WTF::BitVector::operator==): (WTF::BitVector::hash): (WTF::BitVectorHash::hash): (WTF::BitVectorHash::equal): * wtf/HashTraits.h: (WTF::CustomHashTraits::constructDeletedValue): (WTF::CustomHashTraits::isDeletedValue): (WTF::CustomHashTraits::emptyValue): (WTF::CustomHashTraits::isEmptyValue): * wtf/StdLibExtras.h: (WTF::bitCount): Canonical link: https://commits.webkit.org/141315@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@157872 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2013-10-23 18:22:09 +00:00
FTL should generate code to call slow paths lazily https://bugs.webkit.org/show_bug.cgi?id=149936 Reviewed by Saam Barati. Source/JavaScriptCore: We often have complex slow paths in FTL-generated code. Those slow paths may never run. Even if they do run, they don't need stellar performance. So, it doesn't make sense to have LLVM worry about compiling such slow path code. This patch enables us to use our own MacroAssembler for compiling the slow path inside FTL code. It does this by using a crazy lambda thingy (see FTLLowerDFGToLLVM.cpp's lazySlowPath() and its documentation). The result is quite natural to use. Even for straight slow path calls via something like vmCall(), the lazySlowPath offers the benefit that the call marshalling and the exception checking are not expressed using LLVM IR and do not require LLVM to think about it. It also has the benefit that we never generate the code if it never runs. That's great, since function calls usually involve ~10 instructions total (move arguments to argument registers, make the call, check exception, etc.). This patch adds the lazy slow path abstraction and uses it for some slow paths in the FTL. The code we generate with lazy slow paths is worse than the code that LLVM would have generated. Therefore, a lazy slow path only makes sense when we have strong evidence that the slow path will execute infrequently relative to the fast path. This completely precludes the use of lazy slow paths for out-of-line Nodes that unconditionally call a C++ function. It also precludes their use for the GetByVal out-of-bounds handler, since when we generate a GetByVal with an out-of-bounds handler it means that we only know that the out-of-bounds case executed at least once. So, for all we know, it may actually be the common case. So, this patch just deployed the lazy slow path for GC slow paths and masquerades-as-undefined slow paths. It makes sense for GC slow paths because those have a statistical guarantee of slow path frequency - probably bounded at less than 1/10. It makes sense for masquerades-as- undefined because we can say quite confidently that this is an uncommon scenario on the modern Web. Something that's always been challenging about abstractions involving the MacroAssembler is that linking is a separate phase, and there is no way for someone who is just given access to the MacroAssembler& to emit code that requires linking, since linking happens once we have emitted all code and we are creating the LinkBuffer. Moreover, the FTL requires that the final parts of linking happen on the main thread. This patch ran into this issue, and solved it comprehensively, by introducing MacroAssembler::addLinkTask(). This takes a lambda and runs it at the bitter end of linking - when performFinalization() is called. This ensure that the task added by addLinkTask() runs on the main thread. This patch doesn't replace all of the previously existing idioms for dealing with this issue; we can do that later. This shows small speed-ups on a bunch of things. No big win on any benchmark aggregate. But mainly this is done for https://bugs.webkit.org/show_bug.cgi?id=149852, where we found that outlining the slow path in this way was a significant speed boost. * CMakeLists.txt: * JavaScriptCore.vcxproj/JavaScriptCore.vcxproj: * JavaScriptCore.xcodeproj/project.pbxproj: * assembler/AbstractMacroAssembler.h: (JSC::AbstractMacroAssembler::replaceWithAddressComputation): (JSC::AbstractMacroAssembler::addLinkTask): (JSC::AbstractMacroAssembler::AbstractMacroAssembler): * assembler/LinkBuffer.cpp: (JSC::LinkBuffer::linkCode): (JSC::LinkBuffer::allocate): (JSC::LinkBuffer::performFinalization): * assembler/LinkBuffer.h: (JSC::LinkBuffer::wasAlreadyDisassembled): (JSC::LinkBuffer::didAlreadyDisassemble): (JSC::LinkBuffer::vm): (JSC::LinkBuffer::executableOffsetFor): * bytecode/CodeOrigin.h: (JSC::CodeOrigin::CodeOrigin): (JSC::CodeOrigin::isSet): (JSC::CodeOrigin::operator bool): (JSC::CodeOrigin::isHashTableDeletedValue): (JSC::CodeOrigin::operator!): Deleted. * ftl/FTLCompile.cpp: (JSC::FTL::mmAllocateDataSection): * ftl/FTLInlineCacheDescriptor.h: (JSC::FTL::InlineCacheDescriptor::InlineCacheDescriptor): (JSC::FTL::CheckInDescriptor::CheckInDescriptor): (JSC::FTL::LazySlowPathDescriptor::LazySlowPathDescriptor): * ftl/FTLJITCode.h: * ftl/FTLJITFinalizer.cpp: (JSC::FTL::JITFinalizer::finalizeFunction): * ftl/FTLJITFinalizer.h: * ftl/FTLLazySlowPath.cpp: Added. (JSC::FTL::LazySlowPath::LazySlowPath): (JSC::FTL::LazySlowPath::~LazySlowPath): (JSC::FTL::LazySlowPath::generate): * ftl/FTLLazySlowPath.h: Added. (JSC::FTL::LazySlowPath::createGenerator): (JSC::FTL::LazySlowPath::patchpoint): (JSC::FTL::LazySlowPath::usedRegisters): (JSC::FTL::LazySlowPath::callSiteIndex): (JSC::FTL::LazySlowPath::stub): * ftl/FTLLazySlowPathCall.h: Added. (JSC::FTL::createLazyCallGenerator): * ftl/FTLLowerDFGToLLVM.cpp: (JSC::FTL::DFG::LowerDFGToLLVM::compileCreateActivation): (JSC::FTL::DFG::LowerDFGToLLVM::compileNewFunction): (JSC::FTL::DFG::LowerDFGToLLVM::compileCreateDirectArguments): (JSC::FTL::DFG::LowerDFGToLLVM::compileNewArrayWithSize): (JSC::FTL::DFG::LowerDFGToLLVM::compileMakeRope): (JSC::FTL::DFG::LowerDFGToLLVM::compileNotifyWrite): (JSC::FTL::DFG::LowerDFGToLLVM::compileIsObjectOrNull): (JSC::FTL::DFG::LowerDFGToLLVM::compileIsFunction): (JSC::FTL::DFG::LowerDFGToLLVM::compileIn): (JSC::FTL::DFG::LowerDFGToLLVM::compileMaterializeNewObject): (JSC::FTL::DFG::LowerDFGToLLVM::compileMaterializeCreateActivation): (JSC::FTL::DFG::LowerDFGToLLVM::compileCheckWatchdogTimer): (JSC::FTL::DFG::LowerDFGToLLVM::allocatePropertyStorageWithSizeImpl): (JSC::FTL::DFG::LowerDFGToLLVM::allocateObject): (JSC::FTL::DFG::LowerDFGToLLVM::allocateJSArray): (JSC::FTL::DFG::LowerDFGToLLVM::buildTypeOf): (JSC::FTL::DFG::LowerDFGToLLVM::sensibleDoubleToInt32): (JSC::FTL::DFG::LowerDFGToLLVM::lazySlowPath): (JSC::FTL::DFG::LowerDFGToLLVM::speculate): (JSC::FTL::DFG::LowerDFGToLLVM::emitStoreBarrier): * ftl/FTLOperations.cpp: (JSC::FTL::operationMaterializeObjectInOSR): (JSC::FTL::compileFTLLazySlowPath): * ftl/FTLOperations.h: * ftl/FTLSlowPathCall.cpp: (JSC::FTL::SlowPathCallContext::SlowPathCallContext): (JSC::FTL::SlowPathCallContext::~SlowPathCallContext): (JSC::FTL::SlowPathCallContext::keyWithTarget): (JSC::FTL::SlowPathCallContext::makeCall): (JSC::FTL::callSiteIndexForCodeOrigin): (JSC::FTL::storeCodeOrigin): Deleted. (JSC::FTL::callOperation): Deleted. * ftl/FTLSlowPathCall.h: (JSC::FTL::callOperation): * ftl/FTLState.h: * ftl/FTLThunks.cpp: (JSC::FTL::genericGenerationThunkGenerator): (JSC::FTL::osrExitGenerationThunkGenerator): (JSC::FTL::lazySlowPathGenerationThunkGenerator): (JSC::FTL::registerClobberCheck): * ftl/FTLThunks.h: * interpreter/CallFrame.h: (JSC::CallSiteIndex::CallSiteIndex): (JSC::CallSiteIndex::operator bool): (JSC::CallSiteIndex::bits): * jit/CCallHelpers.h: (JSC::CCallHelpers::setupArgument): (JSC::CCallHelpers::setupArgumentsWithExecState): * jit/JITOperations.cpp: Source/WTF: Enables SharedTask to handle any function type, not just void(). It's probably better to use SharedTask instead of std::function in performance-sensitive code. std::function uses the system malloc and has copy semantics. SharedTask uses FastMalloc and has aliasing semantics. So, you can just trust that it will have sensible performance characteristics. * wtf/ParallelHelperPool.cpp: (WTF::ParallelHelperClient::~ParallelHelperClient): (WTF::ParallelHelperClient::setTask): (WTF::ParallelHelperClient::doSomeHelping): (WTF::ParallelHelperClient::runTaskInParallel): (WTF::ParallelHelperClient::finish): (WTF::ParallelHelperClient::claimTask): (WTF::ParallelHelperClient::runTask): (WTF::ParallelHelperPool::doSomeHelping): (WTF::ParallelHelperPool::helperThreadBody): * wtf/ParallelHelperPool.h: (WTF::ParallelHelperClient::setFunction): (WTF::ParallelHelperClient::runFunctionInParallel): (WTF::ParallelHelperClient::pool): * wtf/SharedTask.h: (WTF::createSharedTask): (WTF::SharedTask::SharedTask): Deleted. (WTF::SharedTask::~SharedTask): Deleted. (WTF::SharedTaskFunctor::SharedTaskFunctor): Deleted. Canonical link: https://commits.webkit.org/168154@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@190860 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2015-10-12 17:56:26 +00:00
// The return register doesn't need to be saved.
if (m_returnRegister != InvalidGPRReg)
usedRegisters.clear(m_returnRegister);
FTL should be able to do some simple inline caches using LLVM patchpoints https://bugs.webkit.org/show_bug.cgi?id=123164 Source/JavaScriptCore: Reviewed by Mark Hahnenberg. This implements GetById inline caches in the FTL using llvm.webkit.patchpoint. The idea is that we ask LLVM for a nop slide the size of a GetById inline cache and then fill in the code after LLVM compilation is complete. For now, we just use the system calling convention for the arguments and return. We also still make some assumptions about registers that aren't correct. But, most of the scaffolding is there and this will successfully patch an inline cache. * JavaScriptCore.xcodeproj/project.pbxproj: * assembler/AbstractMacroAssembler.h: * assembler/LinkBuffer.cpp: (JSC::LinkBuffer::finalizeCodeWithoutDisassembly): (JSC::LinkBuffer::linkCode): (JSC::LinkBuffer::allocate): * assembler/LinkBuffer.h: (JSC::LinkBuffer::LinkBuffer): (JSC::LinkBuffer::link): * ftl/FTLAbbreviations.h: (JSC::FTL::constNull): (JSC::FTL::buildCall): * ftl/FTLCapabilities.cpp: (JSC::FTL::canCompile): * ftl/FTLCompile.cpp: (JSC::FTL::fixFunctionBasedOnStackMaps): * ftl/FTLInlineCacheDescriptor.h: Added. (JSC::FTL::InlineCacheDescriptor::InlineCacheDescriptor): (JSC::FTL::GetByIdDescriptor::GetByIdDescriptor): (JSC::FTL::GetByIdDescriptor::stackmapID): (JSC::FTL::GetByIdDescriptor::codeOrigin): (JSC::FTL::GetByIdDescriptor::uid): * ftl/FTLInlineCacheSize.cpp: Added. (JSC::FTL::sizeOfGetById): (JSC::FTL::sizeOfPutById): * ftl/FTLInlineCacheSize.h: Added. * ftl/FTLIntrinsicRepository.h: * ftl/FTLJITFinalizer.cpp: (JSC::FTL::JITFinalizer::finalizeFunction): * ftl/FTLJITFinalizer.h: * ftl/FTLLocation.cpp: (JSC::FTL::Location::directGPR): * ftl/FTLLocation.h: * ftl/FTLLowerDFGToLLVM.cpp: (JSC::FTL::LowerDFGToLLVM::compileGetById): * ftl/FTLOutput.h: (JSC::FTL::Output::call): * ftl/FTLSlowPathCall.cpp: Added. (JSC::FTL::callOperation): * ftl/FTLSlowPathCall.h: Added. (JSC::FTL::SlowPathCall::SlowPathCall): (JSC::FTL::SlowPathCall::call): (JSC::FTL::SlowPathCall::key): * ftl/FTLSlowPathCallKey.cpp: Added. (JSC::FTL::SlowPathCallKey::dump): * ftl/FTLSlowPathCallKey.h: Added. (JSC::FTL::SlowPathCallKey::SlowPathCallKey): (JSC::FTL::SlowPathCallKey::usedRegisters): (JSC::FTL::SlowPathCallKey::callTarget): (JSC::FTL::SlowPathCallKey::offset): (JSC::FTL::SlowPathCallKey::isEmptyValue): (JSC::FTL::SlowPathCallKey::isDeletedValue): (JSC::FTL::SlowPathCallKey::operator==): (JSC::FTL::SlowPathCallKey::hash): (JSC::FTL::SlowPathCallKeyHash::hash): (JSC::FTL::SlowPathCallKeyHash::equal): * ftl/FTLStackMaps.cpp: (JSC::FTL::StackMaps::Location::directGPR): * ftl/FTLStackMaps.h: * ftl/FTLState.h: * ftl/FTLThunks.cpp: (JSC::FTL::slowPathCallThunkGenerator): * ftl/FTLThunks.h: (JSC::FTL::Thunks::getSlowPathCallThunk): * jit/CCallHelpers.h: (JSC::CCallHelpers::setupArguments): * jit/GPRInfo.h: * jit/JITInlineCacheGenerator.cpp: (JSC::garbageStubInfo): (JSC::JITInlineCacheGenerator::JITInlineCacheGenerator): (JSC::JITByIdGenerator::finalize): * jit/JITInlineCacheGenerator.h: (JSC::JITByIdGenerator::slowPathBegin): * jit/RegisterSet.cpp: (JSC::RegisterSet::stackRegisters): (JSC::RegisterSet::specialRegisters): (JSC::RegisterSet::calleeSaveRegisters): (JSC::RegisterSet::allGPRs): (JSC::RegisterSet::allFPRs): (JSC::RegisterSet::allRegisters): (JSC::RegisterSet::dump): * jit/RegisterSet.h: (JSC::RegisterSet::exclude): (JSC::RegisterSet::numberOfSetRegisters): (JSC::RegisterSet::RegisterSet): (JSC::RegisterSet::isEmptyValue): (JSC::RegisterSet::isDeletedValue): (JSC::RegisterSet::operator==): (JSC::RegisterSet::hash): (JSC::RegisterSetHash::hash): (JSC::RegisterSetHash::equal): * runtime/Options.h: Source/WTF: Reviewed by Mark Hahnenberg. This needed some better bitvector support, like merging (|=), excluding (&=~), hashing, and bit counting. * wtf/BitVector.cpp: (WTF::BitVector::setSlow): (WTF::BitVector::excludeSlow): (WTF::BitVector::bitCountSlow): (WTF::BitVector::equalsSlowCase): (WTF::BitVector::hashSlowCase): (WTF::BitVector::dump): * wtf/BitVector.h: (WTF::BitVector::merge): (WTF::BitVector::exclude): (WTF::BitVector::bitCount): (WTF::BitVector::BitVector): (WTF::BitVector::isEmptyValue): (WTF::BitVector::isDeletedValue): (WTF::BitVector::isEmptyOrDeletedValue): (WTF::BitVector::operator==): (WTF::BitVector::hash): (WTF::BitVectorHash::hash): (WTF::BitVectorHash::equal): * wtf/HashTraits.h: (WTF::CustomHashTraits::constructDeletedValue): (WTF::CustomHashTraits::isDeletedValue): (WTF::CustomHashTraits::emptyValue): (WTF::CustomHashTraits::isEmptyValue): * wtf/StdLibExtras.h: (WTF::bitCount): Canonical link: https://commits.webkit.org/141315@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@157872 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2013-10-23 18:22:09 +00:00
FTL should generate code to call slow paths lazily https://bugs.webkit.org/show_bug.cgi?id=149936 Reviewed by Saam Barati. Source/JavaScriptCore: We often have complex slow paths in FTL-generated code. Those slow paths may never run. Even if they do run, they don't need stellar performance. So, it doesn't make sense to have LLVM worry about compiling such slow path code. This patch enables us to use our own MacroAssembler for compiling the slow path inside FTL code. It does this by using a crazy lambda thingy (see FTLLowerDFGToLLVM.cpp's lazySlowPath() and its documentation). The result is quite natural to use. Even for straight slow path calls via something like vmCall(), the lazySlowPath offers the benefit that the call marshalling and the exception checking are not expressed using LLVM IR and do not require LLVM to think about it. It also has the benefit that we never generate the code if it never runs. That's great, since function calls usually involve ~10 instructions total (move arguments to argument registers, make the call, check exception, etc.). This patch adds the lazy slow path abstraction and uses it for some slow paths in the FTL. The code we generate with lazy slow paths is worse than the code that LLVM would have generated. Therefore, a lazy slow path only makes sense when we have strong evidence that the slow path will execute infrequently relative to the fast path. This completely precludes the use of lazy slow paths for out-of-line Nodes that unconditionally call a C++ function. It also precludes their use for the GetByVal out-of-bounds handler, since when we generate a GetByVal with an out-of-bounds handler it means that we only know that the out-of-bounds case executed at least once. So, for all we know, it may actually be the common case. So, this patch just deployed the lazy slow path for GC slow paths and masquerades-as-undefined slow paths. It makes sense for GC slow paths because those have a statistical guarantee of slow path frequency - probably bounded at less than 1/10. It makes sense for masquerades-as- undefined because we can say quite confidently that this is an uncommon scenario on the modern Web. Something that's always been challenging about abstractions involving the MacroAssembler is that linking is a separate phase, and there is no way for someone who is just given access to the MacroAssembler& to emit code that requires linking, since linking happens once we have emitted all code and we are creating the LinkBuffer. Moreover, the FTL requires that the final parts of linking happen on the main thread. This patch ran into this issue, and solved it comprehensively, by introducing MacroAssembler::addLinkTask(). This takes a lambda and runs it at the bitter end of linking - when performFinalization() is called. This ensure that the task added by addLinkTask() runs on the main thread. This patch doesn't replace all of the previously existing idioms for dealing with this issue; we can do that later. This shows small speed-ups on a bunch of things. No big win on any benchmark aggregate. But mainly this is done for https://bugs.webkit.org/show_bug.cgi?id=149852, where we found that outlining the slow path in this way was a significant speed boost. * CMakeLists.txt: * JavaScriptCore.vcxproj/JavaScriptCore.vcxproj: * JavaScriptCore.xcodeproj/project.pbxproj: * assembler/AbstractMacroAssembler.h: (JSC::AbstractMacroAssembler::replaceWithAddressComputation): (JSC::AbstractMacroAssembler::addLinkTask): (JSC::AbstractMacroAssembler::AbstractMacroAssembler): * assembler/LinkBuffer.cpp: (JSC::LinkBuffer::linkCode): (JSC::LinkBuffer::allocate): (JSC::LinkBuffer::performFinalization): * assembler/LinkBuffer.h: (JSC::LinkBuffer::wasAlreadyDisassembled): (JSC::LinkBuffer::didAlreadyDisassemble): (JSC::LinkBuffer::vm): (JSC::LinkBuffer::executableOffsetFor): * bytecode/CodeOrigin.h: (JSC::CodeOrigin::CodeOrigin): (JSC::CodeOrigin::isSet): (JSC::CodeOrigin::operator bool): (JSC::CodeOrigin::isHashTableDeletedValue): (JSC::CodeOrigin::operator!): Deleted. * ftl/FTLCompile.cpp: (JSC::FTL::mmAllocateDataSection): * ftl/FTLInlineCacheDescriptor.h: (JSC::FTL::InlineCacheDescriptor::InlineCacheDescriptor): (JSC::FTL::CheckInDescriptor::CheckInDescriptor): (JSC::FTL::LazySlowPathDescriptor::LazySlowPathDescriptor): * ftl/FTLJITCode.h: * ftl/FTLJITFinalizer.cpp: (JSC::FTL::JITFinalizer::finalizeFunction): * ftl/FTLJITFinalizer.h: * ftl/FTLLazySlowPath.cpp: Added. (JSC::FTL::LazySlowPath::LazySlowPath): (JSC::FTL::LazySlowPath::~LazySlowPath): (JSC::FTL::LazySlowPath::generate): * ftl/FTLLazySlowPath.h: Added. (JSC::FTL::LazySlowPath::createGenerator): (JSC::FTL::LazySlowPath::patchpoint): (JSC::FTL::LazySlowPath::usedRegisters): (JSC::FTL::LazySlowPath::callSiteIndex): (JSC::FTL::LazySlowPath::stub): * ftl/FTLLazySlowPathCall.h: Added. (JSC::FTL::createLazyCallGenerator): * ftl/FTLLowerDFGToLLVM.cpp: (JSC::FTL::DFG::LowerDFGToLLVM::compileCreateActivation): (JSC::FTL::DFG::LowerDFGToLLVM::compileNewFunction): (JSC::FTL::DFG::LowerDFGToLLVM::compileCreateDirectArguments): (JSC::FTL::DFG::LowerDFGToLLVM::compileNewArrayWithSize): (JSC::FTL::DFG::LowerDFGToLLVM::compileMakeRope): (JSC::FTL::DFG::LowerDFGToLLVM::compileNotifyWrite): (JSC::FTL::DFG::LowerDFGToLLVM::compileIsObjectOrNull): (JSC::FTL::DFG::LowerDFGToLLVM::compileIsFunction): (JSC::FTL::DFG::LowerDFGToLLVM::compileIn): (JSC::FTL::DFG::LowerDFGToLLVM::compileMaterializeNewObject): (JSC::FTL::DFG::LowerDFGToLLVM::compileMaterializeCreateActivation): (JSC::FTL::DFG::LowerDFGToLLVM::compileCheckWatchdogTimer): (JSC::FTL::DFG::LowerDFGToLLVM::allocatePropertyStorageWithSizeImpl): (JSC::FTL::DFG::LowerDFGToLLVM::allocateObject): (JSC::FTL::DFG::LowerDFGToLLVM::allocateJSArray): (JSC::FTL::DFG::LowerDFGToLLVM::buildTypeOf): (JSC::FTL::DFG::LowerDFGToLLVM::sensibleDoubleToInt32): (JSC::FTL::DFG::LowerDFGToLLVM::lazySlowPath): (JSC::FTL::DFG::LowerDFGToLLVM::speculate): (JSC::FTL::DFG::LowerDFGToLLVM::emitStoreBarrier): * ftl/FTLOperations.cpp: (JSC::FTL::operationMaterializeObjectInOSR): (JSC::FTL::compileFTLLazySlowPath): * ftl/FTLOperations.h: * ftl/FTLSlowPathCall.cpp: (JSC::FTL::SlowPathCallContext::SlowPathCallContext): (JSC::FTL::SlowPathCallContext::~SlowPathCallContext): (JSC::FTL::SlowPathCallContext::keyWithTarget): (JSC::FTL::SlowPathCallContext::makeCall): (JSC::FTL::callSiteIndexForCodeOrigin): (JSC::FTL::storeCodeOrigin): Deleted. (JSC::FTL::callOperation): Deleted. * ftl/FTLSlowPathCall.h: (JSC::FTL::callOperation): * ftl/FTLState.h: * ftl/FTLThunks.cpp: (JSC::FTL::genericGenerationThunkGenerator): (JSC::FTL::osrExitGenerationThunkGenerator): (JSC::FTL::lazySlowPathGenerationThunkGenerator): (JSC::FTL::registerClobberCheck): * ftl/FTLThunks.h: * interpreter/CallFrame.h: (JSC::CallSiteIndex::CallSiteIndex): (JSC::CallSiteIndex::operator bool): (JSC::CallSiteIndex::bits): * jit/CCallHelpers.h: (JSC::CCallHelpers::setupArgument): (JSC::CCallHelpers::setupArgumentsWithExecState): * jit/JITOperations.cpp: Source/WTF: Enables SharedTask to handle any function type, not just void(). It's probably better to use SharedTask instead of std::function in performance-sensitive code. std::function uses the system malloc and has copy semantics. SharedTask uses FastMalloc and has aliasing semantics. So, you can just trust that it will have sensible performance characteristics. * wtf/ParallelHelperPool.cpp: (WTF::ParallelHelperClient::~ParallelHelperClient): (WTF::ParallelHelperClient::setTask): (WTF::ParallelHelperClient::doSomeHelping): (WTF::ParallelHelperClient::runTaskInParallel): (WTF::ParallelHelperClient::finish): (WTF::ParallelHelperClient::claimTask): (WTF::ParallelHelperClient::runTask): (WTF::ParallelHelperPool::doSomeHelping): (WTF::ParallelHelperPool::helperThreadBody): * wtf/ParallelHelperPool.h: (WTF::ParallelHelperClient::setFunction): (WTF::ParallelHelperClient::runFunctionInParallel): (WTF::ParallelHelperClient::pool): * wtf/SharedTask.h: (WTF::createSharedTask): (WTF::SharedTask::SharedTask): Deleted. (WTF::SharedTask::~SharedTask): Deleted. (WTF::SharedTaskFunctor::SharedTaskFunctor): Deleted. Canonical link: https://commits.webkit.org/168154@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@190860 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2015-10-12 17:56:26 +00:00
size_t stackBytesNeededForReturnAddress = wordSize;
Switch FTL GetById/PutById IC's over to using AnyRegCC https://bugs.webkit.org/show_bug.cgi?id=124094 Source/JavaScriptCore: Reviewed by Sam Weinig. This closes the loop on inline caches (IC's) in the FTL. The goal is to have IC's in LLVM-generated code that are just as efficient (if not more so) than what a custom JIT could do. As in zero sources of overhead. Not a single extra instruction or even register allocation pathology. We accomplish this by having two thingies in LLVM. First is the llvm.experimental.patchpoint intrinsic, which is sort of an inline machine code snippet that we can fill in with whatever we want and then modify subsequently. But you have only two choices of how to pass values to a patchpoint: (1) via the calling convention or (2) via the stackmap. Neither are good for operands to an IC (like the base pointer for a GetById, for example). (1) is bad because it results in things being pinned to certain registers a priori; a custom JIT (like the DFG) will not pin IC operands to any registers a priori but will allow the register allocator to do whatever it wants. (2) is bad because the operands may be spilled or may be represented in other crazy ways. You generally want an IC to have its operands in registers. Also, patchpoints only return values using the calling convention, which is unfortunate since it pins the return value to a register a priori. This is where the second thingy comes in: the AnyRegCC. This is a special calling convention only for use with patchpoints. It means that arguments passed "by CC" in the patchpoint can be placed in any register, and the register that gets used is reported as part of the stackmap. It also means that the return value (if there is one) can be placed in any register, and the stackmap will tell you which one it was. Thus, patchpoints combined with AnyRegCC mean that you not only get the kind of self-modifying code that you want for IC's, but you also get all of the register allocation goodness that a custom JIT would have given you. Except that you're getting it from LLVM and not a custom JIT. Awesome. Even though all of the fun stuff is on the LLVM side, this patch was harder than you'd expect. First the obvious bits: - IC patchpoints now use AnyRegCC instead of the C CC. (CC = calling convention.) - FTL::fixFunctionBasedOnStackMaps() now correctly figures out which registers the IC is supposed to use instead of assuming C CC argument registers. And then all of the stuff that broke and that this patch fixes: - IC sizing based on generating a dummy IC (what FTLInlineCacheSize did) is totally bad on x86-64, where various register permutations lead to bizarre header bytes and eclectic SIB encodings. I changed that to have magic constants, for now. - Slow path calls didn't preserve the CC return register. - Repatch's scratch register allocation would get totally confused if the operand registers weren't one of the DFG-style "temp" registers. And by "totally confused" I mean that it would crash. - We assumed that r10 is callee-saved. It's not. That one dude's PPT about x86-64 cdecl that I found on the intertubes was not a trustworthy source of information, apparently. - Call repatching didn't know that the FTL does its IC slow calls via specially generated thunks. This was particularly fun to fix: basically, now when we relink an IC call in the FTL, we use the old call target to find the SlowPathCallKey, which tells us everything we need to know to generate (or look up) a new thunk for the new function we want to call. * assembler/MacroAssemblerCodeRef.h: (JSC::MacroAssemblerCodePtr::MacroAssemblerCodePtr): (JSC::MacroAssemblerCodePtr::isEmptyValue): (JSC::MacroAssemblerCodePtr::isDeletedValue): (JSC::MacroAssemblerCodePtr::hash): (JSC::MacroAssemblerCodePtr::emptyValue): (JSC::MacroAssemblerCodePtr::deletedValue): (JSC::MacroAssemblerCodePtrHash::hash): (JSC::MacroAssemblerCodePtrHash::equal): * assembler/MacroAssemblerX86Common.h: * assembler/RepatchBuffer.h: (JSC::RepatchBuffer::RepatchBuffer): (JSC::RepatchBuffer::codeBlock): * ftl/FTLAbbreviations.h: (JSC::FTL::setInstructionCallingConvention): * ftl/FTLCompile.cpp: (JSC::FTL::fixFunctionBasedOnStackMaps): * ftl/FTLInlineCacheSize.cpp: (JSC::FTL::sizeOfGetById): (JSC::FTL::sizeOfPutById): * ftl/FTLJITFinalizer.cpp: (JSC::FTL::JITFinalizer::finalizeFunction): * ftl/FTLLocation.cpp: (JSC::FTL::Location::forStackmaps): * ftl/FTLLocation.h: * ftl/FTLLowerDFGToLLVM.cpp: (JSC::FTL::LowerDFGToLLVM::compileGetById): (JSC::FTL::LowerDFGToLLVM::compilePutById): * ftl/FTLOSRExitCompiler.cpp: (JSC::FTL::compileStub): * ftl/FTLSlowPathCall.cpp: * ftl/FTLSlowPathCallKey.h: (JSC::FTL::SlowPathCallKey::withCallTarget): * ftl/FTLStackMaps.cpp: (JSC::FTL::StackMaps::Location::directGPR): (JSC::FTL::StackMaps::Location::restoreInto): * ftl/FTLStackMaps.h: * ftl/FTLThunks.h: (JSC::FTL::generateIfNecessary): (JSC::FTL::keyForThunk): (JSC::FTL::Thunks::keyForSlowPathCallThunk): * jit/FPRInfo.h: (JSC::FPRInfo::toIndex): * jit/GPRInfo.h: (JSC::GPRInfo::toIndex): (JSC::GPRInfo::debugName): * jit/RegisterSet.cpp: (JSC::RegisterSet::calleeSaveRegisters): * jit/RegisterSet.h: (JSC::RegisterSet::filter): * jit/Repatch.cpp: (JSC::readCallTarget): (JSC::repatchCall): (JSC::repatchByIdSelfAccess): (JSC::tryCacheGetByID): (JSC::tryCachePutByID): (JSC::tryBuildPutByIdList): (JSC::resetGetByID): (JSC::resetPutByID): * jit/ScratchRegisterAllocator.h: (JSC::ScratchRegisterAllocator::lock): Source/WTF: Reviewed by Sam Weinig. I needed to add another set operation, namely filter(), which is an in-place set intersection. * wtf/BitVector.cpp: (WTF::BitVector::filterSlow): * wtf/BitVector.h: (WTF::BitVector::filter): Canonical link: https://commits.webkit.org/142336@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@159039 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2013-11-11 07:30:50 +00:00
FTL should generate code to call slow paths lazily https://bugs.webkit.org/show_bug.cgi?id=149936 Reviewed by Saam Barati. Source/JavaScriptCore: We often have complex slow paths in FTL-generated code. Those slow paths may never run. Even if they do run, they don't need stellar performance. So, it doesn't make sense to have LLVM worry about compiling such slow path code. This patch enables us to use our own MacroAssembler for compiling the slow path inside FTL code. It does this by using a crazy lambda thingy (see FTLLowerDFGToLLVM.cpp's lazySlowPath() and its documentation). The result is quite natural to use. Even for straight slow path calls via something like vmCall(), the lazySlowPath offers the benefit that the call marshalling and the exception checking are not expressed using LLVM IR and do not require LLVM to think about it. It also has the benefit that we never generate the code if it never runs. That's great, since function calls usually involve ~10 instructions total (move arguments to argument registers, make the call, check exception, etc.). This patch adds the lazy slow path abstraction and uses it for some slow paths in the FTL. The code we generate with lazy slow paths is worse than the code that LLVM would have generated. Therefore, a lazy slow path only makes sense when we have strong evidence that the slow path will execute infrequently relative to the fast path. This completely precludes the use of lazy slow paths for out-of-line Nodes that unconditionally call a C++ function. It also precludes their use for the GetByVal out-of-bounds handler, since when we generate a GetByVal with an out-of-bounds handler it means that we only know that the out-of-bounds case executed at least once. So, for all we know, it may actually be the common case. So, this patch just deployed the lazy slow path for GC slow paths and masquerades-as-undefined slow paths. It makes sense for GC slow paths because those have a statistical guarantee of slow path frequency - probably bounded at less than 1/10. It makes sense for masquerades-as- undefined because we can say quite confidently that this is an uncommon scenario on the modern Web. Something that's always been challenging about abstractions involving the MacroAssembler is that linking is a separate phase, and there is no way for someone who is just given access to the MacroAssembler& to emit code that requires linking, since linking happens once we have emitted all code and we are creating the LinkBuffer. Moreover, the FTL requires that the final parts of linking happen on the main thread. This patch ran into this issue, and solved it comprehensively, by introducing MacroAssembler::addLinkTask(). This takes a lambda and runs it at the bitter end of linking - when performFinalization() is called. This ensure that the task added by addLinkTask() runs on the main thread. This patch doesn't replace all of the previously existing idioms for dealing with this issue; we can do that later. This shows small speed-ups on a bunch of things. No big win on any benchmark aggregate. But mainly this is done for https://bugs.webkit.org/show_bug.cgi?id=149852, where we found that outlining the slow path in this way was a significant speed boost. * CMakeLists.txt: * JavaScriptCore.vcxproj/JavaScriptCore.vcxproj: * JavaScriptCore.xcodeproj/project.pbxproj: * assembler/AbstractMacroAssembler.h: (JSC::AbstractMacroAssembler::replaceWithAddressComputation): (JSC::AbstractMacroAssembler::addLinkTask): (JSC::AbstractMacroAssembler::AbstractMacroAssembler): * assembler/LinkBuffer.cpp: (JSC::LinkBuffer::linkCode): (JSC::LinkBuffer::allocate): (JSC::LinkBuffer::performFinalization): * assembler/LinkBuffer.h: (JSC::LinkBuffer::wasAlreadyDisassembled): (JSC::LinkBuffer::didAlreadyDisassemble): (JSC::LinkBuffer::vm): (JSC::LinkBuffer::executableOffsetFor): * bytecode/CodeOrigin.h: (JSC::CodeOrigin::CodeOrigin): (JSC::CodeOrigin::isSet): (JSC::CodeOrigin::operator bool): (JSC::CodeOrigin::isHashTableDeletedValue): (JSC::CodeOrigin::operator!): Deleted. * ftl/FTLCompile.cpp: (JSC::FTL::mmAllocateDataSection): * ftl/FTLInlineCacheDescriptor.h: (JSC::FTL::InlineCacheDescriptor::InlineCacheDescriptor): (JSC::FTL::CheckInDescriptor::CheckInDescriptor): (JSC::FTL::LazySlowPathDescriptor::LazySlowPathDescriptor): * ftl/FTLJITCode.h: * ftl/FTLJITFinalizer.cpp: (JSC::FTL::JITFinalizer::finalizeFunction): * ftl/FTLJITFinalizer.h: * ftl/FTLLazySlowPath.cpp: Added. (JSC::FTL::LazySlowPath::LazySlowPath): (JSC::FTL::LazySlowPath::~LazySlowPath): (JSC::FTL::LazySlowPath::generate): * ftl/FTLLazySlowPath.h: Added. (JSC::FTL::LazySlowPath::createGenerator): (JSC::FTL::LazySlowPath::patchpoint): (JSC::FTL::LazySlowPath::usedRegisters): (JSC::FTL::LazySlowPath::callSiteIndex): (JSC::FTL::LazySlowPath::stub): * ftl/FTLLazySlowPathCall.h: Added. (JSC::FTL::createLazyCallGenerator): * ftl/FTLLowerDFGToLLVM.cpp: (JSC::FTL::DFG::LowerDFGToLLVM::compileCreateActivation): (JSC::FTL::DFG::LowerDFGToLLVM::compileNewFunction): (JSC::FTL::DFG::LowerDFGToLLVM::compileCreateDirectArguments): (JSC::FTL::DFG::LowerDFGToLLVM::compileNewArrayWithSize): (JSC::FTL::DFG::LowerDFGToLLVM::compileMakeRope): (JSC::FTL::DFG::LowerDFGToLLVM::compileNotifyWrite): (JSC::FTL::DFG::LowerDFGToLLVM::compileIsObjectOrNull): (JSC::FTL::DFG::LowerDFGToLLVM::compileIsFunction): (JSC::FTL::DFG::LowerDFGToLLVM::compileIn): (JSC::FTL::DFG::LowerDFGToLLVM::compileMaterializeNewObject): (JSC::FTL::DFG::LowerDFGToLLVM::compileMaterializeCreateActivation): (JSC::FTL::DFG::LowerDFGToLLVM::compileCheckWatchdogTimer): (JSC::FTL::DFG::LowerDFGToLLVM::allocatePropertyStorageWithSizeImpl): (JSC::FTL::DFG::LowerDFGToLLVM::allocateObject): (JSC::FTL::DFG::LowerDFGToLLVM::allocateJSArray): (JSC::FTL::DFG::LowerDFGToLLVM::buildTypeOf): (JSC::FTL::DFG::LowerDFGToLLVM::sensibleDoubleToInt32): (JSC::FTL::DFG::LowerDFGToLLVM::lazySlowPath): (JSC::FTL::DFG::LowerDFGToLLVM::speculate): (JSC::FTL::DFG::LowerDFGToLLVM::emitStoreBarrier): * ftl/FTLOperations.cpp: (JSC::FTL::operationMaterializeObjectInOSR): (JSC::FTL::compileFTLLazySlowPath): * ftl/FTLOperations.h: * ftl/FTLSlowPathCall.cpp: (JSC::FTL::SlowPathCallContext::SlowPathCallContext): (JSC::FTL::SlowPathCallContext::~SlowPathCallContext): (JSC::FTL::SlowPathCallContext::keyWithTarget): (JSC::FTL::SlowPathCallContext::makeCall): (JSC::FTL::callSiteIndexForCodeOrigin): (JSC::FTL::storeCodeOrigin): Deleted. (JSC::FTL::callOperation): Deleted. * ftl/FTLSlowPathCall.h: (JSC::FTL::callOperation): * ftl/FTLState.h: * ftl/FTLThunks.cpp: (JSC::FTL::genericGenerationThunkGenerator): (JSC::FTL::osrExitGenerationThunkGenerator): (JSC::FTL::lazySlowPathGenerationThunkGenerator): (JSC::FTL::registerClobberCheck): * ftl/FTLThunks.h: * interpreter/CallFrame.h: (JSC::CallSiteIndex::CallSiteIndex): (JSC::CallSiteIndex::operator bool): (JSC::CallSiteIndex::bits): * jit/CCallHelpers.h: (JSC::CCallHelpers::setupArgument): (JSC::CCallHelpers::setupArgumentsWithExecState): * jit/JITOperations.cpp: Source/WTF: Enables SharedTask to handle any function type, not just void(). It's probably better to use SharedTask instead of std::function in performance-sensitive code. std::function uses the system malloc and has copy semantics. SharedTask uses FastMalloc and has aliasing semantics. So, you can just trust that it will have sensible performance characteristics. * wtf/ParallelHelperPool.cpp: (WTF::ParallelHelperClient::~ParallelHelperClient): (WTF::ParallelHelperClient::setTask): (WTF::ParallelHelperClient::doSomeHelping): (WTF::ParallelHelperClient::runTaskInParallel): (WTF::ParallelHelperClient::finish): (WTF::ParallelHelperClient::claimTask): (WTF::ParallelHelperClient::runTask): (WTF::ParallelHelperPool::doSomeHelping): (WTF::ParallelHelperPool::helperThreadBody): * wtf/ParallelHelperPool.h: (WTF::ParallelHelperClient::setFunction): (WTF::ParallelHelperClient::runFunctionInParallel): (WTF::ParallelHelperClient::pool): * wtf/SharedTask.h: (WTF::createSharedTask): (WTF::SharedTask::SharedTask): Deleted. (WTF::SharedTask::~SharedTask): Deleted. (WTF::SharedTaskFunctor::SharedTaskFunctor): Deleted. Canonical link: https://commits.webkit.org/168154@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@190860 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2015-10-12 17:56:26 +00:00
m_offsetToSavingArea =
(std::max(m_numArgs, NUMBER_OF_ARGUMENT_REGISTERS) - NUMBER_OF_ARGUMENT_REGISTERS) * wordSize;
FTL should be able to do some simple inline caches using LLVM patchpoints https://bugs.webkit.org/show_bug.cgi?id=123164 Source/JavaScriptCore: Reviewed by Mark Hahnenberg. This implements GetById inline caches in the FTL using llvm.webkit.patchpoint. The idea is that we ask LLVM for a nop slide the size of a GetById inline cache and then fill in the code after LLVM compilation is complete. For now, we just use the system calling convention for the arguments and return. We also still make some assumptions about registers that aren't correct. But, most of the scaffolding is there and this will successfully patch an inline cache. * JavaScriptCore.xcodeproj/project.pbxproj: * assembler/AbstractMacroAssembler.h: * assembler/LinkBuffer.cpp: (JSC::LinkBuffer::finalizeCodeWithoutDisassembly): (JSC::LinkBuffer::linkCode): (JSC::LinkBuffer::allocate): * assembler/LinkBuffer.h: (JSC::LinkBuffer::LinkBuffer): (JSC::LinkBuffer::link): * ftl/FTLAbbreviations.h: (JSC::FTL::constNull): (JSC::FTL::buildCall): * ftl/FTLCapabilities.cpp: (JSC::FTL::canCompile): * ftl/FTLCompile.cpp: (JSC::FTL::fixFunctionBasedOnStackMaps): * ftl/FTLInlineCacheDescriptor.h: Added. (JSC::FTL::InlineCacheDescriptor::InlineCacheDescriptor): (JSC::FTL::GetByIdDescriptor::GetByIdDescriptor): (JSC::FTL::GetByIdDescriptor::stackmapID): (JSC::FTL::GetByIdDescriptor::codeOrigin): (JSC::FTL::GetByIdDescriptor::uid): * ftl/FTLInlineCacheSize.cpp: Added. (JSC::FTL::sizeOfGetById): (JSC::FTL::sizeOfPutById): * ftl/FTLInlineCacheSize.h: Added. * ftl/FTLIntrinsicRepository.h: * ftl/FTLJITFinalizer.cpp: (JSC::FTL::JITFinalizer::finalizeFunction): * ftl/FTLJITFinalizer.h: * ftl/FTLLocation.cpp: (JSC::FTL::Location::directGPR): * ftl/FTLLocation.h: * ftl/FTLLowerDFGToLLVM.cpp: (JSC::FTL::LowerDFGToLLVM::compileGetById): * ftl/FTLOutput.h: (JSC::FTL::Output::call): * ftl/FTLSlowPathCall.cpp: Added. (JSC::FTL::callOperation): * ftl/FTLSlowPathCall.h: Added. (JSC::FTL::SlowPathCall::SlowPathCall): (JSC::FTL::SlowPathCall::call): (JSC::FTL::SlowPathCall::key): * ftl/FTLSlowPathCallKey.cpp: Added. (JSC::FTL::SlowPathCallKey::dump): * ftl/FTLSlowPathCallKey.h: Added. (JSC::FTL::SlowPathCallKey::SlowPathCallKey): (JSC::FTL::SlowPathCallKey::usedRegisters): (JSC::FTL::SlowPathCallKey::callTarget): (JSC::FTL::SlowPathCallKey::offset): (JSC::FTL::SlowPathCallKey::isEmptyValue): (JSC::FTL::SlowPathCallKey::isDeletedValue): (JSC::FTL::SlowPathCallKey::operator==): (JSC::FTL::SlowPathCallKey::hash): (JSC::FTL::SlowPathCallKeyHash::hash): (JSC::FTL::SlowPathCallKeyHash::equal): * ftl/FTLStackMaps.cpp: (JSC::FTL::StackMaps::Location::directGPR): * ftl/FTLStackMaps.h: * ftl/FTLState.h: * ftl/FTLThunks.cpp: (JSC::FTL::slowPathCallThunkGenerator): * ftl/FTLThunks.h: (JSC::FTL::Thunks::getSlowPathCallThunk): * jit/CCallHelpers.h: (JSC::CCallHelpers::setupArguments): * jit/GPRInfo.h: * jit/JITInlineCacheGenerator.cpp: (JSC::garbageStubInfo): (JSC::JITInlineCacheGenerator::JITInlineCacheGenerator): (JSC::JITByIdGenerator::finalize): * jit/JITInlineCacheGenerator.h: (JSC::JITByIdGenerator::slowPathBegin): * jit/RegisterSet.cpp: (JSC::RegisterSet::stackRegisters): (JSC::RegisterSet::specialRegisters): (JSC::RegisterSet::calleeSaveRegisters): (JSC::RegisterSet::allGPRs): (JSC::RegisterSet::allFPRs): (JSC::RegisterSet::allRegisters): (JSC::RegisterSet::dump): * jit/RegisterSet.h: (JSC::RegisterSet::exclude): (JSC::RegisterSet::numberOfSetRegisters): (JSC::RegisterSet::RegisterSet): (JSC::RegisterSet::isEmptyValue): (JSC::RegisterSet::isDeletedValue): (JSC::RegisterSet::operator==): (JSC::RegisterSet::hash): (JSC::RegisterSetHash::hash): (JSC::RegisterSetHash::equal): * runtime/Options.h: Source/WTF: Reviewed by Mark Hahnenberg. This needed some better bitvector support, like merging (|=), excluding (&=~), hashing, and bit counting. * wtf/BitVector.cpp: (WTF::BitVector::setSlow): (WTF::BitVector::excludeSlow): (WTF::BitVector::bitCountSlow): (WTF::BitVector::equalsSlowCase): (WTF::BitVector::hashSlowCase): (WTF::BitVector::dump): * wtf/BitVector.h: (WTF::BitVector::merge): (WTF::BitVector::exclude): (WTF::BitVector::bitCount): (WTF::BitVector::BitVector): (WTF::BitVector::isEmptyValue): (WTF::BitVector::isDeletedValue): (WTF::BitVector::isEmptyOrDeletedValue): (WTF::BitVector::operator==): (WTF::BitVector::hash): (WTF::BitVectorHash::hash): (WTF::BitVectorHash::equal): * wtf/HashTraits.h: (WTF::CustomHashTraits::constructDeletedValue): (WTF::CustomHashTraits::isDeletedValue): (WTF::CustomHashTraits::emptyValue): (WTF::CustomHashTraits::isEmptyValue): * wtf/StdLibExtras.h: (WTF::bitCount): Canonical link: https://commits.webkit.org/141315@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@157872 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2013-10-23 18:22:09 +00:00
FTL should generate code to call slow paths lazily https://bugs.webkit.org/show_bug.cgi?id=149936 Reviewed by Saam Barati. Source/JavaScriptCore: We often have complex slow paths in FTL-generated code. Those slow paths may never run. Even if they do run, they don't need stellar performance. So, it doesn't make sense to have LLVM worry about compiling such slow path code. This patch enables us to use our own MacroAssembler for compiling the slow path inside FTL code. It does this by using a crazy lambda thingy (see FTLLowerDFGToLLVM.cpp's lazySlowPath() and its documentation). The result is quite natural to use. Even for straight slow path calls via something like vmCall(), the lazySlowPath offers the benefit that the call marshalling and the exception checking are not expressed using LLVM IR and do not require LLVM to think about it. It also has the benefit that we never generate the code if it never runs. That's great, since function calls usually involve ~10 instructions total (move arguments to argument registers, make the call, check exception, etc.). This patch adds the lazy slow path abstraction and uses it for some slow paths in the FTL. The code we generate with lazy slow paths is worse than the code that LLVM would have generated. Therefore, a lazy slow path only makes sense when we have strong evidence that the slow path will execute infrequently relative to the fast path. This completely precludes the use of lazy slow paths for out-of-line Nodes that unconditionally call a C++ function. It also precludes their use for the GetByVal out-of-bounds handler, since when we generate a GetByVal with an out-of-bounds handler it means that we only know that the out-of-bounds case executed at least once. So, for all we know, it may actually be the common case. So, this patch just deployed the lazy slow path for GC slow paths and masquerades-as-undefined slow paths. It makes sense for GC slow paths because those have a statistical guarantee of slow path frequency - probably bounded at less than 1/10. It makes sense for masquerades-as- undefined because we can say quite confidently that this is an uncommon scenario on the modern Web. Something that's always been challenging about abstractions involving the MacroAssembler is that linking is a separate phase, and there is no way for someone who is just given access to the MacroAssembler& to emit code that requires linking, since linking happens once we have emitted all code and we are creating the LinkBuffer. Moreover, the FTL requires that the final parts of linking happen on the main thread. This patch ran into this issue, and solved it comprehensively, by introducing MacroAssembler::addLinkTask(). This takes a lambda and runs it at the bitter end of linking - when performFinalization() is called. This ensure that the task added by addLinkTask() runs on the main thread. This patch doesn't replace all of the previously existing idioms for dealing with this issue; we can do that later. This shows small speed-ups on a bunch of things. No big win on any benchmark aggregate. But mainly this is done for https://bugs.webkit.org/show_bug.cgi?id=149852, where we found that outlining the slow path in this way was a significant speed boost. * CMakeLists.txt: * JavaScriptCore.vcxproj/JavaScriptCore.vcxproj: * JavaScriptCore.xcodeproj/project.pbxproj: * assembler/AbstractMacroAssembler.h: (JSC::AbstractMacroAssembler::replaceWithAddressComputation): (JSC::AbstractMacroAssembler::addLinkTask): (JSC::AbstractMacroAssembler::AbstractMacroAssembler): * assembler/LinkBuffer.cpp: (JSC::LinkBuffer::linkCode): (JSC::LinkBuffer::allocate): (JSC::LinkBuffer::performFinalization): * assembler/LinkBuffer.h: (JSC::LinkBuffer::wasAlreadyDisassembled): (JSC::LinkBuffer::didAlreadyDisassemble): (JSC::LinkBuffer::vm): (JSC::LinkBuffer::executableOffsetFor): * bytecode/CodeOrigin.h: (JSC::CodeOrigin::CodeOrigin): (JSC::CodeOrigin::isSet): (JSC::CodeOrigin::operator bool): (JSC::CodeOrigin::isHashTableDeletedValue): (JSC::CodeOrigin::operator!): Deleted. * ftl/FTLCompile.cpp: (JSC::FTL::mmAllocateDataSection): * ftl/FTLInlineCacheDescriptor.h: (JSC::FTL::InlineCacheDescriptor::InlineCacheDescriptor): (JSC::FTL::CheckInDescriptor::CheckInDescriptor): (JSC::FTL::LazySlowPathDescriptor::LazySlowPathDescriptor): * ftl/FTLJITCode.h: * ftl/FTLJITFinalizer.cpp: (JSC::FTL::JITFinalizer::finalizeFunction): * ftl/FTLJITFinalizer.h: * ftl/FTLLazySlowPath.cpp: Added. (JSC::FTL::LazySlowPath::LazySlowPath): (JSC::FTL::LazySlowPath::~LazySlowPath): (JSC::FTL::LazySlowPath::generate): * ftl/FTLLazySlowPath.h: Added. (JSC::FTL::LazySlowPath::createGenerator): (JSC::FTL::LazySlowPath::patchpoint): (JSC::FTL::LazySlowPath::usedRegisters): (JSC::FTL::LazySlowPath::callSiteIndex): (JSC::FTL::LazySlowPath::stub): * ftl/FTLLazySlowPathCall.h: Added. (JSC::FTL::createLazyCallGenerator): * ftl/FTLLowerDFGToLLVM.cpp: (JSC::FTL::DFG::LowerDFGToLLVM::compileCreateActivation): (JSC::FTL::DFG::LowerDFGToLLVM::compileNewFunction): (JSC::FTL::DFG::LowerDFGToLLVM::compileCreateDirectArguments): (JSC::FTL::DFG::LowerDFGToLLVM::compileNewArrayWithSize): (JSC::FTL::DFG::LowerDFGToLLVM::compileMakeRope): (JSC::FTL::DFG::LowerDFGToLLVM::compileNotifyWrite): (JSC::FTL::DFG::LowerDFGToLLVM::compileIsObjectOrNull): (JSC::FTL::DFG::LowerDFGToLLVM::compileIsFunction): (JSC::FTL::DFG::LowerDFGToLLVM::compileIn): (JSC::FTL::DFG::LowerDFGToLLVM::compileMaterializeNewObject): (JSC::FTL::DFG::LowerDFGToLLVM::compileMaterializeCreateActivation): (JSC::FTL::DFG::LowerDFGToLLVM::compileCheckWatchdogTimer): (JSC::FTL::DFG::LowerDFGToLLVM::allocatePropertyStorageWithSizeImpl): (JSC::FTL::DFG::LowerDFGToLLVM::allocateObject): (JSC::FTL::DFG::LowerDFGToLLVM::allocateJSArray): (JSC::FTL::DFG::LowerDFGToLLVM::buildTypeOf): (JSC::FTL::DFG::LowerDFGToLLVM::sensibleDoubleToInt32): (JSC::FTL::DFG::LowerDFGToLLVM::lazySlowPath): (JSC::FTL::DFG::LowerDFGToLLVM::speculate): (JSC::FTL::DFG::LowerDFGToLLVM::emitStoreBarrier): * ftl/FTLOperations.cpp: (JSC::FTL::operationMaterializeObjectInOSR): (JSC::FTL::compileFTLLazySlowPath): * ftl/FTLOperations.h: * ftl/FTLSlowPathCall.cpp: (JSC::FTL::SlowPathCallContext::SlowPathCallContext): (JSC::FTL::SlowPathCallContext::~SlowPathCallContext): (JSC::FTL::SlowPathCallContext::keyWithTarget): (JSC::FTL::SlowPathCallContext::makeCall): (JSC::FTL::callSiteIndexForCodeOrigin): (JSC::FTL::storeCodeOrigin): Deleted. (JSC::FTL::callOperation): Deleted. * ftl/FTLSlowPathCall.h: (JSC::FTL::callOperation): * ftl/FTLState.h: * ftl/FTLThunks.cpp: (JSC::FTL::genericGenerationThunkGenerator): (JSC::FTL::osrExitGenerationThunkGenerator): (JSC::FTL::lazySlowPathGenerationThunkGenerator): (JSC::FTL::registerClobberCheck): * ftl/FTLThunks.h: * interpreter/CallFrame.h: (JSC::CallSiteIndex::CallSiteIndex): (JSC::CallSiteIndex::operator bool): (JSC::CallSiteIndex::bits): * jit/CCallHelpers.h: (JSC::CCallHelpers::setupArgument): (JSC::CCallHelpers::setupArgumentsWithExecState): * jit/JITOperations.cpp: Source/WTF: Enables SharedTask to handle any function type, not just void(). It's probably better to use SharedTask instead of std::function in performance-sensitive code. std::function uses the system malloc and has copy semantics. SharedTask uses FastMalloc and has aliasing semantics. So, you can just trust that it will have sensible performance characteristics. * wtf/ParallelHelperPool.cpp: (WTF::ParallelHelperClient::~ParallelHelperClient): (WTF::ParallelHelperClient::setTask): (WTF::ParallelHelperClient::doSomeHelping): (WTF::ParallelHelperClient::runTaskInParallel): (WTF::ParallelHelperClient::finish): (WTF::ParallelHelperClient::claimTask): (WTF::ParallelHelperClient::runTask): (WTF::ParallelHelperPool::doSomeHelping): (WTF::ParallelHelperPool::helperThreadBody): * wtf/ParallelHelperPool.h: (WTF::ParallelHelperClient::setFunction): (WTF::ParallelHelperClient::runFunctionInParallel): (WTF::ParallelHelperClient::pool): * wtf/SharedTask.h: (WTF::createSharedTask): (WTF::SharedTask::SharedTask): Deleted. (WTF::SharedTask::~SharedTask): Deleted. (WTF::SharedTaskFunctor::SharedTaskFunctor): Deleted. Canonical link: https://commits.webkit.org/168154@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@190860 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2015-10-12 17:56:26 +00:00
for (unsigned i = std::min(NUMBER_OF_ARGUMENT_REGISTERS, numArgs); i--;)
m_argumentRegisters.set(GPRInfo::toArgumentRegister(i));
m_callingConventionRegisters.merge(m_argumentRegisters);
if (returnRegister != InvalidGPRReg)
m_callingConventionRegisters.set(GPRInfo::returnValueGPR);
[JSC] Use DataIC for AccessCase https://bugs.webkit.org/show_bug.cgi?id=226072 Reviewed by Saam Barati and Filip Pizlo. Source/JavaScriptCore: This patch adds non-repatching IC for Baseline JIT in ARM64. This does not work in non-ARM64 architectures (including X64) due to the use of link-register. 1. We add non-repatching IC, which is enabled only in Baseline due to performance reason. We are using the existing IC in DFG and FTL. Non-repatching includes fast-path, and slow-path's operation function. 2. We still keep InlineAccess in all tiers. Removing that causes 0.3 ~ 1.0% regression in Speedometer2. This means that we still need some repatching when we first introduce stubs. 3. We add a mechanism to share generated code stubs in non-repatching IC. Currently, getter / setter / custom accessors are excluded since their code relies on JSGlobalObject, CodeBlock etc. which are not included in AccessCase's data structure. 4. This patch still relies on that CodeBlock will be destroyed synchronously since we need to ensure that sharing-hash-table does not include already-dead JIT code stubs. We can fix it (e.g. annotating epoch to these stubs, bump them in finalizeUnconditionally), but we avoid doing that to prevent from further enlarging this patch. This patch is already significant in its size. 5. Added callOperation(Address) support in CCallHelpers, which can save the target in nonArgGPR0, and call it so that we can use Address including GPR which is also used for arguments. Performance is neutral in JetStream2 and Speedometer2. But it offers the way to remove some code generation in Baseline. * assembler/MacroAssemblerARM64E.h: (JSC::MacroAssemblerARM64E::call): * bytecode/AccessCase.cpp: (JSC::AccessCase::create): (JSC::AccessCase::createTransition): (JSC::AccessCase::createDelete): (JSC::AccessCase::createCheckPrivateBrand): (JSC::AccessCase::createSetPrivateBrand): (JSC::AccessCase::fromStructureStubInfo): (JSC::AccessCase::clone const): (JSC::AccessCase::generateWithGuard): (JSC::AccessCase::generateImpl): (JSC::AccessCase::canBeShared): * bytecode/AccessCase.h: (JSC::AccessCase::hash const): (JSC::AccessCase::AccessCase): (JSC::SharedJITStubSet::Hash::Key::Key): (JSC::SharedJITStubSet::Hash::Key::isHashTableDeletedValue const): (JSC::SharedJITStubSet::Hash::Key::operator==): (JSC::SharedJITStubSet::Hash::hash): (JSC::SharedJITStubSet::Hash::equal): (JSC::SharedJITStubSet::Searcher::Translator::hash): (JSC::SharedJITStubSet::Searcher::Translator::equal): (JSC::SharedJITStubSet::PointerTranslator::hash): (JSC::SharedJITStubSet::PointerTranslator::equal): (JSC::SharedJITStubSet::add): (JSC::SharedJITStubSet::remove): (JSC::SharedJITStubSet::find): * bytecode/ByValInfo.h: (JSC::ByValInfo::setUp): (JSC::ByValInfo::offsetOfSlowOperation): (JSC::ByValInfo::offsetOfNotIndexJumpTarget): (JSC::ByValInfo::offsetOfBadTypeJumpTarget): * bytecode/CallLinkInfo.cpp: (JSC::CallLinkInfo::initializeDirectCall): (JSC::CallLinkInfo::setDirectCallTarget): * bytecode/CodeBlock.h: (JSC::CodeBlock::useDataIC const): * bytecode/GetterSetterAccessCase.cpp: (JSC::GetterSetterAccessCase::create): (JSC::GetterSetterAccessCase::clone const): (JSC::GetterSetterAccessCase::emitDOMJITGetter): * bytecode/GetterSetterAccessCase.h: * bytecode/InlineAccess.cpp: (JSC::getScratchRegister): (JSC::InlineAccess::rewireStubAsJumpInAccessNotUsingInlineAccess): (JSC::InlineAccess::rewireStubAsJumpInAccess): (JSC::InlineAccess::resetStubAsJumpInAccess): (JSC::InlineAccess::resetStubAsJumpInAccessNotUsingInlineAccess): (JSC::InlineAccess::rewireStubAsJump): Deleted. * bytecode/InlineAccess.h: * bytecode/InstanceOfAccessCase.cpp: (JSC::InstanceOfAccessCase::create): (JSC::InstanceOfAccessCase::clone const): * bytecode/InstanceOfAccessCase.h: * bytecode/IntrinsicGetterAccessCase.cpp: (JSC::IntrinsicGetterAccessCase::create): (JSC::IntrinsicGetterAccessCase::clone const): * bytecode/IntrinsicGetterAccessCase.h: * bytecode/ModuleNamespaceAccessCase.cpp: (JSC::ModuleNamespaceAccessCase::create): (JSC::ModuleNamespaceAccessCase::clone const): * bytecode/ModuleNamespaceAccessCase.h: * bytecode/ObjectPropertyConditionSet.h: (JSC::ObjectPropertyConditionSet::hash const): (JSC::ObjectPropertyConditionSet::operator==): (JSC::ObjectPropertyConditionSet::operator!=): * bytecode/PolymorphicAccess.cpp: (JSC::AccessGenerationState::installWatchpoint): (JSC::AccessGenerationState::succeed): (JSC::AccessGenerationState::preserveLiveRegistersToStackForCallWithoutExceptions): (JSC::PolymorphicAccess::addCases): (JSC::PolymorphicAccess::addCase): (JSC::PolymorphicAccess::visitWeak const): (JSC::PolymorphicAccess::regenerate): * bytecode/PolymorphicAccess.h: (JSC::AccessGenerationState::AccessGenerationState): (JSC::AccessGenerationState::preserveLiveRegistersToStackForCallWithoutExceptions): * bytecode/ProxyableAccessCase.cpp: (JSC::ProxyableAccessCase::create): (JSC::ProxyableAccessCase::clone const): * bytecode/ProxyableAccessCase.h: * bytecode/StructureStubInfo.cpp: (JSC::StructureStubInfo::StructureStubInfo): (JSC::StructureStubInfo::addAccessCase): (JSC::StructureStubInfo::reset): * bytecode/StructureStubInfo.h: (JSC::StructureStubInfo::offsetOfCodePtr): (JSC::StructureStubInfo::offsetOfSlowPathStartLocation): (JSC::StructureStubInfo::offsetOfSlowOperation): (JSC::StructureStubInfo::patchableJump): Deleted. * dfg/DFGJITCompiler.h: (JSC::DFG::JITCompiler::appendCall): * dfg/DFGSlowPathGenerator.h: (JSC::DFG::slowPathICCall): * dfg/DFGSpeculativeJIT.cpp: (JSC::DFG::SpeculativeJIT::compileGetById): (JSC::DFG::SpeculativeJIT::compileGetByIdFlush): (JSC::DFG::SpeculativeJIT::compileDeleteById): (JSC::DFG::SpeculativeJIT::compileDeleteByVal): (JSC::DFG::SpeculativeJIT::compileInById): (JSC::DFG::SpeculativeJIT::compileInByVal): (JSC::DFG::SpeculativeJIT::compileGetPrivateNameByVal): (JSC::DFG::SpeculativeJIT::compileGetPrivateNameById): (JSC::DFG::SpeculativeJIT::compilePutPrivateNameById): (JSC::DFG::SpeculativeJIT::compileCheckPrivateBrand): (JSC::DFG::SpeculativeJIT::compileSetPrivateBrand): (JSC::DFG::SpeculativeJIT::compileInstanceOfForCells): (JSC::DFG::SpeculativeJIT::compileInstanceOf): (JSC::DFG::SpeculativeJIT::compilePutByIdFlush): (JSC::DFG::SpeculativeJIT::compilePutById): (JSC::DFG::SpeculativeJIT::compilePutByIdDirect): (JSC::DFG::SpeculativeJIT::cachedPutById): * dfg/DFGSpeculativeJIT.h: (JSC::DFG::SpeculativeJIT::callOperation): (JSC::DFG::SpeculativeJIT::appendCall): (JSC::DFG::SpeculativeJIT::appendCallSetResult): * dfg/DFGSpeculativeJIT32_64.cpp: (JSC::DFG::SpeculativeJIT::cachedGetById): (JSC::DFG::SpeculativeJIT::cachedGetByIdWithThis): (JSC::DFG::SpeculativeJIT::compile): * dfg/DFGSpeculativeJIT64.cpp: (JSC::DFG::SpeculativeJIT::cachedGetById): (JSC::DFG::SpeculativeJIT::cachedGetByIdWithThis): (JSC::DFG::SpeculativeJIT::compile): * ftl/FTLLowerDFGToB3.cpp: (JSC::FTL::DFG::LowerDFGToB3::getPrivateName): (JSC::FTL::DFG::LowerDFGToB3::compilePrivateBrandAccess): (JSC::FTL::DFG::LowerDFGToB3::cachedPutById): (JSC::FTL::DFG::LowerDFGToB3::compileGetByVal): (JSC::FTL::DFG::LowerDFGToB3::compileDelBy): (JSC::FTL::DFG::LowerDFGToB3::compileCompareStrictEq): * ftl/FTLSlowPathCall.cpp: (JSC::FTL::SlowPathCallContext::SlowPathCallContext): (JSC::FTL::SlowPathCallContext::keyWithTarget const): (JSC::FTL::SlowPathCallContext::makeCall): * ftl/FTLSlowPathCall.h: (JSC::FTL::callOperation): * ftl/FTLSlowPathCallKey.cpp: (JSC::FTL::SlowPathCallKey::dump const): * ftl/FTLSlowPathCallKey.h: (JSC::FTL::SlowPathCallKey::SlowPathCallKey): (JSC::FTL::SlowPathCallKey::indirectOffset const): (JSC::FTL::SlowPathCallKey::withCallTarget): (JSC::FTL::SlowPathCallKey::operator== const): (JSC::FTL::SlowPathCallKey::hash const): * ftl/FTLThunks.cpp: (JSC::FTL::genericGenerationThunkGenerator): (JSC::FTL::slowPathCallThunkGenerator): * jit/AssemblyHelpers.cpp: (JSC::AssemblyHelpers::emitVirtualCall): * jit/CCallHelpers.cpp: (JSC::CCallHelpers::emitJITCodeOver): * jit/CCallHelpers.h: (JSC::CCallHelpers::ArgCollection::ArgCollection): (JSC::CCallHelpers::ArgCollection::pushRegArg): (JSC::CCallHelpers::ArgCollection::pushExtraRegArg): (JSC::CCallHelpers::ArgCollection::pushNonArg): (JSC::CCallHelpers::ArgCollection::addGPRArg): (JSC::CCallHelpers::ArgCollection::addGPRExtraArg): (JSC::CCallHelpers::ArgCollection::addStackArg): (JSC::CCallHelpers::ArgCollection::addPoke): (JSC::CCallHelpers::calculatePokeOffset): (JSC::CCallHelpers::pokeForArgument): (JSC::CCallHelpers::stackAligned): (JSC::CCallHelpers::marshallArgumentRegister): (JSC::CCallHelpers::setupArgumentsImpl): (JSC::CCallHelpers::pokeArgumentsAligned): (JSC::CCallHelpers::std::is_integral<CURRENT_ARGUMENT_TYPE>::value): (JSC::CCallHelpers::std::is_pointer<CURRENT_ARGUMENT_TYPE>::value): (JSC::CCallHelpers::setupArgumentsEntryImpl): (JSC::CCallHelpers::setupArguments): (JSC::CCallHelpers::setupArgumentsForIndirectCall): * jit/GCAwareJITStubRoutine.cpp: (JSC::GCAwareJITStubRoutine::GCAwareJITStubRoutine): (JSC::GCAwareJITStubRoutine::makeGCAware): (JSC::GCAwareJITStubRoutine::observeZeroRefCount): (JSC::PolymorphicAccessJITStubRoutine::PolymorphicAccessJITStubRoutine): (JSC::PolymorphicAccessJITStubRoutine::observeZeroRefCount): (JSC::PolymorphicAccessJITStubRoutine::computeHash): (JSC::MarkingGCAwareJITStubRoutine::MarkingGCAwareJITStubRoutine): (JSC::GCAwareJITStubRoutineWithExceptionHandler::GCAwareJITStubRoutineWithExceptionHandler): (JSC::createICJITStubRoutine): (JSC::createJITStubRoutine): Deleted. * jit/GCAwareJITStubRoutine.h: (JSC::GCAwareJITStubRoutine::create): (JSC::PolymorphicAccessJITStubRoutine::cases const): (JSC::PolymorphicAccessJITStubRoutine::weakStructures const): (JSC::PolymorphicAccessJITStubRoutine::hash const): * jit/GPRInfo.h: * jit/JIT.cpp: (JSC::JIT::link): * jit/JIT.h: * jit/JITCall.cpp: (JSC::JIT::emit_op_iterator_open): (JSC::JIT::emitSlow_op_iterator_open): (JSC::JIT::emit_op_iterator_next): (JSC::JIT::emitSlow_op_iterator_next): * jit/JITCall32_64.cpp: (JSC::JIT::emit_op_iterator_open): (JSC::JIT::emit_op_iterator_next): * jit/JITCode.h: (JSC::JITCode::useDataIC): * jit/JITInlineCacheGenerator.cpp: (JSC::JITInlineCacheGenerator::JITInlineCacheGenerator): (JSC::JITInlineCacheGenerator::finalize): (JSC::JITByIdGenerator::JITByIdGenerator): (JSC::JITByIdGenerator::finalize): (JSC::JITByIdGenerator::generateFastCommon): (JSC::JITGetByIdGenerator::JITGetByIdGenerator): (JSC::JITGetByIdWithThisGenerator::JITGetByIdWithThisGenerator): (JSC::JITPutByIdGenerator::JITPutByIdGenerator): (JSC::JITDelByValGenerator::JITDelByValGenerator): (JSC::JITDelByValGenerator::generateFastPath): (JSC::JITDelByValGenerator::finalize): (JSC::JITDelByIdGenerator::JITDelByIdGenerator): (JSC::JITDelByIdGenerator::generateFastPath): (JSC::JITDelByIdGenerator::finalize): (JSC::JITInByValGenerator::JITInByValGenerator): (JSC::JITInByValGenerator::generateFastPath): (JSC::JITInByValGenerator::finalize): (JSC::JITInByIdGenerator::JITInByIdGenerator): (JSC::JITInstanceOfGenerator::JITInstanceOfGenerator): (JSC::JITInstanceOfGenerator::generateFastPath): (JSC::JITInstanceOfGenerator::finalize): (JSC::JITGetByValGenerator::JITGetByValGenerator): (JSC::JITGetByValGenerator::generateFastPath): (JSC::JITGetByValGenerator::finalize): (JSC::JITPrivateBrandAccessGenerator::JITPrivateBrandAccessGenerator): (JSC::JITPrivateBrandAccessGenerator::generateFastPath): (JSC::JITPrivateBrandAccessGenerator::finalize): * jit/JITInlineCacheGenerator.h: (JSC::JITGetByIdGenerator::JITGetByIdGenerator): Deleted. (JSC::JITGetByIdWithThisGenerator::JITGetByIdWithThisGenerator): Deleted. (JSC::JITPutByIdGenerator::JITPutByIdGenerator): Deleted. (JSC::JITDelByValGenerator::JITDelByValGenerator): Deleted. (JSC::JITDelByValGenerator::slowPathJump const): Deleted. (JSC::JITDelByIdGenerator::JITDelByIdGenerator): Deleted. (JSC::JITDelByIdGenerator::slowPathJump const): Deleted. (JSC::JITInByIdGenerator::JITInByIdGenerator): Deleted. (JSC::JITInstanceOfGenerator::JITInstanceOfGenerator): Deleted. (JSC::JITGetByValGenerator::JITGetByValGenerator): Deleted. (JSC::JITGetByValGenerator::slowPathJump const): Deleted. (JSC::JITPrivateBrandAccessGenerator::JITPrivateBrandAccessGenerator): Deleted. (JSC::JITPrivateBrandAccessGenerator::slowPathJump const): Deleted. * jit/JITInlines.h: (JSC::JIT::emitLoadForArrayMode): (JSC::JIT::appendCallWithExceptionCheck): (JSC::JIT::appendCallWithExceptionCheckSetJSValueResult): (JSC::JIT::appendCallWithExceptionCheckSetJSValueResultWithProfile): * jit/JITOpcodes.cpp: (JSC::JIT::emit_op_instanceof): (JSC::JIT::emitSlow_op_instanceof): (JSC::JIT::privateCompileHasIndexedProperty): (JSC::JIT::emit_op_has_enumerable_indexed_property): (JSC::JIT::emitSlow_op_has_enumerable_indexed_property): * jit/JITOpcodes32_64.cpp: (JSC::JIT::emit_op_instanceof): (JSC::JIT::privateCompileHasIndexedProperty): (JSC::JIT::emit_op_has_enumerable_indexed_property): * jit/JITOperations.cpp: (JSC::JSC_DEFINE_JIT_OPERATION): * jit/JITPropertyAccess.cpp: (JSC::JIT::emit_op_get_by_val): (JSC::JIT::emitSlow_op_get_by_val): (JSC::JIT::slow_op_get_by_val_prepareCallGenerator): (JSC::JIT::emit_op_get_private_name): (JSC::JIT::emitSlow_op_get_private_name): (JSC::JIT::slow_op_get_private_name_prepareCallGenerator): (JSC::JIT::emit_op_set_private_brand): (JSC::JIT::emitSlow_op_set_private_brand): (JSC::JIT::emit_op_check_private_brand): (JSC::JIT::emitSlow_op_check_private_brand): (JSC::JIT::emit_op_put_by_val): (JSC::JIT::emitGenericContiguousPutByVal): (JSC::JIT::emitArrayStoragePutByVal): (JSC::JIT::emitPutByValWithCachedId): (JSC::JIT::emitSlow_op_put_by_val): (JSC::JIT::slow_op_put_by_val_prepareCallGenerator): (JSC::JIT::emit_op_put_private_name): (JSC::JIT::emitSlow_op_put_private_name): (JSC::JIT::slow_op_put_private_name_prepareCallGenerator): (JSC::JIT::emit_op_del_by_id): (JSC::JIT::emitSlow_op_del_by_id): (JSC::JIT::slow_op_del_by_id_prepareCallGenerator): (JSC::JIT::emit_op_del_by_val): (JSC::JIT::emitSlow_op_del_by_val): (JSC::JIT::slow_op_del_by_val_prepareCallGenerator): (JSC::JIT::emit_op_try_get_by_id): (JSC::JIT::emitSlow_op_try_get_by_id): (JSC::JIT::emit_op_get_by_id_direct): (JSC::JIT::emitSlow_op_get_by_id_direct): (JSC::JIT::emit_op_get_by_id): (JSC::JIT::emit_op_get_by_id_with_this): (JSC::JIT::emitSlow_op_get_by_id): (JSC::JIT::slow_op_get_by_id_prepareCallGenerator): (JSC::JIT::emitSlow_op_get_by_id_with_this): (JSC::JIT::slow_op_get_by_id_with_this_prepareCallGenerator): (JSC::JIT::emit_op_put_by_id): (JSC::JIT::emitSlow_op_put_by_id): (JSC::JIT::slow_op_put_by_id_prepareCallGenerator): (JSC::JIT::emit_op_in_by_id): (JSC::JIT::emitSlow_op_in_by_id): (JSC::JIT::emit_op_in_by_val): (JSC::JIT::emitSlow_op_in_by_val): (JSC::JIT::privateCompilePutByVal): (JSC::JIT::privateCompilePutPrivateNameWithCachedId): (JSC::JIT::privateCompilePutByValWithCachedId): (JSC::JIT::emitDoubleLoad): (JSC::JIT::emitContiguousLoad): (JSC::JIT::emitArrayStorageLoad): (JSC::JIT::emitIntTypedArrayPutByVal): (JSC::JIT::emitFloatTypedArrayPutByVal): * jit/JITPropertyAccess32_64.cpp: (JSC::JIT::emit_op_del_by_id): (JSC::JIT::emit_op_del_by_val): (JSC::JIT::emit_op_get_by_val): (JSC::JIT::emit_op_get_private_name): (JSC::JIT::emit_op_set_private_brand): (JSC::JIT::emit_op_check_private_brand): (JSC::JIT::emit_op_put_by_val): (JSC::JIT::emitGenericContiguousPutByVal): (JSC::JIT::emitArrayStoragePutByVal): (JSC::JIT::emit_op_try_get_by_id): (JSC::JIT::emit_op_get_by_id_direct): (JSC::JIT::emit_op_get_by_id): (JSC::JIT::emit_op_get_by_id_with_this): (JSC::JIT::emit_op_put_by_id): (JSC::JIT::emit_op_in_by_id): (JSC::JIT::emit_op_in_by_val): * jit/JITStubRoutine.h: * jit/PolymorphicCallStubRoutine.cpp: (JSC::PolymorphicCallStubRoutine::PolymorphicCallStubRoutine): * jit/Repatch.cpp: (JSC::readPutICCallTarget): (JSC::repatchSlowPathCall): (JSC::tryCacheGetBy): (JSC::repatchGetBy): (JSC::tryCacheArrayGetByVal): (JSC::repatchArrayGetByVal): (JSC::tryCachePutByID): (JSC::repatchPutByID): (JSC::tryCacheDeleteBy): (JSC::repatchDeleteBy): (JSC::tryCacheInBy): (JSC::repatchInBy): (JSC::tryCacheCheckPrivateBrand): (JSC::repatchCheckPrivateBrand): (JSC::tryCacheSetPrivateBrand): (JSC::repatchSetPrivateBrand): (JSC::tryCacheInstanceOf): (JSC::repatchInstanceOf): (JSC::linkSlowFor): (JSC::linkVirtualFor): (JSC::resetGetBy): (JSC::resetPutByID): (JSC::resetDelBy): (JSC::resetInBy): (JSC::resetInstanceOf): (JSC::resetCheckPrivateBrand): (JSC::resetSetPrivateBrand): (JSC::resetPatchableJump): Deleted. * jit/Repatch.h: * runtime/Options.cpp: (JSC::Options::recomputeDependentOptions): * runtime/OptionsList.h: * runtime/StructureIDTable.h: * runtime/VM.cpp: (JSC::VM::VM): (JSC::VM::~VM): * runtime/VM.h: Tools: * Scripts/run-jsc-stress-tests: Canonical link: https://commits.webkit.org/238638@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@278656 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-06-09 11:17:36 +00:00
if (indirectCallTargetRegister != InvalidGPRReg)
m_callingConventionRegisters.set(indirectCallTargetRegister);
FTL should generate code to call slow paths lazily https://bugs.webkit.org/show_bug.cgi?id=149936 Reviewed by Saam Barati. Source/JavaScriptCore: We often have complex slow paths in FTL-generated code. Those slow paths may never run. Even if they do run, they don't need stellar performance. So, it doesn't make sense to have LLVM worry about compiling such slow path code. This patch enables us to use our own MacroAssembler for compiling the slow path inside FTL code. It does this by using a crazy lambda thingy (see FTLLowerDFGToLLVM.cpp's lazySlowPath() and its documentation). The result is quite natural to use. Even for straight slow path calls via something like vmCall(), the lazySlowPath offers the benefit that the call marshalling and the exception checking are not expressed using LLVM IR and do not require LLVM to think about it. It also has the benefit that we never generate the code if it never runs. That's great, since function calls usually involve ~10 instructions total (move arguments to argument registers, make the call, check exception, etc.). This patch adds the lazy slow path abstraction and uses it for some slow paths in the FTL. The code we generate with lazy slow paths is worse than the code that LLVM would have generated. Therefore, a lazy slow path only makes sense when we have strong evidence that the slow path will execute infrequently relative to the fast path. This completely precludes the use of lazy slow paths for out-of-line Nodes that unconditionally call a C++ function. It also precludes their use for the GetByVal out-of-bounds handler, since when we generate a GetByVal with an out-of-bounds handler it means that we only know that the out-of-bounds case executed at least once. So, for all we know, it may actually be the common case. So, this patch just deployed the lazy slow path for GC slow paths and masquerades-as-undefined slow paths. It makes sense for GC slow paths because those have a statistical guarantee of slow path frequency - probably bounded at less than 1/10. It makes sense for masquerades-as- undefined because we can say quite confidently that this is an uncommon scenario on the modern Web. Something that's always been challenging about abstractions involving the MacroAssembler is that linking is a separate phase, and there is no way for someone who is just given access to the MacroAssembler& to emit code that requires linking, since linking happens once we have emitted all code and we are creating the LinkBuffer. Moreover, the FTL requires that the final parts of linking happen on the main thread. This patch ran into this issue, and solved it comprehensively, by introducing MacroAssembler::addLinkTask(). This takes a lambda and runs it at the bitter end of linking - when performFinalization() is called. This ensure that the task added by addLinkTask() runs on the main thread. This patch doesn't replace all of the previously existing idioms for dealing with this issue; we can do that later. This shows small speed-ups on a bunch of things. No big win on any benchmark aggregate. But mainly this is done for https://bugs.webkit.org/show_bug.cgi?id=149852, where we found that outlining the slow path in this way was a significant speed boost. * CMakeLists.txt: * JavaScriptCore.vcxproj/JavaScriptCore.vcxproj: * JavaScriptCore.xcodeproj/project.pbxproj: * assembler/AbstractMacroAssembler.h: (JSC::AbstractMacroAssembler::replaceWithAddressComputation): (JSC::AbstractMacroAssembler::addLinkTask): (JSC::AbstractMacroAssembler::AbstractMacroAssembler): * assembler/LinkBuffer.cpp: (JSC::LinkBuffer::linkCode): (JSC::LinkBuffer::allocate): (JSC::LinkBuffer::performFinalization): * assembler/LinkBuffer.h: (JSC::LinkBuffer::wasAlreadyDisassembled): (JSC::LinkBuffer::didAlreadyDisassemble): (JSC::LinkBuffer::vm): (JSC::LinkBuffer::executableOffsetFor): * bytecode/CodeOrigin.h: (JSC::CodeOrigin::CodeOrigin): (JSC::CodeOrigin::isSet): (JSC::CodeOrigin::operator bool): (JSC::CodeOrigin::isHashTableDeletedValue): (JSC::CodeOrigin::operator!): Deleted. * ftl/FTLCompile.cpp: (JSC::FTL::mmAllocateDataSection): * ftl/FTLInlineCacheDescriptor.h: (JSC::FTL::InlineCacheDescriptor::InlineCacheDescriptor): (JSC::FTL::CheckInDescriptor::CheckInDescriptor): (JSC::FTL::LazySlowPathDescriptor::LazySlowPathDescriptor): * ftl/FTLJITCode.h: * ftl/FTLJITFinalizer.cpp: (JSC::FTL::JITFinalizer::finalizeFunction): * ftl/FTLJITFinalizer.h: * ftl/FTLLazySlowPath.cpp: Added. (JSC::FTL::LazySlowPath::LazySlowPath): (JSC::FTL::LazySlowPath::~LazySlowPath): (JSC::FTL::LazySlowPath::generate): * ftl/FTLLazySlowPath.h: Added. (JSC::FTL::LazySlowPath::createGenerator): (JSC::FTL::LazySlowPath::patchpoint): (JSC::FTL::LazySlowPath::usedRegisters): (JSC::FTL::LazySlowPath::callSiteIndex): (JSC::FTL::LazySlowPath::stub): * ftl/FTLLazySlowPathCall.h: Added. (JSC::FTL::createLazyCallGenerator): * ftl/FTLLowerDFGToLLVM.cpp: (JSC::FTL::DFG::LowerDFGToLLVM::compileCreateActivation): (JSC::FTL::DFG::LowerDFGToLLVM::compileNewFunction): (JSC::FTL::DFG::LowerDFGToLLVM::compileCreateDirectArguments): (JSC::FTL::DFG::LowerDFGToLLVM::compileNewArrayWithSize): (JSC::FTL::DFG::LowerDFGToLLVM::compileMakeRope): (JSC::FTL::DFG::LowerDFGToLLVM::compileNotifyWrite): (JSC::FTL::DFG::LowerDFGToLLVM::compileIsObjectOrNull): (JSC::FTL::DFG::LowerDFGToLLVM::compileIsFunction): (JSC::FTL::DFG::LowerDFGToLLVM::compileIn): (JSC::FTL::DFG::LowerDFGToLLVM::compileMaterializeNewObject): (JSC::FTL::DFG::LowerDFGToLLVM::compileMaterializeCreateActivation): (JSC::FTL::DFG::LowerDFGToLLVM::compileCheckWatchdogTimer): (JSC::FTL::DFG::LowerDFGToLLVM::allocatePropertyStorageWithSizeImpl): (JSC::FTL::DFG::LowerDFGToLLVM::allocateObject): (JSC::FTL::DFG::LowerDFGToLLVM::allocateJSArray): (JSC::FTL::DFG::LowerDFGToLLVM::buildTypeOf): (JSC::FTL::DFG::LowerDFGToLLVM::sensibleDoubleToInt32): (JSC::FTL::DFG::LowerDFGToLLVM::lazySlowPath): (JSC::FTL::DFG::LowerDFGToLLVM::speculate): (JSC::FTL::DFG::LowerDFGToLLVM::emitStoreBarrier): * ftl/FTLOperations.cpp: (JSC::FTL::operationMaterializeObjectInOSR): (JSC::FTL::compileFTLLazySlowPath): * ftl/FTLOperations.h: * ftl/FTLSlowPathCall.cpp: (JSC::FTL::SlowPathCallContext::SlowPathCallContext): (JSC::FTL::SlowPathCallContext::~SlowPathCallContext): (JSC::FTL::SlowPathCallContext::keyWithTarget): (JSC::FTL::SlowPathCallContext::makeCall): (JSC::FTL::callSiteIndexForCodeOrigin): (JSC::FTL::storeCodeOrigin): Deleted. (JSC::FTL::callOperation): Deleted. * ftl/FTLSlowPathCall.h: (JSC::FTL::callOperation): * ftl/FTLState.h: * ftl/FTLThunks.cpp: (JSC::FTL::genericGenerationThunkGenerator): (JSC::FTL::osrExitGenerationThunkGenerator): (JSC::FTL::lazySlowPathGenerationThunkGenerator): (JSC::FTL::registerClobberCheck): * ftl/FTLThunks.h: * interpreter/CallFrame.h: (JSC::CallSiteIndex::CallSiteIndex): (JSC::CallSiteIndex::operator bool): (JSC::CallSiteIndex::bits): * jit/CCallHelpers.h: (JSC::CCallHelpers::setupArgument): (JSC::CCallHelpers::setupArgumentsWithExecState): * jit/JITOperations.cpp: Source/WTF: Enables SharedTask to handle any function type, not just void(). It's probably better to use SharedTask instead of std::function in performance-sensitive code. std::function uses the system malloc and has copy semantics. SharedTask uses FastMalloc and has aliasing semantics. So, you can just trust that it will have sensible performance characteristics. * wtf/ParallelHelperPool.cpp: (WTF::ParallelHelperClient::~ParallelHelperClient): (WTF::ParallelHelperClient::setTask): (WTF::ParallelHelperClient::doSomeHelping): (WTF::ParallelHelperClient::runTaskInParallel): (WTF::ParallelHelperClient::finish): (WTF::ParallelHelperClient::claimTask): (WTF::ParallelHelperClient::runTask): (WTF::ParallelHelperPool::doSomeHelping): (WTF::ParallelHelperPool::helperThreadBody): * wtf/ParallelHelperPool.h: (WTF::ParallelHelperClient::setFunction): (WTF::ParallelHelperClient::runFunctionInParallel): (WTF::ParallelHelperClient::pool): * wtf/SharedTask.h: (WTF::createSharedTask): (WTF::SharedTask::SharedTask): Deleted. (WTF::SharedTask::~SharedTask): Deleted. (WTF::SharedTaskFunctor::SharedTaskFunctor): Deleted. Canonical link: https://commits.webkit.org/168154@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@190860 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2015-10-12 17:56:26 +00:00
m_callingConventionRegisters.filter(usedRegisters);
FTL should be able to do some simple inline caches using LLVM patchpoints https://bugs.webkit.org/show_bug.cgi?id=123164 Source/JavaScriptCore: Reviewed by Mark Hahnenberg. This implements GetById inline caches in the FTL using llvm.webkit.patchpoint. The idea is that we ask LLVM for a nop slide the size of a GetById inline cache and then fill in the code after LLVM compilation is complete. For now, we just use the system calling convention for the arguments and return. We also still make some assumptions about registers that aren't correct. But, most of the scaffolding is there and this will successfully patch an inline cache. * JavaScriptCore.xcodeproj/project.pbxproj: * assembler/AbstractMacroAssembler.h: * assembler/LinkBuffer.cpp: (JSC::LinkBuffer::finalizeCodeWithoutDisassembly): (JSC::LinkBuffer::linkCode): (JSC::LinkBuffer::allocate): * assembler/LinkBuffer.h: (JSC::LinkBuffer::LinkBuffer): (JSC::LinkBuffer::link): * ftl/FTLAbbreviations.h: (JSC::FTL::constNull): (JSC::FTL::buildCall): * ftl/FTLCapabilities.cpp: (JSC::FTL::canCompile): * ftl/FTLCompile.cpp: (JSC::FTL::fixFunctionBasedOnStackMaps): * ftl/FTLInlineCacheDescriptor.h: Added. (JSC::FTL::InlineCacheDescriptor::InlineCacheDescriptor): (JSC::FTL::GetByIdDescriptor::GetByIdDescriptor): (JSC::FTL::GetByIdDescriptor::stackmapID): (JSC::FTL::GetByIdDescriptor::codeOrigin): (JSC::FTL::GetByIdDescriptor::uid): * ftl/FTLInlineCacheSize.cpp: Added. (JSC::FTL::sizeOfGetById): (JSC::FTL::sizeOfPutById): * ftl/FTLInlineCacheSize.h: Added. * ftl/FTLIntrinsicRepository.h: * ftl/FTLJITFinalizer.cpp: (JSC::FTL::JITFinalizer::finalizeFunction): * ftl/FTLJITFinalizer.h: * ftl/FTLLocation.cpp: (JSC::FTL::Location::directGPR): * ftl/FTLLocation.h: * ftl/FTLLowerDFGToLLVM.cpp: (JSC::FTL::LowerDFGToLLVM::compileGetById): * ftl/FTLOutput.h: (JSC::FTL::Output::call): * ftl/FTLSlowPathCall.cpp: Added. (JSC::FTL::callOperation): * ftl/FTLSlowPathCall.h: Added. (JSC::FTL::SlowPathCall::SlowPathCall): (JSC::FTL::SlowPathCall::call): (JSC::FTL::SlowPathCall::key): * ftl/FTLSlowPathCallKey.cpp: Added. (JSC::FTL::SlowPathCallKey::dump): * ftl/FTLSlowPathCallKey.h: Added. (JSC::FTL::SlowPathCallKey::SlowPathCallKey): (JSC::FTL::SlowPathCallKey::usedRegisters): (JSC::FTL::SlowPathCallKey::callTarget): (JSC::FTL::SlowPathCallKey::offset): (JSC::FTL::SlowPathCallKey::isEmptyValue): (JSC::FTL::SlowPathCallKey::isDeletedValue): (JSC::FTL::SlowPathCallKey::operator==): (JSC::FTL::SlowPathCallKey::hash): (JSC::FTL::SlowPathCallKeyHash::hash): (JSC::FTL::SlowPathCallKeyHash::equal): * ftl/FTLStackMaps.cpp: (JSC::FTL::StackMaps::Location::directGPR): * ftl/FTLStackMaps.h: * ftl/FTLState.h: * ftl/FTLThunks.cpp: (JSC::FTL::slowPathCallThunkGenerator): * ftl/FTLThunks.h: (JSC::FTL::Thunks::getSlowPathCallThunk): * jit/CCallHelpers.h: (JSC::CCallHelpers::setupArguments): * jit/GPRInfo.h: * jit/JITInlineCacheGenerator.cpp: (JSC::garbageStubInfo): (JSC::JITInlineCacheGenerator::JITInlineCacheGenerator): (JSC::JITByIdGenerator::finalize): * jit/JITInlineCacheGenerator.h: (JSC::JITByIdGenerator::slowPathBegin): * jit/RegisterSet.cpp: (JSC::RegisterSet::stackRegisters): (JSC::RegisterSet::specialRegisters): (JSC::RegisterSet::calleeSaveRegisters): (JSC::RegisterSet::allGPRs): (JSC::RegisterSet::allFPRs): (JSC::RegisterSet::allRegisters): (JSC::RegisterSet::dump): * jit/RegisterSet.h: (JSC::RegisterSet::exclude): (JSC::RegisterSet::numberOfSetRegisters): (JSC::RegisterSet::RegisterSet): (JSC::RegisterSet::isEmptyValue): (JSC::RegisterSet::isDeletedValue): (JSC::RegisterSet::operator==): (JSC::RegisterSet::hash): (JSC::RegisterSetHash::hash): (JSC::RegisterSetHash::equal): * runtime/Options.h: Source/WTF: Reviewed by Mark Hahnenberg. This needed some better bitvector support, like merging (|=), excluding (&=~), hashing, and bit counting. * wtf/BitVector.cpp: (WTF::BitVector::setSlow): (WTF::BitVector::excludeSlow): (WTF::BitVector::bitCountSlow): (WTF::BitVector::equalsSlowCase): (WTF::BitVector::hashSlowCase): (WTF::BitVector::dump): * wtf/BitVector.h: (WTF::BitVector::merge): (WTF::BitVector::exclude): (WTF::BitVector::bitCount): (WTF::BitVector::BitVector): (WTF::BitVector::isEmptyValue): (WTF::BitVector::isDeletedValue): (WTF::BitVector::isEmptyOrDeletedValue): (WTF::BitVector::operator==): (WTF::BitVector::hash): (WTF::BitVectorHash::hash): (WTF::BitVectorHash::equal): * wtf/HashTraits.h: (WTF::CustomHashTraits::constructDeletedValue): (WTF::CustomHashTraits::isDeletedValue): (WTF::CustomHashTraits::emptyValue): (WTF::CustomHashTraits::isEmptyValue): * wtf/StdLibExtras.h: (WTF::bitCount): Canonical link: https://commits.webkit.org/141315@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@157872 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2013-10-23 18:22:09 +00:00
FTL should generate code to call slow paths lazily https://bugs.webkit.org/show_bug.cgi?id=149936 Reviewed by Saam Barati. Source/JavaScriptCore: We often have complex slow paths in FTL-generated code. Those slow paths may never run. Even if they do run, they don't need stellar performance. So, it doesn't make sense to have LLVM worry about compiling such slow path code. This patch enables us to use our own MacroAssembler for compiling the slow path inside FTL code. It does this by using a crazy lambda thingy (see FTLLowerDFGToLLVM.cpp's lazySlowPath() and its documentation). The result is quite natural to use. Even for straight slow path calls via something like vmCall(), the lazySlowPath offers the benefit that the call marshalling and the exception checking are not expressed using LLVM IR and do not require LLVM to think about it. It also has the benefit that we never generate the code if it never runs. That's great, since function calls usually involve ~10 instructions total (move arguments to argument registers, make the call, check exception, etc.). This patch adds the lazy slow path abstraction and uses it for some slow paths in the FTL. The code we generate with lazy slow paths is worse than the code that LLVM would have generated. Therefore, a lazy slow path only makes sense when we have strong evidence that the slow path will execute infrequently relative to the fast path. This completely precludes the use of lazy slow paths for out-of-line Nodes that unconditionally call a C++ function. It also precludes their use for the GetByVal out-of-bounds handler, since when we generate a GetByVal with an out-of-bounds handler it means that we only know that the out-of-bounds case executed at least once. So, for all we know, it may actually be the common case. So, this patch just deployed the lazy slow path for GC slow paths and masquerades-as-undefined slow paths. It makes sense for GC slow paths because those have a statistical guarantee of slow path frequency - probably bounded at less than 1/10. It makes sense for masquerades-as- undefined because we can say quite confidently that this is an uncommon scenario on the modern Web. Something that's always been challenging about abstractions involving the MacroAssembler is that linking is a separate phase, and there is no way for someone who is just given access to the MacroAssembler& to emit code that requires linking, since linking happens once we have emitted all code and we are creating the LinkBuffer. Moreover, the FTL requires that the final parts of linking happen on the main thread. This patch ran into this issue, and solved it comprehensively, by introducing MacroAssembler::addLinkTask(). This takes a lambda and runs it at the bitter end of linking - when performFinalization() is called. This ensure that the task added by addLinkTask() runs on the main thread. This patch doesn't replace all of the previously existing idioms for dealing with this issue; we can do that later. This shows small speed-ups on a bunch of things. No big win on any benchmark aggregate. But mainly this is done for https://bugs.webkit.org/show_bug.cgi?id=149852, where we found that outlining the slow path in this way was a significant speed boost. * CMakeLists.txt: * JavaScriptCore.vcxproj/JavaScriptCore.vcxproj: * JavaScriptCore.xcodeproj/project.pbxproj: * assembler/AbstractMacroAssembler.h: (JSC::AbstractMacroAssembler::replaceWithAddressComputation): (JSC::AbstractMacroAssembler::addLinkTask): (JSC::AbstractMacroAssembler::AbstractMacroAssembler): * assembler/LinkBuffer.cpp: (JSC::LinkBuffer::linkCode): (JSC::LinkBuffer::allocate): (JSC::LinkBuffer::performFinalization): * assembler/LinkBuffer.h: (JSC::LinkBuffer::wasAlreadyDisassembled): (JSC::LinkBuffer::didAlreadyDisassemble): (JSC::LinkBuffer::vm): (JSC::LinkBuffer::executableOffsetFor): * bytecode/CodeOrigin.h: (JSC::CodeOrigin::CodeOrigin): (JSC::CodeOrigin::isSet): (JSC::CodeOrigin::operator bool): (JSC::CodeOrigin::isHashTableDeletedValue): (JSC::CodeOrigin::operator!): Deleted. * ftl/FTLCompile.cpp: (JSC::FTL::mmAllocateDataSection): * ftl/FTLInlineCacheDescriptor.h: (JSC::FTL::InlineCacheDescriptor::InlineCacheDescriptor): (JSC::FTL::CheckInDescriptor::CheckInDescriptor): (JSC::FTL::LazySlowPathDescriptor::LazySlowPathDescriptor): * ftl/FTLJITCode.h: * ftl/FTLJITFinalizer.cpp: (JSC::FTL::JITFinalizer::finalizeFunction): * ftl/FTLJITFinalizer.h: * ftl/FTLLazySlowPath.cpp: Added. (JSC::FTL::LazySlowPath::LazySlowPath): (JSC::FTL::LazySlowPath::~LazySlowPath): (JSC::FTL::LazySlowPath::generate): * ftl/FTLLazySlowPath.h: Added. (JSC::FTL::LazySlowPath::createGenerator): (JSC::FTL::LazySlowPath::patchpoint): (JSC::FTL::LazySlowPath::usedRegisters): (JSC::FTL::LazySlowPath::callSiteIndex): (JSC::FTL::LazySlowPath::stub): * ftl/FTLLazySlowPathCall.h: Added. (JSC::FTL::createLazyCallGenerator): * ftl/FTLLowerDFGToLLVM.cpp: (JSC::FTL::DFG::LowerDFGToLLVM::compileCreateActivation): (JSC::FTL::DFG::LowerDFGToLLVM::compileNewFunction): (JSC::FTL::DFG::LowerDFGToLLVM::compileCreateDirectArguments): (JSC::FTL::DFG::LowerDFGToLLVM::compileNewArrayWithSize): (JSC::FTL::DFG::LowerDFGToLLVM::compileMakeRope): (JSC::FTL::DFG::LowerDFGToLLVM::compileNotifyWrite): (JSC::FTL::DFG::LowerDFGToLLVM::compileIsObjectOrNull): (JSC::FTL::DFG::LowerDFGToLLVM::compileIsFunction): (JSC::FTL::DFG::LowerDFGToLLVM::compileIn): (JSC::FTL::DFG::LowerDFGToLLVM::compileMaterializeNewObject): (JSC::FTL::DFG::LowerDFGToLLVM::compileMaterializeCreateActivation): (JSC::FTL::DFG::LowerDFGToLLVM::compileCheckWatchdogTimer): (JSC::FTL::DFG::LowerDFGToLLVM::allocatePropertyStorageWithSizeImpl): (JSC::FTL::DFG::LowerDFGToLLVM::allocateObject): (JSC::FTL::DFG::LowerDFGToLLVM::allocateJSArray): (JSC::FTL::DFG::LowerDFGToLLVM::buildTypeOf): (JSC::FTL::DFG::LowerDFGToLLVM::sensibleDoubleToInt32): (JSC::FTL::DFG::LowerDFGToLLVM::lazySlowPath): (JSC::FTL::DFG::LowerDFGToLLVM::speculate): (JSC::FTL::DFG::LowerDFGToLLVM::emitStoreBarrier): * ftl/FTLOperations.cpp: (JSC::FTL::operationMaterializeObjectInOSR): (JSC::FTL::compileFTLLazySlowPath): * ftl/FTLOperations.h: * ftl/FTLSlowPathCall.cpp: (JSC::FTL::SlowPathCallContext::SlowPathCallContext): (JSC::FTL::SlowPathCallContext::~SlowPathCallContext): (JSC::FTL::SlowPathCallContext::keyWithTarget): (JSC::FTL::SlowPathCallContext::makeCall): (JSC::FTL::callSiteIndexForCodeOrigin): (JSC::FTL::storeCodeOrigin): Deleted. (JSC::FTL::callOperation): Deleted. * ftl/FTLSlowPathCall.h: (JSC::FTL::callOperation): * ftl/FTLState.h: * ftl/FTLThunks.cpp: (JSC::FTL::genericGenerationThunkGenerator): (JSC::FTL::osrExitGenerationThunkGenerator): (JSC::FTL::lazySlowPathGenerationThunkGenerator): (JSC::FTL::registerClobberCheck): * ftl/FTLThunks.h: * interpreter/CallFrame.h: (JSC::CallSiteIndex::CallSiteIndex): (JSC::CallSiteIndex::operator bool): (JSC::CallSiteIndex::bits): * jit/CCallHelpers.h: (JSC::CCallHelpers::setupArgument): (JSC::CCallHelpers::setupArgumentsWithExecState): * jit/JITOperations.cpp: Source/WTF: Enables SharedTask to handle any function type, not just void(). It's probably better to use SharedTask instead of std::function in performance-sensitive code. std::function uses the system malloc and has copy semantics. SharedTask uses FastMalloc and has aliasing semantics. So, you can just trust that it will have sensible performance characteristics. * wtf/ParallelHelperPool.cpp: (WTF::ParallelHelperClient::~ParallelHelperClient): (WTF::ParallelHelperClient::setTask): (WTF::ParallelHelperClient::doSomeHelping): (WTF::ParallelHelperClient::runTaskInParallel): (WTF::ParallelHelperClient::finish): (WTF::ParallelHelperClient::claimTask): (WTF::ParallelHelperClient::runTask): (WTF::ParallelHelperPool::doSomeHelping): (WTF::ParallelHelperPool::helperThreadBody): * wtf/ParallelHelperPool.h: (WTF::ParallelHelperClient::setFunction): (WTF::ParallelHelperClient::runFunctionInParallel): (WTF::ParallelHelperClient::pool): * wtf/SharedTask.h: (WTF::createSharedTask): (WTF::SharedTask::SharedTask): Deleted. (WTF::SharedTask::~SharedTask): Deleted. (WTF::SharedTaskFunctor::SharedTaskFunctor): Deleted. Canonical link: https://commits.webkit.org/168154@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@190860 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2015-10-12 17:56:26 +00:00
unsigned numberOfCallingConventionRegisters =
m_callingConventionRegisters.numberOfSetRegisters();
FTL should be able to do some simple inline caches using LLVM patchpoints https://bugs.webkit.org/show_bug.cgi?id=123164 Source/JavaScriptCore: Reviewed by Mark Hahnenberg. This implements GetById inline caches in the FTL using llvm.webkit.patchpoint. The idea is that we ask LLVM for a nop slide the size of a GetById inline cache and then fill in the code after LLVM compilation is complete. For now, we just use the system calling convention for the arguments and return. We also still make some assumptions about registers that aren't correct. But, most of the scaffolding is there and this will successfully patch an inline cache. * JavaScriptCore.xcodeproj/project.pbxproj: * assembler/AbstractMacroAssembler.h: * assembler/LinkBuffer.cpp: (JSC::LinkBuffer::finalizeCodeWithoutDisassembly): (JSC::LinkBuffer::linkCode): (JSC::LinkBuffer::allocate): * assembler/LinkBuffer.h: (JSC::LinkBuffer::LinkBuffer): (JSC::LinkBuffer::link): * ftl/FTLAbbreviations.h: (JSC::FTL::constNull): (JSC::FTL::buildCall): * ftl/FTLCapabilities.cpp: (JSC::FTL::canCompile): * ftl/FTLCompile.cpp: (JSC::FTL::fixFunctionBasedOnStackMaps): * ftl/FTLInlineCacheDescriptor.h: Added. (JSC::FTL::InlineCacheDescriptor::InlineCacheDescriptor): (JSC::FTL::GetByIdDescriptor::GetByIdDescriptor): (JSC::FTL::GetByIdDescriptor::stackmapID): (JSC::FTL::GetByIdDescriptor::codeOrigin): (JSC::FTL::GetByIdDescriptor::uid): * ftl/FTLInlineCacheSize.cpp: Added. (JSC::FTL::sizeOfGetById): (JSC::FTL::sizeOfPutById): * ftl/FTLInlineCacheSize.h: Added. * ftl/FTLIntrinsicRepository.h: * ftl/FTLJITFinalizer.cpp: (JSC::FTL::JITFinalizer::finalizeFunction): * ftl/FTLJITFinalizer.h: * ftl/FTLLocation.cpp: (JSC::FTL::Location::directGPR): * ftl/FTLLocation.h: * ftl/FTLLowerDFGToLLVM.cpp: (JSC::FTL::LowerDFGToLLVM::compileGetById): * ftl/FTLOutput.h: (JSC::FTL::Output::call): * ftl/FTLSlowPathCall.cpp: Added. (JSC::FTL::callOperation): * ftl/FTLSlowPathCall.h: Added. (JSC::FTL::SlowPathCall::SlowPathCall): (JSC::FTL::SlowPathCall::call): (JSC::FTL::SlowPathCall::key): * ftl/FTLSlowPathCallKey.cpp: Added. (JSC::FTL::SlowPathCallKey::dump): * ftl/FTLSlowPathCallKey.h: Added. (JSC::FTL::SlowPathCallKey::SlowPathCallKey): (JSC::FTL::SlowPathCallKey::usedRegisters): (JSC::FTL::SlowPathCallKey::callTarget): (JSC::FTL::SlowPathCallKey::offset): (JSC::FTL::SlowPathCallKey::isEmptyValue): (JSC::FTL::SlowPathCallKey::isDeletedValue): (JSC::FTL::SlowPathCallKey::operator==): (JSC::FTL::SlowPathCallKey::hash): (JSC::FTL::SlowPathCallKeyHash::hash): (JSC::FTL::SlowPathCallKeyHash::equal): * ftl/FTLStackMaps.cpp: (JSC::FTL::StackMaps::Location::directGPR): * ftl/FTLStackMaps.h: * ftl/FTLState.h: * ftl/FTLThunks.cpp: (JSC::FTL::slowPathCallThunkGenerator): * ftl/FTLThunks.h: (JSC::FTL::Thunks::getSlowPathCallThunk): * jit/CCallHelpers.h: (JSC::CCallHelpers::setupArguments): * jit/GPRInfo.h: * jit/JITInlineCacheGenerator.cpp: (JSC::garbageStubInfo): (JSC::JITInlineCacheGenerator::JITInlineCacheGenerator): (JSC::JITByIdGenerator::finalize): * jit/JITInlineCacheGenerator.h: (JSC::JITByIdGenerator::slowPathBegin): * jit/RegisterSet.cpp: (JSC::RegisterSet::stackRegisters): (JSC::RegisterSet::specialRegisters): (JSC::RegisterSet::calleeSaveRegisters): (JSC::RegisterSet::allGPRs): (JSC::RegisterSet::allFPRs): (JSC::RegisterSet::allRegisters): (JSC::RegisterSet::dump): * jit/RegisterSet.h: (JSC::RegisterSet::exclude): (JSC::RegisterSet::numberOfSetRegisters): (JSC::RegisterSet::RegisterSet): (JSC::RegisterSet::isEmptyValue): (JSC::RegisterSet::isDeletedValue): (JSC::RegisterSet::operator==): (JSC::RegisterSet::hash): (JSC::RegisterSetHash::hash): (JSC::RegisterSetHash::equal): * runtime/Options.h: Source/WTF: Reviewed by Mark Hahnenberg. This needed some better bitvector support, like merging (|=), excluding (&=~), hashing, and bit counting. * wtf/BitVector.cpp: (WTF::BitVector::setSlow): (WTF::BitVector::excludeSlow): (WTF::BitVector::bitCountSlow): (WTF::BitVector::equalsSlowCase): (WTF::BitVector::hashSlowCase): (WTF::BitVector::dump): * wtf/BitVector.h: (WTF::BitVector::merge): (WTF::BitVector::exclude): (WTF::BitVector::bitCount): (WTF::BitVector::BitVector): (WTF::BitVector::isEmptyValue): (WTF::BitVector::isDeletedValue): (WTF::BitVector::isEmptyOrDeletedValue): (WTF::BitVector::operator==): (WTF::BitVector::hash): (WTF::BitVectorHash::hash): (WTF::BitVectorHash::equal): * wtf/HashTraits.h: (WTF::CustomHashTraits::constructDeletedValue): (WTF::CustomHashTraits::isDeletedValue): (WTF::CustomHashTraits::emptyValue): (WTF::CustomHashTraits::isEmptyValue): * wtf/StdLibExtras.h: (WTF::bitCount): Canonical link: https://commits.webkit.org/141315@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@157872 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2013-10-23 18:22:09 +00:00
FTL should generate code to call slow paths lazily https://bugs.webkit.org/show_bug.cgi?id=149936 Reviewed by Saam Barati. Source/JavaScriptCore: We often have complex slow paths in FTL-generated code. Those slow paths may never run. Even if they do run, they don't need stellar performance. So, it doesn't make sense to have LLVM worry about compiling such slow path code. This patch enables us to use our own MacroAssembler for compiling the slow path inside FTL code. It does this by using a crazy lambda thingy (see FTLLowerDFGToLLVM.cpp's lazySlowPath() and its documentation). The result is quite natural to use. Even for straight slow path calls via something like vmCall(), the lazySlowPath offers the benefit that the call marshalling and the exception checking are not expressed using LLVM IR and do not require LLVM to think about it. It also has the benefit that we never generate the code if it never runs. That's great, since function calls usually involve ~10 instructions total (move arguments to argument registers, make the call, check exception, etc.). This patch adds the lazy slow path abstraction and uses it for some slow paths in the FTL. The code we generate with lazy slow paths is worse than the code that LLVM would have generated. Therefore, a lazy slow path only makes sense when we have strong evidence that the slow path will execute infrequently relative to the fast path. This completely precludes the use of lazy slow paths for out-of-line Nodes that unconditionally call a C++ function. It also precludes their use for the GetByVal out-of-bounds handler, since when we generate a GetByVal with an out-of-bounds handler it means that we only know that the out-of-bounds case executed at least once. So, for all we know, it may actually be the common case. So, this patch just deployed the lazy slow path for GC slow paths and masquerades-as-undefined slow paths. It makes sense for GC slow paths because those have a statistical guarantee of slow path frequency - probably bounded at less than 1/10. It makes sense for masquerades-as- undefined because we can say quite confidently that this is an uncommon scenario on the modern Web. Something that's always been challenging about abstractions involving the MacroAssembler is that linking is a separate phase, and there is no way for someone who is just given access to the MacroAssembler& to emit code that requires linking, since linking happens once we have emitted all code and we are creating the LinkBuffer. Moreover, the FTL requires that the final parts of linking happen on the main thread. This patch ran into this issue, and solved it comprehensively, by introducing MacroAssembler::addLinkTask(). This takes a lambda and runs it at the bitter end of linking - when performFinalization() is called. This ensure that the task added by addLinkTask() runs on the main thread. This patch doesn't replace all of the previously existing idioms for dealing with this issue; we can do that later. This shows small speed-ups on a bunch of things. No big win on any benchmark aggregate. But mainly this is done for https://bugs.webkit.org/show_bug.cgi?id=149852, where we found that outlining the slow path in this way was a significant speed boost. * CMakeLists.txt: * JavaScriptCore.vcxproj/JavaScriptCore.vcxproj: * JavaScriptCore.xcodeproj/project.pbxproj: * assembler/AbstractMacroAssembler.h: (JSC::AbstractMacroAssembler::replaceWithAddressComputation): (JSC::AbstractMacroAssembler::addLinkTask): (JSC::AbstractMacroAssembler::AbstractMacroAssembler): * assembler/LinkBuffer.cpp: (JSC::LinkBuffer::linkCode): (JSC::LinkBuffer::allocate): (JSC::LinkBuffer::performFinalization): * assembler/LinkBuffer.h: (JSC::LinkBuffer::wasAlreadyDisassembled): (JSC::LinkBuffer::didAlreadyDisassemble): (JSC::LinkBuffer::vm): (JSC::LinkBuffer::executableOffsetFor): * bytecode/CodeOrigin.h: (JSC::CodeOrigin::CodeOrigin): (JSC::CodeOrigin::isSet): (JSC::CodeOrigin::operator bool): (JSC::CodeOrigin::isHashTableDeletedValue): (JSC::CodeOrigin::operator!): Deleted. * ftl/FTLCompile.cpp: (JSC::FTL::mmAllocateDataSection): * ftl/FTLInlineCacheDescriptor.h: (JSC::FTL::InlineCacheDescriptor::InlineCacheDescriptor): (JSC::FTL::CheckInDescriptor::CheckInDescriptor): (JSC::FTL::LazySlowPathDescriptor::LazySlowPathDescriptor): * ftl/FTLJITCode.h: * ftl/FTLJITFinalizer.cpp: (JSC::FTL::JITFinalizer::finalizeFunction): * ftl/FTLJITFinalizer.h: * ftl/FTLLazySlowPath.cpp: Added. (JSC::FTL::LazySlowPath::LazySlowPath): (JSC::FTL::LazySlowPath::~LazySlowPath): (JSC::FTL::LazySlowPath::generate): * ftl/FTLLazySlowPath.h: Added. (JSC::FTL::LazySlowPath::createGenerator): (JSC::FTL::LazySlowPath::patchpoint): (JSC::FTL::LazySlowPath::usedRegisters): (JSC::FTL::LazySlowPath::callSiteIndex): (JSC::FTL::LazySlowPath::stub): * ftl/FTLLazySlowPathCall.h: Added. (JSC::FTL::createLazyCallGenerator): * ftl/FTLLowerDFGToLLVM.cpp: (JSC::FTL::DFG::LowerDFGToLLVM::compileCreateActivation): (JSC::FTL::DFG::LowerDFGToLLVM::compileNewFunction): (JSC::FTL::DFG::LowerDFGToLLVM::compileCreateDirectArguments): (JSC::FTL::DFG::LowerDFGToLLVM::compileNewArrayWithSize): (JSC::FTL::DFG::LowerDFGToLLVM::compileMakeRope): (JSC::FTL::DFG::LowerDFGToLLVM::compileNotifyWrite): (JSC::FTL::DFG::LowerDFGToLLVM::compileIsObjectOrNull): (JSC::FTL::DFG::LowerDFGToLLVM::compileIsFunction): (JSC::FTL::DFG::LowerDFGToLLVM::compileIn): (JSC::FTL::DFG::LowerDFGToLLVM::compileMaterializeNewObject): (JSC::FTL::DFG::LowerDFGToLLVM::compileMaterializeCreateActivation): (JSC::FTL::DFG::LowerDFGToLLVM::compileCheckWatchdogTimer): (JSC::FTL::DFG::LowerDFGToLLVM::allocatePropertyStorageWithSizeImpl): (JSC::FTL::DFG::LowerDFGToLLVM::allocateObject): (JSC::FTL::DFG::LowerDFGToLLVM::allocateJSArray): (JSC::FTL::DFG::LowerDFGToLLVM::buildTypeOf): (JSC::FTL::DFG::LowerDFGToLLVM::sensibleDoubleToInt32): (JSC::FTL::DFG::LowerDFGToLLVM::lazySlowPath): (JSC::FTL::DFG::LowerDFGToLLVM::speculate): (JSC::FTL::DFG::LowerDFGToLLVM::emitStoreBarrier): * ftl/FTLOperations.cpp: (JSC::FTL::operationMaterializeObjectInOSR): (JSC::FTL::compileFTLLazySlowPath): * ftl/FTLOperations.h: * ftl/FTLSlowPathCall.cpp: (JSC::FTL::SlowPathCallContext::SlowPathCallContext): (JSC::FTL::SlowPathCallContext::~SlowPathCallContext): (JSC::FTL::SlowPathCallContext::keyWithTarget): (JSC::FTL::SlowPathCallContext::makeCall): (JSC::FTL::callSiteIndexForCodeOrigin): (JSC::FTL::storeCodeOrigin): Deleted. (JSC::FTL::callOperation): Deleted. * ftl/FTLSlowPathCall.h: (JSC::FTL::callOperation): * ftl/FTLState.h: * ftl/FTLThunks.cpp: (JSC::FTL::genericGenerationThunkGenerator): (JSC::FTL::osrExitGenerationThunkGenerator): (JSC::FTL::lazySlowPathGenerationThunkGenerator): (JSC::FTL::registerClobberCheck): * ftl/FTLThunks.h: * interpreter/CallFrame.h: (JSC::CallSiteIndex::CallSiteIndex): (JSC::CallSiteIndex::operator bool): (JSC::CallSiteIndex::bits): * jit/CCallHelpers.h: (JSC::CCallHelpers::setupArgument): (JSC::CCallHelpers::setupArgumentsWithExecState): * jit/JITOperations.cpp: Source/WTF: Enables SharedTask to handle any function type, not just void(). It's probably better to use SharedTask instead of std::function in performance-sensitive code. std::function uses the system malloc and has copy semantics. SharedTask uses FastMalloc and has aliasing semantics. So, you can just trust that it will have sensible performance characteristics. * wtf/ParallelHelperPool.cpp: (WTF::ParallelHelperClient::~ParallelHelperClient): (WTF::ParallelHelperClient::setTask): (WTF::ParallelHelperClient::doSomeHelping): (WTF::ParallelHelperClient::runTaskInParallel): (WTF::ParallelHelperClient::finish): (WTF::ParallelHelperClient::claimTask): (WTF::ParallelHelperClient::runTask): (WTF::ParallelHelperPool::doSomeHelping): (WTF::ParallelHelperPool::helperThreadBody): * wtf/ParallelHelperPool.h: (WTF::ParallelHelperClient::setFunction): (WTF::ParallelHelperClient::runFunctionInParallel): (WTF::ParallelHelperClient::pool): * wtf/SharedTask.h: (WTF::createSharedTask): (WTF::SharedTask::SharedTask): Deleted. (WTF::SharedTask::~SharedTask): Deleted. (WTF::SharedTaskFunctor::SharedTaskFunctor): Deleted. Canonical link: https://commits.webkit.org/168154@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@190860 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2015-10-12 17:56:26 +00:00
size_t offsetToThunkSavingArea =
m_offsetToSavingArea +
numberOfCallingConventionRegisters * wordSize;
FTL should be able to do some simple inline caches using LLVM patchpoints https://bugs.webkit.org/show_bug.cgi?id=123164 Source/JavaScriptCore: Reviewed by Mark Hahnenberg. This implements GetById inline caches in the FTL using llvm.webkit.patchpoint. The idea is that we ask LLVM for a nop slide the size of a GetById inline cache and then fill in the code after LLVM compilation is complete. For now, we just use the system calling convention for the arguments and return. We also still make some assumptions about registers that aren't correct. But, most of the scaffolding is there and this will successfully patch an inline cache. * JavaScriptCore.xcodeproj/project.pbxproj: * assembler/AbstractMacroAssembler.h: * assembler/LinkBuffer.cpp: (JSC::LinkBuffer::finalizeCodeWithoutDisassembly): (JSC::LinkBuffer::linkCode): (JSC::LinkBuffer::allocate): * assembler/LinkBuffer.h: (JSC::LinkBuffer::LinkBuffer): (JSC::LinkBuffer::link): * ftl/FTLAbbreviations.h: (JSC::FTL::constNull): (JSC::FTL::buildCall): * ftl/FTLCapabilities.cpp: (JSC::FTL::canCompile): * ftl/FTLCompile.cpp: (JSC::FTL::fixFunctionBasedOnStackMaps): * ftl/FTLInlineCacheDescriptor.h: Added. (JSC::FTL::InlineCacheDescriptor::InlineCacheDescriptor): (JSC::FTL::GetByIdDescriptor::GetByIdDescriptor): (JSC::FTL::GetByIdDescriptor::stackmapID): (JSC::FTL::GetByIdDescriptor::codeOrigin): (JSC::FTL::GetByIdDescriptor::uid): * ftl/FTLInlineCacheSize.cpp: Added. (JSC::FTL::sizeOfGetById): (JSC::FTL::sizeOfPutById): * ftl/FTLInlineCacheSize.h: Added. * ftl/FTLIntrinsicRepository.h: * ftl/FTLJITFinalizer.cpp: (JSC::FTL::JITFinalizer::finalizeFunction): * ftl/FTLJITFinalizer.h: * ftl/FTLLocation.cpp: (JSC::FTL::Location::directGPR): * ftl/FTLLocation.h: * ftl/FTLLowerDFGToLLVM.cpp: (JSC::FTL::LowerDFGToLLVM::compileGetById): * ftl/FTLOutput.h: (JSC::FTL::Output::call): * ftl/FTLSlowPathCall.cpp: Added. (JSC::FTL::callOperation): * ftl/FTLSlowPathCall.h: Added. (JSC::FTL::SlowPathCall::SlowPathCall): (JSC::FTL::SlowPathCall::call): (JSC::FTL::SlowPathCall::key): * ftl/FTLSlowPathCallKey.cpp: Added. (JSC::FTL::SlowPathCallKey::dump): * ftl/FTLSlowPathCallKey.h: Added. (JSC::FTL::SlowPathCallKey::SlowPathCallKey): (JSC::FTL::SlowPathCallKey::usedRegisters): (JSC::FTL::SlowPathCallKey::callTarget): (JSC::FTL::SlowPathCallKey::offset): (JSC::FTL::SlowPathCallKey::isEmptyValue): (JSC::FTL::SlowPathCallKey::isDeletedValue): (JSC::FTL::SlowPathCallKey::operator==): (JSC::FTL::SlowPathCallKey::hash): (JSC::FTL::SlowPathCallKeyHash::hash): (JSC::FTL::SlowPathCallKeyHash::equal): * ftl/FTLStackMaps.cpp: (JSC::FTL::StackMaps::Location::directGPR): * ftl/FTLStackMaps.h: * ftl/FTLState.h: * ftl/FTLThunks.cpp: (JSC::FTL::slowPathCallThunkGenerator): * ftl/FTLThunks.h: (JSC::FTL::Thunks::getSlowPathCallThunk): * jit/CCallHelpers.h: (JSC::CCallHelpers::setupArguments): * jit/GPRInfo.h: * jit/JITInlineCacheGenerator.cpp: (JSC::garbageStubInfo): (JSC::JITInlineCacheGenerator::JITInlineCacheGenerator): (JSC::JITByIdGenerator::finalize): * jit/JITInlineCacheGenerator.h: (JSC::JITByIdGenerator::slowPathBegin): * jit/RegisterSet.cpp: (JSC::RegisterSet::stackRegisters): (JSC::RegisterSet::specialRegisters): (JSC::RegisterSet::calleeSaveRegisters): (JSC::RegisterSet::allGPRs): (JSC::RegisterSet::allFPRs): (JSC::RegisterSet::allRegisters): (JSC::RegisterSet::dump): * jit/RegisterSet.h: (JSC::RegisterSet::exclude): (JSC::RegisterSet::numberOfSetRegisters): (JSC::RegisterSet::RegisterSet): (JSC::RegisterSet::isEmptyValue): (JSC::RegisterSet::isDeletedValue): (JSC::RegisterSet::operator==): (JSC::RegisterSet::hash): (JSC::RegisterSetHash::hash): (JSC::RegisterSetHash::equal): * runtime/Options.h: Source/WTF: Reviewed by Mark Hahnenberg. This needed some better bitvector support, like merging (|=), excluding (&=~), hashing, and bit counting. * wtf/BitVector.cpp: (WTF::BitVector::setSlow): (WTF::BitVector::excludeSlow): (WTF::BitVector::bitCountSlow): (WTF::BitVector::equalsSlowCase): (WTF::BitVector::hashSlowCase): (WTF::BitVector::dump): * wtf/BitVector.h: (WTF::BitVector::merge): (WTF::BitVector::exclude): (WTF::BitVector::bitCount): (WTF::BitVector::BitVector): (WTF::BitVector::isEmptyValue): (WTF::BitVector::isDeletedValue): (WTF::BitVector::isEmptyOrDeletedValue): (WTF::BitVector::operator==): (WTF::BitVector::hash): (WTF::BitVectorHash::hash): (WTF::BitVectorHash::equal): * wtf/HashTraits.h: (WTF::CustomHashTraits::constructDeletedValue): (WTF::CustomHashTraits::isDeletedValue): (WTF::CustomHashTraits::emptyValue): (WTF::CustomHashTraits::isEmptyValue): * wtf/StdLibExtras.h: (WTF::bitCount): Canonical link: https://commits.webkit.org/141315@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@157872 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2013-10-23 18:22:09 +00:00
FTL should generate code to call slow paths lazily https://bugs.webkit.org/show_bug.cgi?id=149936 Reviewed by Saam Barati. Source/JavaScriptCore: We often have complex slow paths in FTL-generated code. Those slow paths may never run. Even if they do run, they don't need stellar performance. So, it doesn't make sense to have LLVM worry about compiling such slow path code. This patch enables us to use our own MacroAssembler for compiling the slow path inside FTL code. It does this by using a crazy lambda thingy (see FTLLowerDFGToLLVM.cpp's lazySlowPath() and its documentation). The result is quite natural to use. Even for straight slow path calls via something like vmCall(), the lazySlowPath offers the benefit that the call marshalling and the exception checking are not expressed using LLVM IR and do not require LLVM to think about it. It also has the benefit that we never generate the code if it never runs. That's great, since function calls usually involve ~10 instructions total (move arguments to argument registers, make the call, check exception, etc.). This patch adds the lazy slow path abstraction and uses it for some slow paths in the FTL. The code we generate with lazy slow paths is worse than the code that LLVM would have generated. Therefore, a lazy slow path only makes sense when we have strong evidence that the slow path will execute infrequently relative to the fast path. This completely precludes the use of lazy slow paths for out-of-line Nodes that unconditionally call a C++ function. It also precludes their use for the GetByVal out-of-bounds handler, since when we generate a GetByVal with an out-of-bounds handler it means that we only know that the out-of-bounds case executed at least once. So, for all we know, it may actually be the common case. So, this patch just deployed the lazy slow path for GC slow paths and masquerades-as-undefined slow paths. It makes sense for GC slow paths because those have a statistical guarantee of slow path frequency - probably bounded at less than 1/10. It makes sense for masquerades-as- undefined because we can say quite confidently that this is an uncommon scenario on the modern Web. Something that's always been challenging about abstractions involving the MacroAssembler is that linking is a separate phase, and there is no way for someone who is just given access to the MacroAssembler& to emit code that requires linking, since linking happens once we have emitted all code and we are creating the LinkBuffer. Moreover, the FTL requires that the final parts of linking happen on the main thread. This patch ran into this issue, and solved it comprehensively, by introducing MacroAssembler::addLinkTask(). This takes a lambda and runs it at the bitter end of linking - when performFinalization() is called. This ensure that the task added by addLinkTask() runs on the main thread. This patch doesn't replace all of the previously existing idioms for dealing with this issue; we can do that later. This shows small speed-ups on a bunch of things. No big win on any benchmark aggregate. But mainly this is done for https://bugs.webkit.org/show_bug.cgi?id=149852, where we found that outlining the slow path in this way was a significant speed boost. * CMakeLists.txt: * JavaScriptCore.vcxproj/JavaScriptCore.vcxproj: * JavaScriptCore.xcodeproj/project.pbxproj: * assembler/AbstractMacroAssembler.h: (JSC::AbstractMacroAssembler::replaceWithAddressComputation): (JSC::AbstractMacroAssembler::addLinkTask): (JSC::AbstractMacroAssembler::AbstractMacroAssembler): * assembler/LinkBuffer.cpp: (JSC::LinkBuffer::linkCode): (JSC::LinkBuffer::allocate): (JSC::LinkBuffer::performFinalization): * assembler/LinkBuffer.h: (JSC::LinkBuffer::wasAlreadyDisassembled): (JSC::LinkBuffer::didAlreadyDisassemble): (JSC::LinkBuffer::vm): (JSC::LinkBuffer::executableOffsetFor): * bytecode/CodeOrigin.h: (JSC::CodeOrigin::CodeOrigin): (JSC::CodeOrigin::isSet): (JSC::CodeOrigin::operator bool): (JSC::CodeOrigin::isHashTableDeletedValue): (JSC::CodeOrigin::operator!): Deleted. * ftl/FTLCompile.cpp: (JSC::FTL::mmAllocateDataSection): * ftl/FTLInlineCacheDescriptor.h: (JSC::FTL::InlineCacheDescriptor::InlineCacheDescriptor): (JSC::FTL::CheckInDescriptor::CheckInDescriptor): (JSC::FTL::LazySlowPathDescriptor::LazySlowPathDescriptor): * ftl/FTLJITCode.h: * ftl/FTLJITFinalizer.cpp: (JSC::FTL::JITFinalizer::finalizeFunction): * ftl/FTLJITFinalizer.h: * ftl/FTLLazySlowPath.cpp: Added. (JSC::FTL::LazySlowPath::LazySlowPath): (JSC::FTL::LazySlowPath::~LazySlowPath): (JSC::FTL::LazySlowPath::generate): * ftl/FTLLazySlowPath.h: Added. (JSC::FTL::LazySlowPath::createGenerator): (JSC::FTL::LazySlowPath::patchpoint): (JSC::FTL::LazySlowPath::usedRegisters): (JSC::FTL::LazySlowPath::callSiteIndex): (JSC::FTL::LazySlowPath::stub): * ftl/FTLLazySlowPathCall.h: Added. (JSC::FTL::createLazyCallGenerator): * ftl/FTLLowerDFGToLLVM.cpp: (JSC::FTL::DFG::LowerDFGToLLVM::compileCreateActivation): (JSC::FTL::DFG::LowerDFGToLLVM::compileNewFunction): (JSC::FTL::DFG::LowerDFGToLLVM::compileCreateDirectArguments): (JSC::FTL::DFG::LowerDFGToLLVM::compileNewArrayWithSize): (JSC::FTL::DFG::LowerDFGToLLVM::compileMakeRope): (JSC::FTL::DFG::LowerDFGToLLVM::compileNotifyWrite): (JSC::FTL::DFG::LowerDFGToLLVM::compileIsObjectOrNull): (JSC::FTL::DFG::LowerDFGToLLVM::compileIsFunction): (JSC::FTL::DFG::LowerDFGToLLVM::compileIn): (JSC::FTL::DFG::LowerDFGToLLVM::compileMaterializeNewObject): (JSC::FTL::DFG::LowerDFGToLLVM::compileMaterializeCreateActivation): (JSC::FTL::DFG::LowerDFGToLLVM::compileCheckWatchdogTimer): (JSC::FTL::DFG::LowerDFGToLLVM::allocatePropertyStorageWithSizeImpl): (JSC::FTL::DFG::LowerDFGToLLVM::allocateObject): (JSC::FTL::DFG::LowerDFGToLLVM::allocateJSArray): (JSC::FTL::DFG::LowerDFGToLLVM::buildTypeOf): (JSC::FTL::DFG::LowerDFGToLLVM::sensibleDoubleToInt32): (JSC::FTL::DFG::LowerDFGToLLVM::lazySlowPath): (JSC::FTL::DFG::LowerDFGToLLVM::speculate): (JSC::FTL::DFG::LowerDFGToLLVM::emitStoreBarrier): * ftl/FTLOperations.cpp: (JSC::FTL::operationMaterializeObjectInOSR): (JSC::FTL::compileFTLLazySlowPath): * ftl/FTLOperations.h: * ftl/FTLSlowPathCall.cpp: (JSC::FTL::SlowPathCallContext::SlowPathCallContext): (JSC::FTL::SlowPathCallContext::~SlowPathCallContext): (JSC::FTL::SlowPathCallContext::keyWithTarget): (JSC::FTL::SlowPathCallContext::makeCall): (JSC::FTL::callSiteIndexForCodeOrigin): (JSC::FTL::storeCodeOrigin): Deleted. (JSC::FTL::callOperation): Deleted. * ftl/FTLSlowPathCall.h: (JSC::FTL::callOperation): * ftl/FTLState.h: * ftl/FTLThunks.cpp: (JSC::FTL::genericGenerationThunkGenerator): (JSC::FTL::osrExitGenerationThunkGenerator): (JSC::FTL::lazySlowPathGenerationThunkGenerator): (JSC::FTL::registerClobberCheck): * ftl/FTLThunks.h: * interpreter/CallFrame.h: (JSC::CallSiteIndex::CallSiteIndex): (JSC::CallSiteIndex::operator bool): (JSC::CallSiteIndex::bits): * jit/CCallHelpers.h: (JSC::CCallHelpers::setupArgument): (JSC::CCallHelpers::setupArgumentsWithExecState): * jit/JITOperations.cpp: Source/WTF: Enables SharedTask to handle any function type, not just void(). It's probably better to use SharedTask instead of std::function in performance-sensitive code. std::function uses the system malloc and has copy semantics. SharedTask uses FastMalloc and has aliasing semantics. So, you can just trust that it will have sensible performance characteristics. * wtf/ParallelHelperPool.cpp: (WTF::ParallelHelperClient::~ParallelHelperClient): (WTF::ParallelHelperClient::setTask): (WTF::ParallelHelperClient::doSomeHelping): (WTF::ParallelHelperClient::runTaskInParallel): (WTF::ParallelHelperClient::finish): (WTF::ParallelHelperClient::claimTask): (WTF::ParallelHelperClient::runTask): (WTF::ParallelHelperPool::doSomeHelping): (WTF::ParallelHelperPool::helperThreadBody): * wtf/ParallelHelperPool.h: (WTF::ParallelHelperClient::setFunction): (WTF::ParallelHelperClient::runFunctionInParallel): (WTF::ParallelHelperClient::pool): * wtf/SharedTask.h: (WTF::createSharedTask): (WTF::SharedTask::SharedTask): Deleted. (WTF::SharedTask::~SharedTask): Deleted. (WTF::SharedTaskFunctor::SharedTaskFunctor): Deleted. Canonical link: https://commits.webkit.org/168154@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@190860 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2015-10-12 17:56:26 +00:00
m_stackBytesNeeded =
offsetToThunkSavingArea +
stackBytesNeededForReturnAddress +
(usedRegisters.numberOfSetRegisters() - numberOfCallingConventionRegisters) * wordSize;
FTL should be able to do some simple inline caches using LLVM patchpoints https://bugs.webkit.org/show_bug.cgi?id=123164 Source/JavaScriptCore: Reviewed by Mark Hahnenberg. This implements GetById inline caches in the FTL using llvm.webkit.patchpoint. The idea is that we ask LLVM for a nop slide the size of a GetById inline cache and then fill in the code after LLVM compilation is complete. For now, we just use the system calling convention for the arguments and return. We also still make some assumptions about registers that aren't correct. But, most of the scaffolding is there and this will successfully patch an inline cache. * JavaScriptCore.xcodeproj/project.pbxproj: * assembler/AbstractMacroAssembler.h: * assembler/LinkBuffer.cpp: (JSC::LinkBuffer::finalizeCodeWithoutDisassembly): (JSC::LinkBuffer::linkCode): (JSC::LinkBuffer::allocate): * assembler/LinkBuffer.h: (JSC::LinkBuffer::LinkBuffer): (JSC::LinkBuffer::link): * ftl/FTLAbbreviations.h: (JSC::FTL::constNull): (JSC::FTL::buildCall): * ftl/FTLCapabilities.cpp: (JSC::FTL::canCompile): * ftl/FTLCompile.cpp: (JSC::FTL::fixFunctionBasedOnStackMaps): * ftl/FTLInlineCacheDescriptor.h: Added. (JSC::FTL::InlineCacheDescriptor::InlineCacheDescriptor): (JSC::FTL::GetByIdDescriptor::GetByIdDescriptor): (JSC::FTL::GetByIdDescriptor::stackmapID): (JSC::FTL::GetByIdDescriptor::codeOrigin): (JSC::FTL::GetByIdDescriptor::uid): * ftl/FTLInlineCacheSize.cpp: Added. (JSC::FTL::sizeOfGetById): (JSC::FTL::sizeOfPutById): * ftl/FTLInlineCacheSize.h: Added. * ftl/FTLIntrinsicRepository.h: * ftl/FTLJITFinalizer.cpp: (JSC::FTL::JITFinalizer::finalizeFunction): * ftl/FTLJITFinalizer.h: * ftl/FTLLocation.cpp: (JSC::FTL::Location::directGPR): * ftl/FTLLocation.h: * ftl/FTLLowerDFGToLLVM.cpp: (JSC::FTL::LowerDFGToLLVM::compileGetById): * ftl/FTLOutput.h: (JSC::FTL::Output::call): * ftl/FTLSlowPathCall.cpp: Added. (JSC::FTL::callOperation): * ftl/FTLSlowPathCall.h: Added. (JSC::FTL::SlowPathCall::SlowPathCall): (JSC::FTL::SlowPathCall::call): (JSC::FTL::SlowPathCall::key): * ftl/FTLSlowPathCallKey.cpp: Added. (JSC::FTL::SlowPathCallKey::dump): * ftl/FTLSlowPathCallKey.h: Added. (JSC::FTL::SlowPathCallKey::SlowPathCallKey): (JSC::FTL::SlowPathCallKey::usedRegisters): (JSC::FTL::SlowPathCallKey::callTarget): (JSC::FTL::SlowPathCallKey::offset): (JSC::FTL::SlowPathCallKey::isEmptyValue): (JSC::FTL::SlowPathCallKey::isDeletedValue): (JSC::FTL::SlowPathCallKey::operator==): (JSC::FTL::SlowPathCallKey::hash): (JSC::FTL::SlowPathCallKeyHash::hash): (JSC::FTL::SlowPathCallKeyHash::equal): * ftl/FTLStackMaps.cpp: (JSC::FTL::StackMaps::Location::directGPR): * ftl/FTLStackMaps.h: * ftl/FTLState.h: * ftl/FTLThunks.cpp: (JSC::FTL::slowPathCallThunkGenerator): * ftl/FTLThunks.h: (JSC::FTL::Thunks::getSlowPathCallThunk): * jit/CCallHelpers.h: (JSC::CCallHelpers::setupArguments): * jit/GPRInfo.h: * jit/JITInlineCacheGenerator.cpp: (JSC::garbageStubInfo): (JSC::JITInlineCacheGenerator::JITInlineCacheGenerator): (JSC::JITByIdGenerator::finalize): * jit/JITInlineCacheGenerator.h: (JSC::JITByIdGenerator::slowPathBegin): * jit/RegisterSet.cpp: (JSC::RegisterSet::stackRegisters): (JSC::RegisterSet::specialRegisters): (JSC::RegisterSet::calleeSaveRegisters): (JSC::RegisterSet::allGPRs): (JSC::RegisterSet::allFPRs): (JSC::RegisterSet::allRegisters): (JSC::RegisterSet::dump): * jit/RegisterSet.h: (JSC::RegisterSet::exclude): (JSC::RegisterSet::numberOfSetRegisters): (JSC::RegisterSet::RegisterSet): (JSC::RegisterSet::isEmptyValue): (JSC::RegisterSet::isDeletedValue): (JSC::RegisterSet::operator==): (JSC::RegisterSet::hash): (JSC::RegisterSetHash::hash): (JSC::RegisterSetHash::equal): * runtime/Options.h: Source/WTF: Reviewed by Mark Hahnenberg. This needed some better bitvector support, like merging (|=), excluding (&=~), hashing, and bit counting. * wtf/BitVector.cpp: (WTF::BitVector::setSlow): (WTF::BitVector::excludeSlow): (WTF::BitVector::bitCountSlow): (WTF::BitVector::equalsSlowCase): (WTF::BitVector::hashSlowCase): (WTF::BitVector::dump): * wtf/BitVector.h: (WTF::BitVector::merge): (WTF::BitVector::exclude): (WTF::BitVector::bitCount): (WTF::BitVector::BitVector): (WTF::BitVector::isEmptyValue): (WTF::BitVector::isDeletedValue): (WTF::BitVector::isEmptyOrDeletedValue): (WTF::BitVector::operator==): (WTF::BitVector::hash): (WTF::BitVectorHash::hash): (WTF::BitVectorHash::equal): * wtf/HashTraits.h: (WTF::CustomHashTraits::constructDeletedValue): (WTF::CustomHashTraits::isDeletedValue): (WTF::CustomHashTraits::emptyValue): (WTF::CustomHashTraits::isEmptyValue): * wtf/StdLibExtras.h: (WTF::bitCount): Canonical link: https://commits.webkit.org/141315@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@157872 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2013-10-23 18:22:09 +00:00
FTL should generate code to call slow paths lazily https://bugs.webkit.org/show_bug.cgi?id=149936 Reviewed by Saam Barati. Source/JavaScriptCore: We often have complex slow paths in FTL-generated code. Those slow paths may never run. Even if they do run, they don't need stellar performance. So, it doesn't make sense to have LLVM worry about compiling such slow path code. This patch enables us to use our own MacroAssembler for compiling the slow path inside FTL code. It does this by using a crazy lambda thingy (see FTLLowerDFGToLLVM.cpp's lazySlowPath() and its documentation). The result is quite natural to use. Even for straight slow path calls via something like vmCall(), the lazySlowPath offers the benefit that the call marshalling and the exception checking are not expressed using LLVM IR and do not require LLVM to think about it. It also has the benefit that we never generate the code if it never runs. That's great, since function calls usually involve ~10 instructions total (move arguments to argument registers, make the call, check exception, etc.). This patch adds the lazy slow path abstraction and uses it for some slow paths in the FTL. The code we generate with lazy slow paths is worse than the code that LLVM would have generated. Therefore, a lazy slow path only makes sense when we have strong evidence that the slow path will execute infrequently relative to the fast path. This completely precludes the use of lazy slow paths for out-of-line Nodes that unconditionally call a C++ function. It also precludes their use for the GetByVal out-of-bounds handler, since when we generate a GetByVal with an out-of-bounds handler it means that we only know that the out-of-bounds case executed at least once. So, for all we know, it may actually be the common case. So, this patch just deployed the lazy slow path for GC slow paths and masquerades-as-undefined slow paths. It makes sense for GC slow paths because those have a statistical guarantee of slow path frequency - probably bounded at less than 1/10. It makes sense for masquerades-as- undefined because we can say quite confidently that this is an uncommon scenario on the modern Web. Something that's always been challenging about abstractions involving the MacroAssembler is that linking is a separate phase, and there is no way for someone who is just given access to the MacroAssembler& to emit code that requires linking, since linking happens once we have emitted all code and we are creating the LinkBuffer. Moreover, the FTL requires that the final parts of linking happen on the main thread. This patch ran into this issue, and solved it comprehensively, by introducing MacroAssembler::addLinkTask(). This takes a lambda and runs it at the bitter end of linking - when performFinalization() is called. This ensure that the task added by addLinkTask() runs on the main thread. This patch doesn't replace all of the previously existing idioms for dealing with this issue; we can do that later. This shows small speed-ups on a bunch of things. No big win on any benchmark aggregate. But mainly this is done for https://bugs.webkit.org/show_bug.cgi?id=149852, where we found that outlining the slow path in this way was a significant speed boost. * CMakeLists.txt: * JavaScriptCore.vcxproj/JavaScriptCore.vcxproj: * JavaScriptCore.xcodeproj/project.pbxproj: * assembler/AbstractMacroAssembler.h: (JSC::AbstractMacroAssembler::replaceWithAddressComputation): (JSC::AbstractMacroAssembler::addLinkTask): (JSC::AbstractMacroAssembler::AbstractMacroAssembler): * assembler/LinkBuffer.cpp: (JSC::LinkBuffer::linkCode): (JSC::LinkBuffer::allocate): (JSC::LinkBuffer::performFinalization): * assembler/LinkBuffer.h: (JSC::LinkBuffer::wasAlreadyDisassembled): (JSC::LinkBuffer::didAlreadyDisassemble): (JSC::LinkBuffer::vm): (JSC::LinkBuffer::executableOffsetFor): * bytecode/CodeOrigin.h: (JSC::CodeOrigin::CodeOrigin): (JSC::CodeOrigin::isSet): (JSC::CodeOrigin::operator bool): (JSC::CodeOrigin::isHashTableDeletedValue): (JSC::CodeOrigin::operator!): Deleted. * ftl/FTLCompile.cpp: (JSC::FTL::mmAllocateDataSection): * ftl/FTLInlineCacheDescriptor.h: (JSC::FTL::InlineCacheDescriptor::InlineCacheDescriptor): (JSC::FTL::CheckInDescriptor::CheckInDescriptor): (JSC::FTL::LazySlowPathDescriptor::LazySlowPathDescriptor): * ftl/FTLJITCode.h: * ftl/FTLJITFinalizer.cpp: (JSC::FTL::JITFinalizer::finalizeFunction): * ftl/FTLJITFinalizer.h: * ftl/FTLLazySlowPath.cpp: Added. (JSC::FTL::LazySlowPath::LazySlowPath): (JSC::FTL::LazySlowPath::~LazySlowPath): (JSC::FTL::LazySlowPath::generate): * ftl/FTLLazySlowPath.h: Added. (JSC::FTL::LazySlowPath::createGenerator): (JSC::FTL::LazySlowPath::patchpoint): (JSC::FTL::LazySlowPath::usedRegisters): (JSC::FTL::LazySlowPath::callSiteIndex): (JSC::FTL::LazySlowPath::stub): * ftl/FTLLazySlowPathCall.h: Added. (JSC::FTL::createLazyCallGenerator): * ftl/FTLLowerDFGToLLVM.cpp: (JSC::FTL::DFG::LowerDFGToLLVM::compileCreateActivation): (JSC::FTL::DFG::LowerDFGToLLVM::compileNewFunction): (JSC::FTL::DFG::LowerDFGToLLVM::compileCreateDirectArguments): (JSC::FTL::DFG::LowerDFGToLLVM::compileNewArrayWithSize): (JSC::FTL::DFG::LowerDFGToLLVM::compileMakeRope): (JSC::FTL::DFG::LowerDFGToLLVM::compileNotifyWrite): (JSC::FTL::DFG::LowerDFGToLLVM::compileIsObjectOrNull): (JSC::FTL::DFG::LowerDFGToLLVM::compileIsFunction): (JSC::FTL::DFG::LowerDFGToLLVM::compileIn): (JSC::FTL::DFG::LowerDFGToLLVM::compileMaterializeNewObject): (JSC::FTL::DFG::LowerDFGToLLVM::compileMaterializeCreateActivation): (JSC::FTL::DFG::LowerDFGToLLVM::compileCheckWatchdogTimer): (JSC::FTL::DFG::LowerDFGToLLVM::allocatePropertyStorageWithSizeImpl): (JSC::FTL::DFG::LowerDFGToLLVM::allocateObject): (JSC::FTL::DFG::LowerDFGToLLVM::allocateJSArray): (JSC::FTL::DFG::LowerDFGToLLVM::buildTypeOf): (JSC::FTL::DFG::LowerDFGToLLVM::sensibleDoubleToInt32): (JSC::FTL::DFG::LowerDFGToLLVM::lazySlowPath): (JSC::FTL::DFG::LowerDFGToLLVM::speculate): (JSC::FTL::DFG::LowerDFGToLLVM::emitStoreBarrier): * ftl/FTLOperations.cpp: (JSC::FTL::operationMaterializeObjectInOSR): (JSC::FTL::compileFTLLazySlowPath): * ftl/FTLOperations.h: * ftl/FTLSlowPathCall.cpp: (JSC::FTL::SlowPathCallContext::SlowPathCallContext): (JSC::FTL::SlowPathCallContext::~SlowPathCallContext): (JSC::FTL::SlowPathCallContext::keyWithTarget): (JSC::FTL::SlowPathCallContext::makeCall): (JSC::FTL::callSiteIndexForCodeOrigin): (JSC::FTL::storeCodeOrigin): Deleted. (JSC::FTL::callOperation): Deleted. * ftl/FTLSlowPathCall.h: (JSC::FTL::callOperation): * ftl/FTLState.h: * ftl/FTLThunks.cpp: (JSC::FTL::genericGenerationThunkGenerator): (JSC::FTL::osrExitGenerationThunkGenerator): (JSC::FTL::lazySlowPathGenerationThunkGenerator): (JSC::FTL::registerClobberCheck): * ftl/FTLThunks.h: * interpreter/CallFrame.h: (JSC::CallSiteIndex::CallSiteIndex): (JSC::CallSiteIndex::operator bool): (JSC::CallSiteIndex::bits): * jit/CCallHelpers.h: (JSC::CCallHelpers::setupArgument): (JSC::CCallHelpers::setupArgumentsWithExecState): * jit/JITOperations.cpp: Source/WTF: Enables SharedTask to handle any function type, not just void(). It's probably better to use SharedTask instead of std::function in performance-sensitive code. std::function uses the system malloc and has copy semantics. SharedTask uses FastMalloc and has aliasing semantics. So, you can just trust that it will have sensible performance characteristics. * wtf/ParallelHelperPool.cpp: (WTF::ParallelHelperClient::~ParallelHelperClient): (WTF::ParallelHelperClient::setTask): (WTF::ParallelHelperClient::doSomeHelping): (WTF::ParallelHelperClient::runTaskInParallel): (WTF::ParallelHelperClient::finish): (WTF::ParallelHelperClient::claimTask): (WTF::ParallelHelperClient::runTask): (WTF::ParallelHelperPool::doSomeHelping): (WTF::ParallelHelperPool::helperThreadBody): * wtf/ParallelHelperPool.h: (WTF::ParallelHelperClient::setFunction): (WTF::ParallelHelperClient::runFunctionInParallel): (WTF::ParallelHelperClient::pool): * wtf/SharedTask.h: (WTF::createSharedTask): (WTF::SharedTask::SharedTask): Deleted. (WTF::SharedTask::~SharedTask): Deleted. (WTF::SharedTaskFunctor::SharedTaskFunctor): Deleted. Canonical link: https://commits.webkit.org/168154@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@190860 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2015-10-12 17:56:26 +00:00
m_stackBytesNeeded = (m_stackBytesNeeded + stackAlignmentBytes() - 1) & ~(stackAlignmentBytes() - 1);
FTL should be able to do some simple inline caches using LLVM patchpoints https://bugs.webkit.org/show_bug.cgi?id=123164 Source/JavaScriptCore: Reviewed by Mark Hahnenberg. This implements GetById inline caches in the FTL using llvm.webkit.patchpoint. The idea is that we ask LLVM for a nop slide the size of a GetById inline cache and then fill in the code after LLVM compilation is complete. For now, we just use the system calling convention for the arguments and return. We also still make some assumptions about registers that aren't correct. But, most of the scaffolding is there and this will successfully patch an inline cache. * JavaScriptCore.xcodeproj/project.pbxproj: * assembler/AbstractMacroAssembler.h: * assembler/LinkBuffer.cpp: (JSC::LinkBuffer::finalizeCodeWithoutDisassembly): (JSC::LinkBuffer::linkCode): (JSC::LinkBuffer::allocate): * assembler/LinkBuffer.h: (JSC::LinkBuffer::LinkBuffer): (JSC::LinkBuffer::link): * ftl/FTLAbbreviations.h: (JSC::FTL::constNull): (JSC::FTL::buildCall): * ftl/FTLCapabilities.cpp: (JSC::FTL::canCompile): * ftl/FTLCompile.cpp: (JSC::FTL::fixFunctionBasedOnStackMaps): * ftl/FTLInlineCacheDescriptor.h: Added. (JSC::FTL::InlineCacheDescriptor::InlineCacheDescriptor): (JSC::FTL::GetByIdDescriptor::GetByIdDescriptor): (JSC::FTL::GetByIdDescriptor::stackmapID): (JSC::FTL::GetByIdDescriptor::codeOrigin): (JSC::FTL::GetByIdDescriptor::uid): * ftl/FTLInlineCacheSize.cpp: Added. (JSC::FTL::sizeOfGetById): (JSC::FTL::sizeOfPutById): * ftl/FTLInlineCacheSize.h: Added. * ftl/FTLIntrinsicRepository.h: * ftl/FTLJITFinalizer.cpp: (JSC::FTL::JITFinalizer::finalizeFunction): * ftl/FTLJITFinalizer.h: * ftl/FTLLocation.cpp: (JSC::FTL::Location::directGPR): * ftl/FTLLocation.h: * ftl/FTLLowerDFGToLLVM.cpp: (JSC::FTL::LowerDFGToLLVM::compileGetById): * ftl/FTLOutput.h: (JSC::FTL::Output::call): * ftl/FTLSlowPathCall.cpp: Added. (JSC::FTL::callOperation): * ftl/FTLSlowPathCall.h: Added. (JSC::FTL::SlowPathCall::SlowPathCall): (JSC::FTL::SlowPathCall::call): (JSC::FTL::SlowPathCall::key): * ftl/FTLSlowPathCallKey.cpp: Added. (JSC::FTL::SlowPathCallKey::dump): * ftl/FTLSlowPathCallKey.h: Added. (JSC::FTL::SlowPathCallKey::SlowPathCallKey): (JSC::FTL::SlowPathCallKey::usedRegisters): (JSC::FTL::SlowPathCallKey::callTarget): (JSC::FTL::SlowPathCallKey::offset): (JSC::FTL::SlowPathCallKey::isEmptyValue): (JSC::FTL::SlowPathCallKey::isDeletedValue): (JSC::FTL::SlowPathCallKey::operator==): (JSC::FTL::SlowPathCallKey::hash): (JSC::FTL::SlowPathCallKeyHash::hash): (JSC::FTL::SlowPathCallKeyHash::equal): * ftl/FTLStackMaps.cpp: (JSC::FTL::StackMaps::Location::directGPR): * ftl/FTLStackMaps.h: * ftl/FTLState.h: * ftl/FTLThunks.cpp: (JSC::FTL::slowPathCallThunkGenerator): * ftl/FTLThunks.h: (JSC::FTL::Thunks::getSlowPathCallThunk): * jit/CCallHelpers.h: (JSC::CCallHelpers::setupArguments): * jit/GPRInfo.h: * jit/JITInlineCacheGenerator.cpp: (JSC::garbageStubInfo): (JSC::JITInlineCacheGenerator::JITInlineCacheGenerator): (JSC::JITByIdGenerator::finalize): * jit/JITInlineCacheGenerator.h: (JSC::JITByIdGenerator::slowPathBegin): * jit/RegisterSet.cpp: (JSC::RegisterSet::stackRegisters): (JSC::RegisterSet::specialRegisters): (JSC::RegisterSet::calleeSaveRegisters): (JSC::RegisterSet::allGPRs): (JSC::RegisterSet::allFPRs): (JSC::RegisterSet::allRegisters): (JSC::RegisterSet::dump): * jit/RegisterSet.h: (JSC::RegisterSet::exclude): (JSC::RegisterSet::numberOfSetRegisters): (JSC::RegisterSet::RegisterSet): (JSC::RegisterSet::isEmptyValue): (JSC::RegisterSet::isDeletedValue): (JSC::RegisterSet::operator==): (JSC::RegisterSet::hash): (JSC::RegisterSetHash::hash): (JSC::RegisterSetHash::equal): * runtime/Options.h: Source/WTF: Reviewed by Mark Hahnenberg. This needed some better bitvector support, like merging (|=), excluding (&=~), hashing, and bit counting. * wtf/BitVector.cpp: (WTF::BitVector::setSlow): (WTF::BitVector::excludeSlow): (WTF::BitVector::bitCountSlow): (WTF::BitVector::equalsSlowCase): (WTF::BitVector::hashSlowCase): (WTF::BitVector::dump): * wtf/BitVector.h: (WTF::BitVector::merge): (WTF::BitVector::exclude): (WTF::BitVector::bitCount): (WTF::BitVector::BitVector): (WTF::BitVector::isEmptyValue): (WTF::BitVector::isDeletedValue): (WTF::BitVector::isEmptyOrDeletedValue): (WTF::BitVector::operator==): (WTF::BitVector::hash): (WTF::BitVectorHash::hash): (WTF::BitVectorHash::equal): * wtf/HashTraits.h: (WTF::CustomHashTraits::constructDeletedValue): (WTF::CustomHashTraits::isDeletedValue): (WTF::CustomHashTraits::emptyValue): (WTF::CustomHashTraits::isEmptyValue): * wtf/StdLibExtras.h: (WTF::bitCount): Canonical link: https://commits.webkit.org/141315@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@157872 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2013-10-23 18:22:09 +00:00
FTL should generate code to call slow paths lazily https://bugs.webkit.org/show_bug.cgi?id=149936 Reviewed by Saam Barati. Source/JavaScriptCore: We often have complex slow paths in FTL-generated code. Those slow paths may never run. Even if they do run, they don't need stellar performance. So, it doesn't make sense to have LLVM worry about compiling such slow path code. This patch enables us to use our own MacroAssembler for compiling the slow path inside FTL code. It does this by using a crazy lambda thingy (see FTLLowerDFGToLLVM.cpp's lazySlowPath() and its documentation). The result is quite natural to use. Even for straight slow path calls via something like vmCall(), the lazySlowPath offers the benefit that the call marshalling and the exception checking are not expressed using LLVM IR and do not require LLVM to think about it. It also has the benefit that we never generate the code if it never runs. That's great, since function calls usually involve ~10 instructions total (move arguments to argument registers, make the call, check exception, etc.). This patch adds the lazy slow path abstraction and uses it for some slow paths in the FTL. The code we generate with lazy slow paths is worse than the code that LLVM would have generated. Therefore, a lazy slow path only makes sense when we have strong evidence that the slow path will execute infrequently relative to the fast path. This completely precludes the use of lazy slow paths for out-of-line Nodes that unconditionally call a C++ function. It also precludes their use for the GetByVal out-of-bounds handler, since when we generate a GetByVal with an out-of-bounds handler it means that we only know that the out-of-bounds case executed at least once. So, for all we know, it may actually be the common case. So, this patch just deployed the lazy slow path for GC slow paths and masquerades-as-undefined slow paths. It makes sense for GC slow paths because those have a statistical guarantee of slow path frequency - probably bounded at less than 1/10. It makes sense for masquerades-as- undefined because we can say quite confidently that this is an uncommon scenario on the modern Web. Something that's always been challenging about abstractions involving the MacroAssembler is that linking is a separate phase, and there is no way for someone who is just given access to the MacroAssembler& to emit code that requires linking, since linking happens once we have emitted all code and we are creating the LinkBuffer. Moreover, the FTL requires that the final parts of linking happen on the main thread. This patch ran into this issue, and solved it comprehensively, by introducing MacroAssembler::addLinkTask(). This takes a lambda and runs it at the bitter end of linking - when performFinalization() is called. This ensure that the task added by addLinkTask() runs on the main thread. This patch doesn't replace all of the previously existing idioms for dealing with this issue; we can do that later. This shows small speed-ups on a bunch of things. No big win on any benchmark aggregate. But mainly this is done for https://bugs.webkit.org/show_bug.cgi?id=149852, where we found that outlining the slow path in this way was a significant speed boost. * CMakeLists.txt: * JavaScriptCore.vcxproj/JavaScriptCore.vcxproj: * JavaScriptCore.xcodeproj/project.pbxproj: * assembler/AbstractMacroAssembler.h: (JSC::AbstractMacroAssembler::replaceWithAddressComputation): (JSC::AbstractMacroAssembler::addLinkTask): (JSC::AbstractMacroAssembler::AbstractMacroAssembler): * assembler/LinkBuffer.cpp: (JSC::LinkBuffer::linkCode): (JSC::LinkBuffer::allocate): (JSC::LinkBuffer::performFinalization): * assembler/LinkBuffer.h: (JSC::LinkBuffer::wasAlreadyDisassembled): (JSC::LinkBuffer::didAlreadyDisassemble): (JSC::LinkBuffer::vm): (JSC::LinkBuffer::executableOffsetFor): * bytecode/CodeOrigin.h: (JSC::CodeOrigin::CodeOrigin): (JSC::CodeOrigin::isSet): (JSC::CodeOrigin::operator bool): (JSC::CodeOrigin::isHashTableDeletedValue): (JSC::CodeOrigin::operator!): Deleted. * ftl/FTLCompile.cpp: (JSC::FTL::mmAllocateDataSection): * ftl/FTLInlineCacheDescriptor.h: (JSC::FTL::InlineCacheDescriptor::InlineCacheDescriptor): (JSC::FTL::CheckInDescriptor::CheckInDescriptor): (JSC::FTL::LazySlowPathDescriptor::LazySlowPathDescriptor): * ftl/FTLJITCode.h: * ftl/FTLJITFinalizer.cpp: (JSC::FTL::JITFinalizer::finalizeFunction): * ftl/FTLJITFinalizer.h: * ftl/FTLLazySlowPath.cpp: Added. (JSC::FTL::LazySlowPath::LazySlowPath): (JSC::FTL::LazySlowPath::~LazySlowPath): (JSC::FTL::LazySlowPath::generate): * ftl/FTLLazySlowPath.h: Added. (JSC::FTL::LazySlowPath::createGenerator): (JSC::FTL::LazySlowPath::patchpoint): (JSC::FTL::LazySlowPath::usedRegisters): (JSC::FTL::LazySlowPath::callSiteIndex): (JSC::FTL::LazySlowPath::stub): * ftl/FTLLazySlowPathCall.h: Added. (JSC::FTL::createLazyCallGenerator): * ftl/FTLLowerDFGToLLVM.cpp: (JSC::FTL::DFG::LowerDFGToLLVM::compileCreateActivation): (JSC::FTL::DFG::LowerDFGToLLVM::compileNewFunction): (JSC::FTL::DFG::LowerDFGToLLVM::compileCreateDirectArguments): (JSC::FTL::DFG::LowerDFGToLLVM::compileNewArrayWithSize): (JSC::FTL::DFG::LowerDFGToLLVM::compileMakeRope): (JSC::FTL::DFG::LowerDFGToLLVM::compileNotifyWrite): (JSC::FTL::DFG::LowerDFGToLLVM::compileIsObjectOrNull): (JSC::FTL::DFG::LowerDFGToLLVM::compileIsFunction): (JSC::FTL::DFG::LowerDFGToLLVM::compileIn): (JSC::FTL::DFG::LowerDFGToLLVM::compileMaterializeNewObject): (JSC::FTL::DFG::LowerDFGToLLVM::compileMaterializeCreateActivation): (JSC::FTL::DFG::LowerDFGToLLVM::compileCheckWatchdogTimer): (JSC::FTL::DFG::LowerDFGToLLVM::allocatePropertyStorageWithSizeImpl): (JSC::FTL::DFG::LowerDFGToLLVM::allocateObject): (JSC::FTL::DFG::LowerDFGToLLVM::allocateJSArray): (JSC::FTL::DFG::LowerDFGToLLVM::buildTypeOf): (JSC::FTL::DFG::LowerDFGToLLVM::sensibleDoubleToInt32): (JSC::FTL::DFG::LowerDFGToLLVM::lazySlowPath): (JSC::FTL::DFG::LowerDFGToLLVM::speculate): (JSC::FTL::DFG::LowerDFGToLLVM::emitStoreBarrier): * ftl/FTLOperations.cpp: (JSC::FTL::operationMaterializeObjectInOSR): (JSC::FTL::compileFTLLazySlowPath): * ftl/FTLOperations.h: * ftl/FTLSlowPathCall.cpp: (JSC::FTL::SlowPathCallContext::SlowPathCallContext): (JSC::FTL::SlowPathCallContext::~SlowPathCallContext): (JSC::FTL::SlowPathCallContext::keyWithTarget): (JSC::FTL::SlowPathCallContext::makeCall): (JSC::FTL::callSiteIndexForCodeOrigin): (JSC::FTL::storeCodeOrigin): Deleted. (JSC::FTL::callOperation): Deleted. * ftl/FTLSlowPathCall.h: (JSC::FTL::callOperation): * ftl/FTLState.h: * ftl/FTLThunks.cpp: (JSC::FTL::genericGenerationThunkGenerator): (JSC::FTL::osrExitGenerationThunkGenerator): (JSC::FTL::lazySlowPathGenerationThunkGenerator): (JSC::FTL::registerClobberCheck): * ftl/FTLThunks.h: * interpreter/CallFrame.h: (JSC::CallSiteIndex::CallSiteIndex): (JSC::CallSiteIndex::operator bool): (JSC::CallSiteIndex::bits): * jit/CCallHelpers.h: (JSC::CCallHelpers::setupArgument): (JSC::CCallHelpers::setupArgumentsWithExecState): * jit/JITOperations.cpp: Source/WTF: Enables SharedTask to handle any function type, not just void(). It's probably better to use SharedTask instead of std::function in performance-sensitive code. std::function uses the system malloc and has copy semantics. SharedTask uses FastMalloc and has aliasing semantics. So, you can just trust that it will have sensible performance characteristics. * wtf/ParallelHelperPool.cpp: (WTF::ParallelHelperClient::~ParallelHelperClient): (WTF::ParallelHelperClient::setTask): (WTF::ParallelHelperClient::doSomeHelping): (WTF::ParallelHelperClient::runTaskInParallel): (WTF::ParallelHelperClient::finish): (WTF::ParallelHelperClient::claimTask): (WTF::ParallelHelperClient::runTask): (WTF::ParallelHelperPool::doSomeHelping): (WTF::ParallelHelperPool::helperThreadBody): * wtf/ParallelHelperPool.h: (WTF::ParallelHelperClient::setFunction): (WTF::ParallelHelperClient::runFunctionInParallel): (WTF::ParallelHelperClient::pool): * wtf/SharedTask.h: (WTF::createSharedTask): (WTF::SharedTask::SharedTask): Deleted. (WTF::SharedTask::~SharedTask): Deleted. (WTF::SharedTaskFunctor::SharedTaskFunctor): Deleted. Canonical link: https://commits.webkit.org/168154@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@190860 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2015-10-12 17:56:26 +00:00
m_jit.subPtr(CCallHelpers::TrustedImm32(m_stackBytesNeeded), CCallHelpers::stackPointerRegister);
m_thunkSaveSet = usedRegisters;
FTL should be able to do some simple inline caches using LLVM patchpoints https://bugs.webkit.org/show_bug.cgi?id=123164 Source/JavaScriptCore: Reviewed by Mark Hahnenberg. This implements GetById inline caches in the FTL using llvm.webkit.patchpoint. The idea is that we ask LLVM for a nop slide the size of a GetById inline cache and then fill in the code after LLVM compilation is complete. For now, we just use the system calling convention for the arguments and return. We also still make some assumptions about registers that aren't correct. But, most of the scaffolding is there and this will successfully patch an inline cache. * JavaScriptCore.xcodeproj/project.pbxproj: * assembler/AbstractMacroAssembler.h: * assembler/LinkBuffer.cpp: (JSC::LinkBuffer::finalizeCodeWithoutDisassembly): (JSC::LinkBuffer::linkCode): (JSC::LinkBuffer::allocate): * assembler/LinkBuffer.h: (JSC::LinkBuffer::LinkBuffer): (JSC::LinkBuffer::link): * ftl/FTLAbbreviations.h: (JSC::FTL::constNull): (JSC::FTL::buildCall): * ftl/FTLCapabilities.cpp: (JSC::FTL::canCompile): * ftl/FTLCompile.cpp: (JSC::FTL::fixFunctionBasedOnStackMaps): * ftl/FTLInlineCacheDescriptor.h: Added. (JSC::FTL::InlineCacheDescriptor::InlineCacheDescriptor): (JSC::FTL::GetByIdDescriptor::GetByIdDescriptor): (JSC::FTL::GetByIdDescriptor::stackmapID): (JSC::FTL::GetByIdDescriptor::codeOrigin): (JSC::FTL::GetByIdDescriptor::uid): * ftl/FTLInlineCacheSize.cpp: Added. (JSC::FTL::sizeOfGetById): (JSC::FTL::sizeOfPutById): * ftl/FTLInlineCacheSize.h: Added. * ftl/FTLIntrinsicRepository.h: * ftl/FTLJITFinalizer.cpp: (JSC::FTL::JITFinalizer::finalizeFunction): * ftl/FTLJITFinalizer.h: * ftl/FTLLocation.cpp: (JSC::FTL::Location::directGPR): * ftl/FTLLocation.h: * ftl/FTLLowerDFGToLLVM.cpp: (JSC::FTL::LowerDFGToLLVM::compileGetById): * ftl/FTLOutput.h: (JSC::FTL::Output::call): * ftl/FTLSlowPathCall.cpp: Added. (JSC::FTL::callOperation): * ftl/FTLSlowPathCall.h: Added. (JSC::FTL::SlowPathCall::SlowPathCall): (JSC::FTL::SlowPathCall::call): (JSC::FTL::SlowPathCall::key): * ftl/FTLSlowPathCallKey.cpp: Added. (JSC::FTL::SlowPathCallKey::dump): * ftl/FTLSlowPathCallKey.h: Added. (JSC::FTL::SlowPathCallKey::SlowPathCallKey): (JSC::FTL::SlowPathCallKey::usedRegisters): (JSC::FTL::SlowPathCallKey::callTarget): (JSC::FTL::SlowPathCallKey::offset): (JSC::FTL::SlowPathCallKey::isEmptyValue): (JSC::FTL::SlowPathCallKey::isDeletedValue): (JSC::FTL::SlowPathCallKey::operator==): (JSC::FTL::SlowPathCallKey::hash): (JSC::FTL::SlowPathCallKeyHash::hash): (JSC::FTL::SlowPathCallKeyHash::equal): * ftl/FTLStackMaps.cpp: (JSC::FTL::StackMaps::Location::directGPR): * ftl/FTLStackMaps.h: * ftl/FTLState.h: * ftl/FTLThunks.cpp: (JSC::FTL::slowPathCallThunkGenerator): * ftl/FTLThunks.h: (JSC::FTL::Thunks::getSlowPathCallThunk): * jit/CCallHelpers.h: (JSC::CCallHelpers::setupArguments): * jit/GPRInfo.h: * jit/JITInlineCacheGenerator.cpp: (JSC::garbageStubInfo): (JSC::JITInlineCacheGenerator::JITInlineCacheGenerator): (JSC::JITByIdGenerator::finalize): * jit/JITInlineCacheGenerator.h: (JSC::JITByIdGenerator::slowPathBegin): * jit/RegisterSet.cpp: (JSC::RegisterSet::stackRegisters): (JSC::RegisterSet::specialRegisters): (JSC::RegisterSet::calleeSaveRegisters): (JSC::RegisterSet::allGPRs): (JSC::RegisterSet::allFPRs): (JSC::RegisterSet::allRegisters): (JSC::RegisterSet::dump): * jit/RegisterSet.h: (JSC::RegisterSet::exclude): (JSC::RegisterSet::numberOfSetRegisters): (JSC::RegisterSet::RegisterSet): (JSC::RegisterSet::isEmptyValue): (JSC::RegisterSet::isDeletedValue): (JSC::RegisterSet::operator==): (JSC::RegisterSet::hash): (JSC::RegisterSetHash::hash): (JSC::RegisterSetHash::equal): * runtime/Options.h: Source/WTF: Reviewed by Mark Hahnenberg. This needed some better bitvector support, like merging (|=), excluding (&=~), hashing, and bit counting. * wtf/BitVector.cpp: (WTF::BitVector::setSlow): (WTF::BitVector::excludeSlow): (WTF::BitVector::bitCountSlow): (WTF::BitVector::equalsSlowCase): (WTF::BitVector::hashSlowCase): (WTF::BitVector::dump): * wtf/BitVector.h: (WTF::BitVector::merge): (WTF::BitVector::exclude): (WTF::BitVector::bitCount): (WTF::BitVector::BitVector): (WTF::BitVector::isEmptyValue): (WTF::BitVector::isDeletedValue): (WTF::BitVector::isEmptyOrDeletedValue): (WTF::BitVector::operator==): (WTF::BitVector::hash): (WTF::BitVectorHash::hash): (WTF::BitVectorHash::equal): * wtf/HashTraits.h: (WTF::CustomHashTraits::constructDeletedValue): (WTF::CustomHashTraits::isDeletedValue): (WTF::CustomHashTraits::emptyValue): (WTF::CustomHashTraits::isEmptyValue): * wtf/StdLibExtras.h: (WTF::bitCount): Canonical link: https://commits.webkit.org/141315@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@157872 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2013-10-23 18:22:09 +00:00
FTL should generate code to call slow paths lazily https://bugs.webkit.org/show_bug.cgi?id=149936 Reviewed by Saam Barati. Source/JavaScriptCore: We often have complex slow paths in FTL-generated code. Those slow paths may never run. Even if they do run, they don't need stellar performance. So, it doesn't make sense to have LLVM worry about compiling such slow path code. This patch enables us to use our own MacroAssembler for compiling the slow path inside FTL code. It does this by using a crazy lambda thingy (see FTLLowerDFGToLLVM.cpp's lazySlowPath() and its documentation). The result is quite natural to use. Even for straight slow path calls via something like vmCall(), the lazySlowPath offers the benefit that the call marshalling and the exception checking are not expressed using LLVM IR and do not require LLVM to think about it. It also has the benefit that we never generate the code if it never runs. That's great, since function calls usually involve ~10 instructions total (move arguments to argument registers, make the call, check exception, etc.). This patch adds the lazy slow path abstraction and uses it for some slow paths in the FTL. The code we generate with lazy slow paths is worse than the code that LLVM would have generated. Therefore, a lazy slow path only makes sense when we have strong evidence that the slow path will execute infrequently relative to the fast path. This completely precludes the use of lazy slow paths for out-of-line Nodes that unconditionally call a C++ function. It also precludes their use for the GetByVal out-of-bounds handler, since when we generate a GetByVal with an out-of-bounds handler it means that we only know that the out-of-bounds case executed at least once. So, for all we know, it may actually be the common case. So, this patch just deployed the lazy slow path for GC slow paths and masquerades-as-undefined slow paths. It makes sense for GC slow paths because those have a statistical guarantee of slow path frequency - probably bounded at less than 1/10. It makes sense for masquerades-as- undefined because we can say quite confidently that this is an uncommon scenario on the modern Web. Something that's always been challenging about abstractions involving the MacroAssembler is that linking is a separate phase, and there is no way for someone who is just given access to the MacroAssembler& to emit code that requires linking, since linking happens once we have emitted all code and we are creating the LinkBuffer. Moreover, the FTL requires that the final parts of linking happen on the main thread. This patch ran into this issue, and solved it comprehensively, by introducing MacroAssembler::addLinkTask(). This takes a lambda and runs it at the bitter end of linking - when performFinalization() is called. This ensure that the task added by addLinkTask() runs on the main thread. This patch doesn't replace all of the previously existing idioms for dealing with this issue; we can do that later. This shows small speed-ups on a bunch of things. No big win on any benchmark aggregate. But mainly this is done for https://bugs.webkit.org/show_bug.cgi?id=149852, where we found that outlining the slow path in this way was a significant speed boost. * CMakeLists.txt: * JavaScriptCore.vcxproj/JavaScriptCore.vcxproj: * JavaScriptCore.xcodeproj/project.pbxproj: * assembler/AbstractMacroAssembler.h: (JSC::AbstractMacroAssembler::replaceWithAddressComputation): (JSC::AbstractMacroAssembler::addLinkTask): (JSC::AbstractMacroAssembler::AbstractMacroAssembler): * assembler/LinkBuffer.cpp: (JSC::LinkBuffer::linkCode): (JSC::LinkBuffer::allocate): (JSC::LinkBuffer::performFinalization): * assembler/LinkBuffer.h: (JSC::LinkBuffer::wasAlreadyDisassembled): (JSC::LinkBuffer::didAlreadyDisassemble): (JSC::LinkBuffer::vm): (JSC::LinkBuffer::executableOffsetFor): * bytecode/CodeOrigin.h: (JSC::CodeOrigin::CodeOrigin): (JSC::CodeOrigin::isSet): (JSC::CodeOrigin::operator bool): (JSC::CodeOrigin::isHashTableDeletedValue): (JSC::CodeOrigin::operator!): Deleted. * ftl/FTLCompile.cpp: (JSC::FTL::mmAllocateDataSection): * ftl/FTLInlineCacheDescriptor.h: (JSC::FTL::InlineCacheDescriptor::InlineCacheDescriptor): (JSC::FTL::CheckInDescriptor::CheckInDescriptor): (JSC::FTL::LazySlowPathDescriptor::LazySlowPathDescriptor): * ftl/FTLJITCode.h: * ftl/FTLJITFinalizer.cpp: (JSC::FTL::JITFinalizer::finalizeFunction): * ftl/FTLJITFinalizer.h: * ftl/FTLLazySlowPath.cpp: Added. (JSC::FTL::LazySlowPath::LazySlowPath): (JSC::FTL::LazySlowPath::~LazySlowPath): (JSC::FTL::LazySlowPath::generate): * ftl/FTLLazySlowPath.h: Added. (JSC::FTL::LazySlowPath::createGenerator): (JSC::FTL::LazySlowPath::patchpoint): (JSC::FTL::LazySlowPath::usedRegisters): (JSC::FTL::LazySlowPath::callSiteIndex): (JSC::FTL::LazySlowPath::stub): * ftl/FTLLazySlowPathCall.h: Added. (JSC::FTL::createLazyCallGenerator): * ftl/FTLLowerDFGToLLVM.cpp: (JSC::FTL::DFG::LowerDFGToLLVM::compileCreateActivation): (JSC::FTL::DFG::LowerDFGToLLVM::compileNewFunction): (JSC::FTL::DFG::LowerDFGToLLVM::compileCreateDirectArguments): (JSC::FTL::DFG::LowerDFGToLLVM::compileNewArrayWithSize): (JSC::FTL::DFG::LowerDFGToLLVM::compileMakeRope): (JSC::FTL::DFG::LowerDFGToLLVM::compileNotifyWrite): (JSC::FTL::DFG::LowerDFGToLLVM::compileIsObjectOrNull): (JSC::FTL::DFG::LowerDFGToLLVM::compileIsFunction): (JSC::FTL::DFG::LowerDFGToLLVM::compileIn): (JSC::FTL::DFG::LowerDFGToLLVM::compileMaterializeNewObject): (JSC::FTL::DFG::LowerDFGToLLVM::compileMaterializeCreateActivation): (JSC::FTL::DFG::LowerDFGToLLVM::compileCheckWatchdogTimer): (JSC::FTL::DFG::LowerDFGToLLVM::allocatePropertyStorageWithSizeImpl): (JSC::FTL::DFG::LowerDFGToLLVM::allocateObject): (JSC::FTL::DFG::LowerDFGToLLVM::allocateJSArray): (JSC::FTL::DFG::LowerDFGToLLVM::buildTypeOf): (JSC::FTL::DFG::LowerDFGToLLVM::sensibleDoubleToInt32): (JSC::FTL::DFG::LowerDFGToLLVM::lazySlowPath): (JSC::FTL::DFG::LowerDFGToLLVM::speculate): (JSC::FTL::DFG::LowerDFGToLLVM::emitStoreBarrier): * ftl/FTLOperations.cpp: (JSC::FTL::operationMaterializeObjectInOSR): (JSC::FTL::compileFTLLazySlowPath): * ftl/FTLOperations.h: * ftl/FTLSlowPathCall.cpp: (JSC::FTL::SlowPathCallContext::SlowPathCallContext): (JSC::FTL::SlowPathCallContext::~SlowPathCallContext): (JSC::FTL::SlowPathCallContext::keyWithTarget): (JSC::FTL::SlowPathCallContext::makeCall): (JSC::FTL::callSiteIndexForCodeOrigin): (JSC::FTL::storeCodeOrigin): Deleted. (JSC::FTL::callOperation): Deleted. * ftl/FTLSlowPathCall.h: (JSC::FTL::callOperation): * ftl/FTLState.h: * ftl/FTLThunks.cpp: (JSC::FTL::genericGenerationThunkGenerator): (JSC::FTL::osrExitGenerationThunkGenerator): (JSC::FTL::lazySlowPathGenerationThunkGenerator): (JSC::FTL::registerClobberCheck): * ftl/FTLThunks.h: * interpreter/CallFrame.h: (JSC::CallSiteIndex::CallSiteIndex): (JSC::CallSiteIndex::operator bool): (JSC::CallSiteIndex::bits): * jit/CCallHelpers.h: (JSC::CCallHelpers::setupArgument): (JSC::CCallHelpers::setupArgumentsWithExecState): * jit/JITOperations.cpp: Source/WTF: Enables SharedTask to handle any function type, not just void(). It's probably better to use SharedTask instead of std::function in performance-sensitive code. std::function uses the system malloc and has copy semantics. SharedTask uses FastMalloc and has aliasing semantics. So, you can just trust that it will have sensible performance characteristics. * wtf/ParallelHelperPool.cpp: (WTF::ParallelHelperClient::~ParallelHelperClient): (WTF::ParallelHelperClient::setTask): (WTF::ParallelHelperClient::doSomeHelping): (WTF::ParallelHelperClient::runTaskInParallel): (WTF::ParallelHelperClient::finish): (WTF::ParallelHelperClient::claimTask): (WTF::ParallelHelperClient::runTask): (WTF::ParallelHelperPool::doSomeHelping): (WTF::ParallelHelperPool::helperThreadBody): * wtf/ParallelHelperPool.h: (WTF::ParallelHelperClient::setFunction): (WTF::ParallelHelperClient::runFunctionInParallel): (WTF::ParallelHelperClient::pool): * wtf/SharedTask.h: (WTF::createSharedTask): (WTF::SharedTask::SharedTask): Deleted. (WTF::SharedTask::~SharedTask): Deleted. (WTF::SharedTaskFunctor::SharedTaskFunctor): Deleted. Canonical link: https://commits.webkit.org/168154@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@190860 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2015-10-12 17:56:26 +00:00
// This relies on all calling convention registers also being temp registers.
unsigned stackIndex = 0;
for (unsigned i = GPRInfo::numberOfRegisters; i--;) {
GPRReg reg = GPRInfo::toRegister(i);
if (!m_callingConventionRegisters.get(reg))
continue;
m_jit.storePtr(reg, CCallHelpers::Address(CCallHelpers::stackPointerRegister, m_offsetToSavingArea + (stackIndex++) * wordSize));
m_thunkSaveSet.clear(reg);
FTL should be able to do some simple inline caches using LLVM patchpoints https://bugs.webkit.org/show_bug.cgi?id=123164 Source/JavaScriptCore: Reviewed by Mark Hahnenberg. This implements GetById inline caches in the FTL using llvm.webkit.patchpoint. The idea is that we ask LLVM for a nop slide the size of a GetById inline cache and then fill in the code after LLVM compilation is complete. For now, we just use the system calling convention for the arguments and return. We also still make some assumptions about registers that aren't correct. But, most of the scaffolding is there and this will successfully patch an inline cache. * JavaScriptCore.xcodeproj/project.pbxproj: * assembler/AbstractMacroAssembler.h: * assembler/LinkBuffer.cpp: (JSC::LinkBuffer::finalizeCodeWithoutDisassembly): (JSC::LinkBuffer::linkCode): (JSC::LinkBuffer::allocate): * assembler/LinkBuffer.h: (JSC::LinkBuffer::LinkBuffer): (JSC::LinkBuffer::link): * ftl/FTLAbbreviations.h: (JSC::FTL::constNull): (JSC::FTL::buildCall): * ftl/FTLCapabilities.cpp: (JSC::FTL::canCompile): * ftl/FTLCompile.cpp: (JSC::FTL::fixFunctionBasedOnStackMaps): * ftl/FTLInlineCacheDescriptor.h: Added. (JSC::FTL::InlineCacheDescriptor::InlineCacheDescriptor): (JSC::FTL::GetByIdDescriptor::GetByIdDescriptor): (JSC::FTL::GetByIdDescriptor::stackmapID): (JSC::FTL::GetByIdDescriptor::codeOrigin): (JSC::FTL::GetByIdDescriptor::uid): * ftl/FTLInlineCacheSize.cpp: Added. (JSC::FTL::sizeOfGetById): (JSC::FTL::sizeOfPutById): * ftl/FTLInlineCacheSize.h: Added. * ftl/FTLIntrinsicRepository.h: * ftl/FTLJITFinalizer.cpp: (JSC::FTL::JITFinalizer::finalizeFunction): * ftl/FTLJITFinalizer.h: * ftl/FTLLocation.cpp: (JSC::FTL::Location::directGPR): * ftl/FTLLocation.h: * ftl/FTLLowerDFGToLLVM.cpp: (JSC::FTL::LowerDFGToLLVM::compileGetById): * ftl/FTLOutput.h: (JSC::FTL::Output::call): * ftl/FTLSlowPathCall.cpp: Added. (JSC::FTL::callOperation): * ftl/FTLSlowPathCall.h: Added. (JSC::FTL::SlowPathCall::SlowPathCall): (JSC::FTL::SlowPathCall::call): (JSC::FTL::SlowPathCall::key): * ftl/FTLSlowPathCallKey.cpp: Added. (JSC::FTL::SlowPathCallKey::dump): * ftl/FTLSlowPathCallKey.h: Added. (JSC::FTL::SlowPathCallKey::SlowPathCallKey): (JSC::FTL::SlowPathCallKey::usedRegisters): (JSC::FTL::SlowPathCallKey::callTarget): (JSC::FTL::SlowPathCallKey::offset): (JSC::FTL::SlowPathCallKey::isEmptyValue): (JSC::FTL::SlowPathCallKey::isDeletedValue): (JSC::FTL::SlowPathCallKey::operator==): (JSC::FTL::SlowPathCallKey::hash): (JSC::FTL::SlowPathCallKeyHash::hash): (JSC::FTL::SlowPathCallKeyHash::equal): * ftl/FTLStackMaps.cpp: (JSC::FTL::StackMaps::Location::directGPR): * ftl/FTLStackMaps.h: * ftl/FTLState.h: * ftl/FTLThunks.cpp: (JSC::FTL::slowPathCallThunkGenerator): * ftl/FTLThunks.h: (JSC::FTL::Thunks::getSlowPathCallThunk): * jit/CCallHelpers.h: (JSC::CCallHelpers::setupArguments): * jit/GPRInfo.h: * jit/JITInlineCacheGenerator.cpp: (JSC::garbageStubInfo): (JSC::JITInlineCacheGenerator::JITInlineCacheGenerator): (JSC::JITByIdGenerator::finalize): * jit/JITInlineCacheGenerator.h: (JSC::JITByIdGenerator::slowPathBegin): * jit/RegisterSet.cpp: (JSC::RegisterSet::stackRegisters): (JSC::RegisterSet::specialRegisters): (JSC::RegisterSet::calleeSaveRegisters): (JSC::RegisterSet::allGPRs): (JSC::RegisterSet::allFPRs): (JSC::RegisterSet::allRegisters): (JSC::RegisterSet::dump): * jit/RegisterSet.h: (JSC::RegisterSet::exclude): (JSC::RegisterSet::numberOfSetRegisters): (JSC::RegisterSet::RegisterSet): (JSC::RegisterSet::isEmptyValue): (JSC::RegisterSet::isDeletedValue): (JSC::RegisterSet::operator==): (JSC::RegisterSet::hash): (JSC::RegisterSetHash::hash): (JSC::RegisterSetHash::equal): * runtime/Options.h: Source/WTF: Reviewed by Mark Hahnenberg. This needed some better bitvector support, like merging (|=), excluding (&=~), hashing, and bit counting. * wtf/BitVector.cpp: (WTF::BitVector::setSlow): (WTF::BitVector::excludeSlow): (WTF::BitVector::bitCountSlow): (WTF::BitVector::equalsSlowCase): (WTF::BitVector::hashSlowCase): (WTF::BitVector::dump): * wtf/BitVector.h: (WTF::BitVector::merge): (WTF::BitVector::exclude): (WTF::BitVector::bitCount): (WTF::BitVector::BitVector): (WTF::BitVector::isEmptyValue): (WTF::BitVector::isDeletedValue): (WTF::BitVector::isEmptyOrDeletedValue): (WTF::BitVector::operator==): (WTF::BitVector::hash): (WTF::BitVectorHash::hash): (WTF::BitVectorHash::equal): * wtf/HashTraits.h: (WTF::CustomHashTraits::constructDeletedValue): (WTF::CustomHashTraits::isDeletedValue): (WTF::CustomHashTraits::emptyValue): (WTF::CustomHashTraits::isEmptyValue): * wtf/StdLibExtras.h: (WTF::bitCount): Canonical link: https://commits.webkit.org/141315@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@157872 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2013-10-23 18:22:09 +00:00
}
FTL should generate code to call slow paths lazily https://bugs.webkit.org/show_bug.cgi?id=149936 Reviewed by Saam Barati. Source/JavaScriptCore: We often have complex slow paths in FTL-generated code. Those slow paths may never run. Even if they do run, they don't need stellar performance. So, it doesn't make sense to have LLVM worry about compiling such slow path code. This patch enables us to use our own MacroAssembler for compiling the slow path inside FTL code. It does this by using a crazy lambda thingy (see FTLLowerDFGToLLVM.cpp's lazySlowPath() and its documentation). The result is quite natural to use. Even for straight slow path calls via something like vmCall(), the lazySlowPath offers the benefit that the call marshalling and the exception checking are not expressed using LLVM IR and do not require LLVM to think about it. It also has the benefit that we never generate the code if it never runs. That's great, since function calls usually involve ~10 instructions total (move arguments to argument registers, make the call, check exception, etc.). This patch adds the lazy slow path abstraction and uses it for some slow paths in the FTL. The code we generate with lazy slow paths is worse than the code that LLVM would have generated. Therefore, a lazy slow path only makes sense when we have strong evidence that the slow path will execute infrequently relative to the fast path. This completely precludes the use of lazy slow paths for out-of-line Nodes that unconditionally call a C++ function. It also precludes their use for the GetByVal out-of-bounds handler, since when we generate a GetByVal with an out-of-bounds handler it means that we only know that the out-of-bounds case executed at least once. So, for all we know, it may actually be the common case. So, this patch just deployed the lazy slow path for GC slow paths and masquerades-as-undefined slow paths. It makes sense for GC slow paths because those have a statistical guarantee of slow path frequency - probably bounded at less than 1/10. It makes sense for masquerades-as- undefined because we can say quite confidently that this is an uncommon scenario on the modern Web. Something that's always been challenging about abstractions involving the MacroAssembler is that linking is a separate phase, and there is no way for someone who is just given access to the MacroAssembler& to emit code that requires linking, since linking happens once we have emitted all code and we are creating the LinkBuffer. Moreover, the FTL requires that the final parts of linking happen on the main thread. This patch ran into this issue, and solved it comprehensively, by introducing MacroAssembler::addLinkTask(). This takes a lambda and runs it at the bitter end of linking - when performFinalization() is called. This ensure that the task added by addLinkTask() runs on the main thread. This patch doesn't replace all of the previously existing idioms for dealing with this issue; we can do that later. This shows small speed-ups on a bunch of things. No big win on any benchmark aggregate. But mainly this is done for https://bugs.webkit.org/show_bug.cgi?id=149852, where we found that outlining the slow path in this way was a significant speed boost. * CMakeLists.txt: * JavaScriptCore.vcxproj/JavaScriptCore.vcxproj: * JavaScriptCore.xcodeproj/project.pbxproj: * assembler/AbstractMacroAssembler.h: (JSC::AbstractMacroAssembler::replaceWithAddressComputation): (JSC::AbstractMacroAssembler::addLinkTask): (JSC::AbstractMacroAssembler::AbstractMacroAssembler): * assembler/LinkBuffer.cpp: (JSC::LinkBuffer::linkCode): (JSC::LinkBuffer::allocate): (JSC::LinkBuffer::performFinalization): * assembler/LinkBuffer.h: (JSC::LinkBuffer::wasAlreadyDisassembled): (JSC::LinkBuffer::didAlreadyDisassemble): (JSC::LinkBuffer::vm): (JSC::LinkBuffer::executableOffsetFor): * bytecode/CodeOrigin.h: (JSC::CodeOrigin::CodeOrigin): (JSC::CodeOrigin::isSet): (JSC::CodeOrigin::operator bool): (JSC::CodeOrigin::isHashTableDeletedValue): (JSC::CodeOrigin::operator!): Deleted. * ftl/FTLCompile.cpp: (JSC::FTL::mmAllocateDataSection): * ftl/FTLInlineCacheDescriptor.h: (JSC::FTL::InlineCacheDescriptor::InlineCacheDescriptor): (JSC::FTL::CheckInDescriptor::CheckInDescriptor): (JSC::FTL::LazySlowPathDescriptor::LazySlowPathDescriptor): * ftl/FTLJITCode.h: * ftl/FTLJITFinalizer.cpp: (JSC::FTL::JITFinalizer::finalizeFunction): * ftl/FTLJITFinalizer.h: * ftl/FTLLazySlowPath.cpp: Added. (JSC::FTL::LazySlowPath::LazySlowPath): (JSC::FTL::LazySlowPath::~LazySlowPath): (JSC::FTL::LazySlowPath::generate): * ftl/FTLLazySlowPath.h: Added. (JSC::FTL::LazySlowPath::createGenerator): (JSC::FTL::LazySlowPath::patchpoint): (JSC::FTL::LazySlowPath::usedRegisters): (JSC::FTL::LazySlowPath::callSiteIndex): (JSC::FTL::LazySlowPath::stub): * ftl/FTLLazySlowPathCall.h: Added. (JSC::FTL::createLazyCallGenerator): * ftl/FTLLowerDFGToLLVM.cpp: (JSC::FTL::DFG::LowerDFGToLLVM::compileCreateActivation): (JSC::FTL::DFG::LowerDFGToLLVM::compileNewFunction): (JSC::FTL::DFG::LowerDFGToLLVM::compileCreateDirectArguments): (JSC::FTL::DFG::LowerDFGToLLVM::compileNewArrayWithSize): (JSC::FTL::DFG::LowerDFGToLLVM::compileMakeRope): (JSC::FTL::DFG::LowerDFGToLLVM::compileNotifyWrite): (JSC::FTL::DFG::LowerDFGToLLVM::compileIsObjectOrNull): (JSC::FTL::DFG::LowerDFGToLLVM::compileIsFunction): (JSC::FTL::DFG::LowerDFGToLLVM::compileIn): (JSC::FTL::DFG::LowerDFGToLLVM::compileMaterializeNewObject): (JSC::FTL::DFG::LowerDFGToLLVM::compileMaterializeCreateActivation): (JSC::FTL::DFG::LowerDFGToLLVM::compileCheckWatchdogTimer): (JSC::FTL::DFG::LowerDFGToLLVM::allocatePropertyStorageWithSizeImpl): (JSC::FTL::DFG::LowerDFGToLLVM::allocateObject): (JSC::FTL::DFG::LowerDFGToLLVM::allocateJSArray): (JSC::FTL::DFG::LowerDFGToLLVM::buildTypeOf): (JSC::FTL::DFG::LowerDFGToLLVM::sensibleDoubleToInt32): (JSC::FTL::DFG::LowerDFGToLLVM::lazySlowPath): (JSC::FTL::DFG::LowerDFGToLLVM::speculate): (JSC::FTL::DFG::LowerDFGToLLVM::emitStoreBarrier): * ftl/FTLOperations.cpp: (JSC::FTL::operationMaterializeObjectInOSR): (JSC::FTL::compileFTLLazySlowPath): * ftl/FTLOperations.h: * ftl/FTLSlowPathCall.cpp: (JSC::FTL::SlowPathCallContext::SlowPathCallContext): (JSC::FTL::SlowPathCallContext::~SlowPathCallContext): (JSC::FTL::SlowPathCallContext::keyWithTarget): (JSC::FTL::SlowPathCallContext::makeCall): (JSC::FTL::callSiteIndexForCodeOrigin): (JSC::FTL::storeCodeOrigin): Deleted. (JSC::FTL::callOperation): Deleted. * ftl/FTLSlowPathCall.h: (JSC::FTL::callOperation): * ftl/FTLState.h: * ftl/FTLThunks.cpp: (JSC::FTL::genericGenerationThunkGenerator): (JSC::FTL::osrExitGenerationThunkGenerator): (JSC::FTL::lazySlowPathGenerationThunkGenerator): (JSC::FTL::registerClobberCheck): * ftl/FTLThunks.h: * interpreter/CallFrame.h: (JSC::CallSiteIndex::CallSiteIndex): (JSC::CallSiteIndex::operator bool): (JSC::CallSiteIndex::bits): * jit/CCallHelpers.h: (JSC::CCallHelpers::setupArgument): (JSC::CCallHelpers::setupArgumentsWithExecState): * jit/JITOperations.cpp: Source/WTF: Enables SharedTask to handle any function type, not just void(). It's probably better to use SharedTask instead of std::function in performance-sensitive code. std::function uses the system malloc and has copy semantics. SharedTask uses FastMalloc and has aliasing semantics. So, you can just trust that it will have sensible performance characteristics. * wtf/ParallelHelperPool.cpp: (WTF::ParallelHelperClient::~ParallelHelperClient): (WTF::ParallelHelperClient::setTask): (WTF::ParallelHelperClient::doSomeHelping): (WTF::ParallelHelperClient::runTaskInParallel): (WTF::ParallelHelperClient::finish): (WTF::ParallelHelperClient::claimTask): (WTF::ParallelHelperClient::runTask): (WTF::ParallelHelperPool::doSomeHelping): (WTF::ParallelHelperPool::helperThreadBody): * wtf/ParallelHelperPool.h: (WTF::ParallelHelperClient::setFunction): (WTF::ParallelHelperClient::runFunctionInParallel): (WTF::ParallelHelperClient::pool): * wtf/SharedTask.h: (WTF::createSharedTask): (WTF::SharedTask::SharedTask): Deleted. (WTF::SharedTask::~SharedTask): Deleted. (WTF::SharedTaskFunctor::SharedTaskFunctor): Deleted. Canonical link: https://commits.webkit.org/168154@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@190860 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2015-10-12 17:56:26 +00:00
m_offset = offsetToThunkSavingArea;
}
FTL should be able to do some simple inline caches using LLVM patchpoints https://bugs.webkit.org/show_bug.cgi?id=123164 Source/JavaScriptCore: Reviewed by Mark Hahnenberg. This implements GetById inline caches in the FTL using llvm.webkit.patchpoint. The idea is that we ask LLVM for a nop slide the size of a GetById inline cache and then fill in the code after LLVM compilation is complete. For now, we just use the system calling convention for the arguments and return. We also still make some assumptions about registers that aren't correct. But, most of the scaffolding is there and this will successfully patch an inline cache. * JavaScriptCore.xcodeproj/project.pbxproj: * assembler/AbstractMacroAssembler.h: * assembler/LinkBuffer.cpp: (JSC::LinkBuffer::finalizeCodeWithoutDisassembly): (JSC::LinkBuffer::linkCode): (JSC::LinkBuffer::allocate): * assembler/LinkBuffer.h: (JSC::LinkBuffer::LinkBuffer): (JSC::LinkBuffer::link): * ftl/FTLAbbreviations.h: (JSC::FTL::constNull): (JSC::FTL::buildCall): * ftl/FTLCapabilities.cpp: (JSC::FTL::canCompile): * ftl/FTLCompile.cpp: (JSC::FTL::fixFunctionBasedOnStackMaps): * ftl/FTLInlineCacheDescriptor.h: Added. (JSC::FTL::InlineCacheDescriptor::InlineCacheDescriptor): (JSC::FTL::GetByIdDescriptor::GetByIdDescriptor): (JSC::FTL::GetByIdDescriptor::stackmapID): (JSC::FTL::GetByIdDescriptor::codeOrigin): (JSC::FTL::GetByIdDescriptor::uid): * ftl/FTLInlineCacheSize.cpp: Added. (JSC::FTL::sizeOfGetById): (JSC::FTL::sizeOfPutById): * ftl/FTLInlineCacheSize.h: Added. * ftl/FTLIntrinsicRepository.h: * ftl/FTLJITFinalizer.cpp: (JSC::FTL::JITFinalizer::finalizeFunction): * ftl/FTLJITFinalizer.h: * ftl/FTLLocation.cpp: (JSC::FTL::Location::directGPR): * ftl/FTLLocation.h: * ftl/FTLLowerDFGToLLVM.cpp: (JSC::FTL::LowerDFGToLLVM::compileGetById): * ftl/FTLOutput.h: (JSC::FTL::Output::call): * ftl/FTLSlowPathCall.cpp: Added. (JSC::FTL::callOperation): * ftl/FTLSlowPathCall.h: Added. (JSC::FTL::SlowPathCall::SlowPathCall): (JSC::FTL::SlowPathCall::call): (JSC::FTL::SlowPathCall::key): * ftl/FTLSlowPathCallKey.cpp: Added. (JSC::FTL::SlowPathCallKey::dump): * ftl/FTLSlowPathCallKey.h: Added. (JSC::FTL::SlowPathCallKey::SlowPathCallKey): (JSC::FTL::SlowPathCallKey::usedRegisters): (JSC::FTL::SlowPathCallKey::callTarget): (JSC::FTL::SlowPathCallKey::offset): (JSC::FTL::SlowPathCallKey::isEmptyValue): (JSC::FTL::SlowPathCallKey::isDeletedValue): (JSC::FTL::SlowPathCallKey::operator==): (JSC::FTL::SlowPathCallKey::hash): (JSC::FTL::SlowPathCallKeyHash::hash): (JSC::FTL::SlowPathCallKeyHash::equal): * ftl/FTLStackMaps.cpp: (JSC::FTL::StackMaps::Location::directGPR): * ftl/FTLStackMaps.h: * ftl/FTLState.h: * ftl/FTLThunks.cpp: (JSC::FTL::slowPathCallThunkGenerator): * ftl/FTLThunks.h: (JSC::FTL::Thunks::getSlowPathCallThunk): * jit/CCallHelpers.h: (JSC::CCallHelpers::setupArguments): * jit/GPRInfo.h: * jit/JITInlineCacheGenerator.cpp: (JSC::garbageStubInfo): (JSC::JITInlineCacheGenerator::JITInlineCacheGenerator): (JSC::JITByIdGenerator::finalize): * jit/JITInlineCacheGenerator.h: (JSC::JITByIdGenerator::slowPathBegin): * jit/RegisterSet.cpp: (JSC::RegisterSet::stackRegisters): (JSC::RegisterSet::specialRegisters): (JSC::RegisterSet::calleeSaveRegisters): (JSC::RegisterSet::allGPRs): (JSC::RegisterSet::allFPRs): (JSC::RegisterSet::allRegisters): (JSC::RegisterSet::dump): * jit/RegisterSet.h: (JSC::RegisterSet::exclude): (JSC::RegisterSet::numberOfSetRegisters): (JSC::RegisterSet::RegisterSet): (JSC::RegisterSet::isEmptyValue): (JSC::RegisterSet::isDeletedValue): (JSC::RegisterSet::operator==): (JSC::RegisterSet::hash): (JSC::RegisterSetHash::hash): (JSC::RegisterSetHash::equal): * runtime/Options.h: Source/WTF: Reviewed by Mark Hahnenberg. This needed some better bitvector support, like merging (|=), excluding (&=~), hashing, and bit counting. * wtf/BitVector.cpp: (WTF::BitVector::setSlow): (WTF::BitVector::excludeSlow): (WTF::BitVector::bitCountSlow): (WTF::BitVector::equalsSlowCase): (WTF::BitVector::hashSlowCase): (WTF::BitVector::dump): * wtf/BitVector.h: (WTF::BitVector::merge): (WTF::BitVector::exclude): (WTF::BitVector::bitCount): (WTF::BitVector::BitVector): (WTF::BitVector::isEmptyValue): (WTF::BitVector::isDeletedValue): (WTF::BitVector::isEmptyOrDeletedValue): (WTF::BitVector::operator==): (WTF::BitVector::hash): (WTF::BitVectorHash::hash): (WTF::BitVectorHash::equal): * wtf/HashTraits.h: (WTF::CustomHashTraits::constructDeletedValue): (WTF::CustomHashTraits::isDeletedValue): (WTF::CustomHashTraits::emptyValue): (WTF::CustomHashTraits::isEmptyValue): * wtf/StdLibExtras.h: (WTF::bitCount): Canonical link: https://commits.webkit.org/141315@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@157872 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2013-10-23 18:22:09 +00:00
FTL should generate code to call slow paths lazily https://bugs.webkit.org/show_bug.cgi?id=149936 Reviewed by Saam Barati. Source/JavaScriptCore: We often have complex slow paths in FTL-generated code. Those slow paths may never run. Even if they do run, they don't need stellar performance. So, it doesn't make sense to have LLVM worry about compiling such slow path code. This patch enables us to use our own MacroAssembler for compiling the slow path inside FTL code. It does this by using a crazy lambda thingy (see FTLLowerDFGToLLVM.cpp's lazySlowPath() and its documentation). The result is quite natural to use. Even for straight slow path calls via something like vmCall(), the lazySlowPath offers the benefit that the call marshalling and the exception checking are not expressed using LLVM IR and do not require LLVM to think about it. It also has the benefit that we never generate the code if it never runs. That's great, since function calls usually involve ~10 instructions total (move arguments to argument registers, make the call, check exception, etc.). This patch adds the lazy slow path abstraction and uses it for some slow paths in the FTL. The code we generate with lazy slow paths is worse than the code that LLVM would have generated. Therefore, a lazy slow path only makes sense when we have strong evidence that the slow path will execute infrequently relative to the fast path. This completely precludes the use of lazy slow paths for out-of-line Nodes that unconditionally call a C++ function. It also precludes their use for the GetByVal out-of-bounds handler, since when we generate a GetByVal with an out-of-bounds handler it means that we only know that the out-of-bounds case executed at least once. So, for all we know, it may actually be the common case. So, this patch just deployed the lazy slow path for GC slow paths and masquerades-as-undefined slow paths. It makes sense for GC slow paths because those have a statistical guarantee of slow path frequency - probably bounded at less than 1/10. It makes sense for masquerades-as- undefined because we can say quite confidently that this is an uncommon scenario on the modern Web. Something that's always been challenging about abstractions involving the MacroAssembler is that linking is a separate phase, and there is no way for someone who is just given access to the MacroAssembler& to emit code that requires linking, since linking happens once we have emitted all code and we are creating the LinkBuffer. Moreover, the FTL requires that the final parts of linking happen on the main thread. This patch ran into this issue, and solved it comprehensively, by introducing MacroAssembler::addLinkTask(). This takes a lambda and runs it at the bitter end of linking - when performFinalization() is called. This ensure that the task added by addLinkTask() runs on the main thread. This patch doesn't replace all of the previously existing idioms for dealing with this issue; we can do that later. This shows small speed-ups on a bunch of things. No big win on any benchmark aggregate. But mainly this is done for https://bugs.webkit.org/show_bug.cgi?id=149852, where we found that outlining the slow path in this way was a significant speed boost. * CMakeLists.txt: * JavaScriptCore.vcxproj/JavaScriptCore.vcxproj: * JavaScriptCore.xcodeproj/project.pbxproj: * assembler/AbstractMacroAssembler.h: (JSC::AbstractMacroAssembler::replaceWithAddressComputation): (JSC::AbstractMacroAssembler::addLinkTask): (JSC::AbstractMacroAssembler::AbstractMacroAssembler): * assembler/LinkBuffer.cpp: (JSC::LinkBuffer::linkCode): (JSC::LinkBuffer::allocate): (JSC::LinkBuffer::performFinalization): * assembler/LinkBuffer.h: (JSC::LinkBuffer::wasAlreadyDisassembled): (JSC::LinkBuffer::didAlreadyDisassemble): (JSC::LinkBuffer::vm): (JSC::LinkBuffer::executableOffsetFor): * bytecode/CodeOrigin.h: (JSC::CodeOrigin::CodeOrigin): (JSC::CodeOrigin::isSet): (JSC::CodeOrigin::operator bool): (JSC::CodeOrigin::isHashTableDeletedValue): (JSC::CodeOrigin::operator!): Deleted. * ftl/FTLCompile.cpp: (JSC::FTL::mmAllocateDataSection): * ftl/FTLInlineCacheDescriptor.h: (JSC::FTL::InlineCacheDescriptor::InlineCacheDescriptor): (JSC::FTL::CheckInDescriptor::CheckInDescriptor): (JSC::FTL::LazySlowPathDescriptor::LazySlowPathDescriptor): * ftl/FTLJITCode.h: * ftl/FTLJITFinalizer.cpp: (JSC::FTL::JITFinalizer::finalizeFunction): * ftl/FTLJITFinalizer.h: * ftl/FTLLazySlowPath.cpp: Added. (JSC::FTL::LazySlowPath::LazySlowPath): (JSC::FTL::LazySlowPath::~LazySlowPath): (JSC::FTL::LazySlowPath::generate): * ftl/FTLLazySlowPath.h: Added. (JSC::FTL::LazySlowPath::createGenerator): (JSC::FTL::LazySlowPath::patchpoint): (JSC::FTL::LazySlowPath::usedRegisters): (JSC::FTL::LazySlowPath::callSiteIndex): (JSC::FTL::LazySlowPath::stub): * ftl/FTLLazySlowPathCall.h: Added. (JSC::FTL::createLazyCallGenerator): * ftl/FTLLowerDFGToLLVM.cpp: (JSC::FTL::DFG::LowerDFGToLLVM::compileCreateActivation): (JSC::FTL::DFG::LowerDFGToLLVM::compileNewFunction): (JSC::FTL::DFG::LowerDFGToLLVM::compileCreateDirectArguments): (JSC::FTL::DFG::LowerDFGToLLVM::compileNewArrayWithSize): (JSC::FTL::DFG::LowerDFGToLLVM::compileMakeRope): (JSC::FTL::DFG::LowerDFGToLLVM::compileNotifyWrite): (JSC::FTL::DFG::LowerDFGToLLVM::compileIsObjectOrNull): (JSC::FTL::DFG::LowerDFGToLLVM::compileIsFunction): (JSC::FTL::DFG::LowerDFGToLLVM::compileIn): (JSC::FTL::DFG::LowerDFGToLLVM::compileMaterializeNewObject): (JSC::FTL::DFG::LowerDFGToLLVM::compileMaterializeCreateActivation): (JSC::FTL::DFG::LowerDFGToLLVM::compileCheckWatchdogTimer): (JSC::FTL::DFG::LowerDFGToLLVM::allocatePropertyStorageWithSizeImpl): (JSC::FTL::DFG::LowerDFGToLLVM::allocateObject): (JSC::FTL::DFG::LowerDFGToLLVM::allocateJSArray): (JSC::FTL::DFG::LowerDFGToLLVM::buildTypeOf): (JSC::FTL::DFG::LowerDFGToLLVM::sensibleDoubleToInt32): (JSC::FTL::DFG::LowerDFGToLLVM::lazySlowPath): (JSC::FTL::DFG::LowerDFGToLLVM::speculate): (JSC::FTL::DFG::LowerDFGToLLVM::emitStoreBarrier): * ftl/FTLOperations.cpp: (JSC::FTL::operationMaterializeObjectInOSR): (JSC::FTL::compileFTLLazySlowPath): * ftl/FTLOperations.h: * ftl/FTLSlowPathCall.cpp: (JSC::FTL::SlowPathCallContext::SlowPathCallContext): (JSC::FTL::SlowPathCallContext::~SlowPathCallContext): (JSC::FTL::SlowPathCallContext::keyWithTarget): (JSC::FTL::SlowPathCallContext::makeCall): (JSC::FTL::callSiteIndexForCodeOrigin): (JSC::FTL::storeCodeOrigin): Deleted. (JSC::FTL::callOperation): Deleted. * ftl/FTLSlowPathCall.h: (JSC::FTL::callOperation): * ftl/FTLState.h: * ftl/FTLThunks.cpp: (JSC::FTL::genericGenerationThunkGenerator): (JSC::FTL::osrExitGenerationThunkGenerator): (JSC::FTL::lazySlowPathGenerationThunkGenerator): (JSC::FTL::registerClobberCheck): * ftl/FTLThunks.h: * interpreter/CallFrame.h: (JSC::CallSiteIndex::CallSiteIndex): (JSC::CallSiteIndex::operator bool): (JSC::CallSiteIndex::bits): * jit/CCallHelpers.h: (JSC::CCallHelpers::setupArgument): (JSC::CCallHelpers::setupArgumentsWithExecState): * jit/JITOperations.cpp: Source/WTF: Enables SharedTask to handle any function type, not just void(). It's probably better to use SharedTask instead of std::function in performance-sensitive code. std::function uses the system malloc and has copy semantics. SharedTask uses FastMalloc and has aliasing semantics. So, you can just trust that it will have sensible performance characteristics. * wtf/ParallelHelperPool.cpp: (WTF::ParallelHelperClient::~ParallelHelperClient): (WTF::ParallelHelperClient::setTask): (WTF::ParallelHelperClient::doSomeHelping): (WTF::ParallelHelperClient::runTaskInParallel): (WTF::ParallelHelperClient::finish): (WTF::ParallelHelperClient::claimTask): (WTF::ParallelHelperClient::runTask): (WTF::ParallelHelperPool::doSomeHelping): (WTF::ParallelHelperPool::helperThreadBody): * wtf/ParallelHelperPool.h: (WTF::ParallelHelperClient::setFunction): (WTF::ParallelHelperClient::runFunctionInParallel): (WTF::ParallelHelperClient::pool): * wtf/SharedTask.h: (WTF::createSharedTask): (WTF::SharedTask::SharedTask): Deleted. (WTF::SharedTask::~SharedTask): Deleted. (WTF::SharedTaskFunctor::SharedTaskFunctor): Deleted. Canonical link: https://commits.webkit.org/168154@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@190860 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2015-10-12 17:56:26 +00:00
SlowPathCallContext::~SlowPathCallContext()
{
if (m_returnRegister != InvalidGPRReg)
m_jit.move(GPRInfo::returnValueGPR, m_returnRegister);
FTL should be able to do some simple inline caches using LLVM patchpoints https://bugs.webkit.org/show_bug.cgi?id=123164 Source/JavaScriptCore: Reviewed by Mark Hahnenberg. This implements GetById inline caches in the FTL using llvm.webkit.patchpoint. The idea is that we ask LLVM for a nop slide the size of a GetById inline cache and then fill in the code after LLVM compilation is complete. For now, we just use the system calling convention for the arguments and return. We also still make some assumptions about registers that aren't correct. But, most of the scaffolding is there and this will successfully patch an inline cache. * JavaScriptCore.xcodeproj/project.pbxproj: * assembler/AbstractMacroAssembler.h: * assembler/LinkBuffer.cpp: (JSC::LinkBuffer::finalizeCodeWithoutDisassembly): (JSC::LinkBuffer::linkCode): (JSC::LinkBuffer::allocate): * assembler/LinkBuffer.h: (JSC::LinkBuffer::LinkBuffer): (JSC::LinkBuffer::link): * ftl/FTLAbbreviations.h: (JSC::FTL::constNull): (JSC::FTL::buildCall): * ftl/FTLCapabilities.cpp: (JSC::FTL::canCompile): * ftl/FTLCompile.cpp: (JSC::FTL::fixFunctionBasedOnStackMaps): * ftl/FTLInlineCacheDescriptor.h: Added. (JSC::FTL::InlineCacheDescriptor::InlineCacheDescriptor): (JSC::FTL::GetByIdDescriptor::GetByIdDescriptor): (JSC::FTL::GetByIdDescriptor::stackmapID): (JSC::FTL::GetByIdDescriptor::codeOrigin): (JSC::FTL::GetByIdDescriptor::uid): * ftl/FTLInlineCacheSize.cpp: Added. (JSC::FTL::sizeOfGetById): (JSC::FTL::sizeOfPutById): * ftl/FTLInlineCacheSize.h: Added. * ftl/FTLIntrinsicRepository.h: * ftl/FTLJITFinalizer.cpp: (JSC::FTL::JITFinalizer::finalizeFunction): * ftl/FTLJITFinalizer.h: * ftl/FTLLocation.cpp: (JSC::FTL::Location::directGPR): * ftl/FTLLocation.h: * ftl/FTLLowerDFGToLLVM.cpp: (JSC::FTL::LowerDFGToLLVM::compileGetById): * ftl/FTLOutput.h: (JSC::FTL::Output::call): * ftl/FTLSlowPathCall.cpp: Added. (JSC::FTL::callOperation): * ftl/FTLSlowPathCall.h: Added. (JSC::FTL::SlowPathCall::SlowPathCall): (JSC::FTL::SlowPathCall::call): (JSC::FTL::SlowPathCall::key): * ftl/FTLSlowPathCallKey.cpp: Added. (JSC::FTL::SlowPathCallKey::dump): * ftl/FTLSlowPathCallKey.h: Added. (JSC::FTL::SlowPathCallKey::SlowPathCallKey): (JSC::FTL::SlowPathCallKey::usedRegisters): (JSC::FTL::SlowPathCallKey::callTarget): (JSC::FTL::SlowPathCallKey::offset): (JSC::FTL::SlowPathCallKey::isEmptyValue): (JSC::FTL::SlowPathCallKey::isDeletedValue): (JSC::FTL::SlowPathCallKey::operator==): (JSC::FTL::SlowPathCallKey::hash): (JSC::FTL::SlowPathCallKeyHash::hash): (JSC::FTL::SlowPathCallKeyHash::equal): * ftl/FTLStackMaps.cpp: (JSC::FTL::StackMaps::Location::directGPR): * ftl/FTLStackMaps.h: * ftl/FTLState.h: * ftl/FTLThunks.cpp: (JSC::FTL::slowPathCallThunkGenerator): * ftl/FTLThunks.h: (JSC::FTL::Thunks::getSlowPathCallThunk): * jit/CCallHelpers.h: (JSC::CCallHelpers::setupArguments): * jit/GPRInfo.h: * jit/JITInlineCacheGenerator.cpp: (JSC::garbageStubInfo): (JSC::JITInlineCacheGenerator::JITInlineCacheGenerator): (JSC::JITByIdGenerator::finalize): * jit/JITInlineCacheGenerator.h: (JSC::JITByIdGenerator::slowPathBegin): * jit/RegisterSet.cpp: (JSC::RegisterSet::stackRegisters): (JSC::RegisterSet::specialRegisters): (JSC::RegisterSet::calleeSaveRegisters): (JSC::RegisterSet::allGPRs): (JSC::RegisterSet::allFPRs): (JSC::RegisterSet::allRegisters): (JSC::RegisterSet::dump): * jit/RegisterSet.h: (JSC::RegisterSet::exclude): (JSC::RegisterSet::numberOfSetRegisters): (JSC::RegisterSet::RegisterSet): (JSC::RegisterSet::isEmptyValue): (JSC::RegisterSet::isDeletedValue): (JSC::RegisterSet::operator==): (JSC::RegisterSet::hash): (JSC::RegisterSetHash::hash): (JSC::RegisterSetHash::equal): * runtime/Options.h: Source/WTF: Reviewed by Mark Hahnenberg. This needed some better bitvector support, like merging (|=), excluding (&=~), hashing, and bit counting. * wtf/BitVector.cpp: (WTF::BitVector::setSlow): (WTF::BitVector::excludeSlow): (WTF::BitVector::bitCountSlow): (WTF::BitVector::equalsSlowCase): (WTF::BitVector::hashSlowCase): (WTF::BitVector::dump): * wtf/BitVector.h: (WTF::BitVector::merge): (WTF::BitVector::exclude): (WTF::BitVector::bitCount): (WTF::BitVector::BitVector): (WTF::BitVector::isEmptyValue): (WTF::BitVector::isDeletedValue): (WTF::BitVector::isEmptyOrDeletedValue): (WTF::BitVector::operator==): (WTF::BitVector::hash): (WTF::BitVectorHash::hash): (WTF::BitVectorHash::equal): * wtf/HashTraits.h: (WTF::CustomHashTraits::constructDeletedValue): (WTF::CustomHashTraits::isDeletedValue): (WTF::CustomHashTraits::emptyValue): (WTF::CustomHashTraits::isEmptyValue): * wtf/StdLibExtras.h: (WTF::bitCount): Canonical link: https://commits.webkit.org/141315@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@157872 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2013-10-23 18:22:09 +00:00
FTL should generate code to call slow paths lazily https://bugs.webkit.org/show_bug.cgi?id=149936 Reviewed by Saam Barati. Source/JavaScriptCore: We often have complex slow paths in FTL-generated code. Those slow paths may never run. Even if they do run, they don't need stellar performance. So, it doesn't make sense to have LLVM worry about compiling such slow path code. This patch enables us to use our own MacroAssembler for compiling the slow path inside FTL code. It does this by using a crazy lambda thingy (see FTLLowerDFGToLLVM.cpp's lazySlowPath() and its documentation). The result is quite natural to use. Even for straight slow path calls via something like vmCall(), the lazySlowPath offers the benefit that the call marshalling and the exception checking are not expressed using LLVM IR and do not require LLVM to think about it. It also has the benefit that we never generate the code if it never runs. That's great, since function calls usually involve ~10 instructions total (move arguments to argument registers, make the call, check exception, etc.). This patch adds the lazy slow path abstraction and uses it for some slow paths in the FTL. The code we generate with lazy slow paths is worse than the code that LLVM would have generated. Therefore, a lazy slow path only makes sense when we have strong evidence that the slow path will execute infrequently relative to the fast path. This completely precludes the use of lazy slow paths for out-of-line Nodes that unconditionally call a C++ function. It also precludes their use for the GetByVal out-of-bounds handler, since when we generate a GetByVal with an out-of-bounds handler it means that we only know that the out-of-bounds case executed at least once. So, for all we know, it may actually be the common case. So, this patch just deployed the lazy slow path for GC slow paths and masquerades-as-undefined slow paths. It makes sense for GC slow paths because those have a statistical guarantee of slow path frequency - probably bounded at less than 1/10. It makes sense for masquerades-as- undefined because we can say quite confidently that this is an uncommon scenario on the modern Web. Something that's always been challenging about abstractions involving the MacroAssembler is that linking is a separate phase, and there is no way for someone who is just given access to the MacroAssembler& to emit code that requires linking, since linking happens once we have emitted all code and we are creating the LinkBuffer. Moreover, the FTL requires that the final parts of linking happen on the main thread. This patch ran into this issue, and solved it comprehensively, by introducing MacroAssembler::addLinkTask(). This takes a lambda and runs it at the bitter end of linking - when performFinalization() is called. This ensure that the task added by addLinkTask() runs on the main thread. This patch doesn't replace all of the previously existing idioms for dealing with this issue; we can do that later. This shows small speed-ups on a bunch of things. No big win on any benchmark aggregate. But mainly this is done for https://bugs.webkit.org/show_bug.cgi?id=149852, where we found that outlining the slow path in this way was a significant speed boost. * CMakeLists.txt: * JavaScriptCore.vcxproj/JavaScriptCore.vcxproj: * JavaScriptCore.xcodeproj/project.pbxproj: * assembler/AbstractMacroAssembler.h: (JSC::AbstractMacroAssembler::replaceWithAddressComputation): (JSC::AbstractMacroAssembler::addLinkTask): (JSC::AbstractMacroAssembler::AbstractMacroAssembler): * assembler/LinkBuffer.cpp: (JSC::LinkBuffer::linkCode): (JSC::LinkBuffer::allocate): (JSC::LinkBuffer::performFinalization): * assembler/LinkBuffer.h: (JSC::LinkBuffer::wasAlreadyDisassembled): (JSC::LinkBuffer::didAlreadyDisassemble): (JSC::LinkBuffer::vm): (JSC::LinkBuffer::executableOffsetFor): * bytecode/CodeOrigin.h: (JSC::CodeOrigin::CodeOrigin): (JSC::CodeOrigin::isSet): (JSC::CodeOrigin::operator bool): (JSC::CodeOrigin::isHashTableDeletedValue): (JSC::CodeOrigin::operator!): Deleted. * ftl/FTLCompile.cpp: (JSC::FTL::mmAllocateDataSection): * ftl/FTLInlineCacheDescriptor.h: (JSC::FTL::InlineCacheDescriptor::InlineCacheDescriptor): (JSC::FTL::CheckInDescriptor::CheckInDescriptor): (JSC::FTL::LazySlowPathDescriptor::LazySlowPathDescriptor): * ftl/FTLJITCode.h: * ftl/FTLJITFinalizer.cpp: (JSC::FTL::JITFinalizer::finalizeFunction): * ftl/FTLJITFinalizer.h: * ftl/FTLLazySlowPath.cpp: Added. (JSC::FTL::LazySlowPath::LazySlowPath): (JSC::FTL::LazySlowPath::~LazySlowPath): (JSC::FTL::LazySlowPath::generate): * ftl/FTLLazySlowPath.h: Added. (JSC::FTL::LazySlowPath::createGenerator): (JSC::FTL::LazySlowPath::patchpoint): (JSC::FTL::LazySlowPath::usedRegisters): (JSC::FTL::LazySlowPath::callSiteIndex): (JSC::FTL::LazySlowPath::stub): * ftl/FTLLazySlowPathCall.h: Added. (JSC::FTL::createLazyCallGenerator): * ftl/FTLLowerDFGToLLVM.cpp: (JSC::FTL::DFG::LowerDFGToLLVM::compileCreateActivation): (JSC::FTL::DFG::LowerDFGToLLVM::compileNewFunction): (JSC::FTL::DFG::LowerDFGToLLVM::compileCreateDirectArguments): (JSC::FTL::DFG::LowerDFGToLLVM::compileNewArrayWithSize): (JSC::FTL::DFG::LowerDFGToLLVM::compileMakeRope): (JSC::FTL::DFG::LowerDFGToLLVM::compileNotifyWrite): (JSC::FTL::DFG::LowerDFGToLLVM::compileIsObjectOrNull): (JSC::FTL::DFG::LowerDFGToLLVM::compileIsFunction): (JSC::FTL::DFG::LowerDFGToLLVM::compileIn): (JSC::FTL::DFG::LowerDFGToLLVM::compileMaterializeNewObject): (JSC::FTL::DFG::LowerDFGToLLVM::compileMaterializeCreateActivation): (JSC::FTL::DFG::LowerDFGToLLVM::compileCheckWatchdogTimer): (JSC::FTL::DFG::LowerDFGToLLVM::allocatePropertyStorageWithSizeImpl): (JSC::FTL::DFG::LowerDFGToLLVM::allocateObject): (JSC::FTL::DFG::LowerDFGToLLVM::allocateJSArray): (JSC::FTL::DFG::LowerDFGToLLVM::buildTypeOf): (JSC::FTL::DFG::LowerDFGToLLVM::sensibleDoubleToInt32): (JSC::FTL::DFG::LowerDFGToLLVM::lazySlowPath): (JSC::FTL::DFG::LowerDFGToLLVM::speculate): (JSC::FTL::DFG::LowerDFGToLLVM::emitStoreBarrier): * ftl/FTLOperations.cpp: (JSC::FTL::operationMaterializeObjectInOSR): (JSC::FTL::compileFTLLazySlowPath): * ftl/FTLOperations.h: * ftl/FTLSlowPathCall.cpp: (JSC::FTL::SlowPathCallContext::SlowPathCallContext): (JSC::FTL::SlowPathCallContext::~SlowPathCallContext): (JSC::FTL::SlowPathCallContext::keyWithTarget): (JSC::FTL::SlowPathCallContext::makeCall): (JSC::FTL::callSiteIndexForCodeOrigin): (JSC::FTL::storeCodeOrigin): Deleted. (JSC::FTL::callOperation): Deleted. * ftl/FTLSlowPathCall.h: (JSC::FTL::callOperation): * ftl/FTLState.h: * ftl/FTLThunks.cpp: (JSC::FTL::genericGenerationThunkGenerator): (JSC::FTL::osrExitGenerationThunkGenerator): (JSC::FTL::lazySlowPathGenerationThunkGenerator): (JSC::FTL::registerClobberCheck): * ftl/FTLThunks.h: * interpreter/CallFrame.h: (JSC::CallSiteIndex::CallSiteIndex): (JSC::CallSiteIndex::operator bool): (JSC::CallSiteIndex::bits): * jit/CCallHelpers.h: (JSC::CCallHelpers::setupArgument): (JSC::CCallHelpers::setupArgumentsWithExecState): * jit/JITOperations.cpp: Source/WTF: Enables SharedTask to handle any function type, not just void(). It's probably better to use SharedTask instead of std::function in performance-sensitive code. std::function uses the system malloc and has copy semantics. SharedTask uses FastMalloc and has aliasing semantics. So, you can just trust that it will have sensible performance characteristics. * wtf/ParallelHelperPool.cpp: (WTF::ParallelHelperClient::~ParallelHelperClient): (WTF::ParallelHelperClient::setTask): (WTF::ParallelHelperClient::doSomeHelping): (WTF::ParallelHelperClient::runTaskInParallel): (WTF::ParallelHelperClient::finish): (WTF::ParallelHelperClient::claimTask): (WTF::ParallelHelperClient::runTask): (WTF::ParallelHelperPool::doSomeHelping): (WTF::ParallelHelperPool::helperThreadBody): * wtf/ParallelHelperPool.h: (WTF::ParallelHelperClient::setFunction): (WTF::ParallelHelperClient::runFunctionInParallel): (WTF::ParallelHelperClient::pool): * wtf/SharedTask.h: (WTF::createSharedTask): (WTF::SharedTask::SharedTask): Deleted. (WTF::SharedTask::~SharedTask): Deleted. (WTF::SharedTaskFunctor::SharedTaskFunctor): Deleted. Canonical link: https://commits.webkit.org/168154@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@190860 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2015-10-12 17:56:26 +00:00
unsigned stackIndex = 0;
for (unsigned i = GPRInfo::numberOfRegisters; i--;) {
GPRReg reg = GPRInfo::toRegister(i);
if (!m_callingConventionRegisters.get(reg))
continue;
m_jit.loadPtr(CCallHelpers::Address(CCallHelpers::stackPointerRegister, m_offsetToSavingArea + (stackIndex++) * wordSize), reg);
FTL should be able to do some simple inline caches using LLVM patchpoints https://bugs.webkit.org/show_bug.cgi?id=123164 Source/JavaScriptCore: Reviewed by Mark Hahnenberg. This implements GetById inline caches in the FTL using llvm.webkit.patchpoint. The idea is that we ask LLVM for a nop slide the size of a GetById inline cache and then fill in the code after LLVM compilation is complete. For now, we just use the system calling convention for the arguments and return. We also still make some assumptions about registers that aren't correct. But, most of the scaffolding is there and this will successfully patch an inline cache. * JavaScriptCore.xcodeproj/project.pbxproj: * assembler/AbstractMacroAssembler.h: * assembler/LinkBuffer.cpp: (JSC::LinkBuffer::finalizeCodeWithoutDisassembly): (JSC::LinkBuffer::linkCode): (JSC::LinkBuffer::allocate): * assembler/LinkBuffer.h: (JSC::LinkBuffer::LinkBuffer): (JSC::LinkBuffer::link): * ftl/FTLAbbreviations.h: (JSC::FTL::constNull): (JSC::FTL::buildCall): * ftl/FTLCapabilities.cpp: (JSC::FTL::canCompile): * ftl/FTLCompile.cpp: (JSC::FTL::fixFunctionBasedOnStackMaps): * ftl/FTLInlineCacheDescriptor.h: Added. (JSC::FTL::InlineCacheDescriptor::InlineCacheDescriptor): (JSC::FTL::GetByIdDescriptor::GetByIdDescriptor): (JSC::FTL::GetByIdDescriptor::stackmapID): (JSC::FTL::GetByIdDescriptor::codeOrigin): (JSC::FTL::GetByIdDescriptor::uid): * ftl/FTLInlineCacheSize.cpp: Added. (JSC::FTL::sizeOfGetById): (JSC::FTL::sizeOfPutById): * ftl/FTLInlineCacheSize.h: Added. * ftl/FTLIntrinsicRepository.h: * ftl/FTLJITFinalizer.cpp: (JSC::FTL::JITFinalizer::finalizeFunction): * ftl/FTLJITFinalizer.h: * ftl/FTLLocation.cpp: (JSC::FTL::Location::directGPR): * ftl/FTLLocation.h: * ftl/FTLLowerDFGToLLVM.cpp: (JSC::FTL::LowerDFGToLLVM::compileGetById): * ftl/FTLOutput.h: (JSC::FTL::Output::call): * ftl/FTLSlowPathCall.cpp: Added. (JSC::FTL::callOperation): * ftl/FTLSlowPathCall.h: Added. (JSC::FTL::SlowPathCall::SlowPathCall): (JSC::FTL::SlowPathCall::call): (JSC::FTL::SlowPathCall::key): * ftl/FTLSlowPathCallKey.cpp: Added. (JSC::FTL::SlowPathCallKey::dump): * ftl/FTLSlowPathCallKey.h: Added. (JSC::FTL::SlowPathCallKey::SlowPathCallKey): (JSC::FTL::SlowPathCallKey::usedRegisters): (JSC::FTL::SlowPathCallKey::callTarget): (JSC::FTL::SlowPathCallKey::offset): (JSC::FTL::SlowPathCallKey::isEmptyValue): (JSC::FTL::SlowPathCallKey::isDeletedValue): (JSC::FTL::SlowPathCallKey::operator==): (JSC::FTL::SlowPathCallKey::hash): (JSC::FTL::SlowPathCallKeyHash::hash): (JSC::FTL::SlowPathCallKeyHash::equal): * ftl/FTLStackMaps.cpp: (JSC::FTL::StackMaps::Location::directGPR): * ftl/FTLStackMaps.h: * ftl/FTLState.h: * ftl/FTLThunks.cpp: (JSC::FTL::slowPathCallThunkGenerator): * ftl/FTLThunks.h: (JSC::FTL::Thunks::getSlowPathCallThunk): * jit/CCallHelpers.h: (JSC::CCallHelpers::setupArguments): * jit/GPRInfo.h: * jit/JITInlineCacheGenerator.cpp: (JSC::garbageStubInfo): (JSC::JITInlineCacheGenerator::JITInlineCacheGenerator): (JSC::JITByIdGenerator::finalize): * jit/JITInlineCacheGenerator.h: (JSC::JITByIdGenerator::slowPathBegin): * jit/RegisterSet.cpp: (JSC::RegisterSet::stackRegisters): (JSC::RegisterSet::specialRegisters): (JSC::RegisterSet::calleeSaveRegisters): (JSC::RegisterSet::allGPRs): (JSC::RegisterSet::allFPRs): (JSC::RegisterSet::allRegisters): (JSC::RegisterSet::dump): * jit/RegisterSet.h: (JSC::RegisterSet::exclude): (JSC::RegisterSet::numberOfSetRegisters): (JSC::RegisterSet::RegisterSet): (JSC::RegisterSet::isEmptyValue): (JSC::RegisterSet::isDeletedValue): (JSC::RegisterSet::operator==): (JSC::RegisterSet::hash): (JSC::RegisterSetHash::hash): (JSC::RegisterSetHash::equal): * runtime/Options.h: Source/WTF: Reviewed by Mark Hahnenberg. This needed some better bitvector support, like merging (|=), excluding (&=~), hashing, and bit counting. * wtf/BitVector.cpp: (WTF::BitVector::setSlow): (WTF::BitVector::excludeSlow): (WTF::BitVector::bitCountSlow): (WTF::BitVector::equalsSlowCase): (WTF::BitVector::hashSlowCase): (WTF::BitVector::dump): * wtf/BitVector.h: (WTF::BitVector::merge): (WTF::BitVector::exclude): (WTF::BitVector::bitCount): (WTF::BitVector::BitVector): (WTF::BitVector::isEmptyValue): (WTF::BitVector::isDeletedValue): (WTF::BitVector::isEmptyOrDeletedValue): (WTF::BitVector::operator==): (WTF::BitVector::hash): (WTF::BitVectorHash::hash): (WTF::BitVectorHash::equal): * wtf/HashTraits.h: (WTF::CustomHashTraits::constructDeletedValue): (WTF::CustomHashTraits::isDeletedValue): (WTF::CustomHashTraits::emptyValue): (WTF::CustomHashTraits::isEmptyValue): * wtf/StdLibExtras.h: (WTF::bitCount): Canonical link: https://commits.webkit.org/141315@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@157872 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2013-10-23 18:22:09 +00:00
}
FTL should generate code to call slow paths lazily https://bugs.webkit.org/show_bug.cgi?id=149936 Reviewed by Saam Barati. Source/JavaScriptCore: We often have complex slow paths in FTL-generated code. Those slow paths may never run. Even if they do run, they don't need stellar performance. So, it doesn't make sense to have LLVM worry about compiling such slow path code. This patch enables us to use our own MacroAssembler for compiling the slow path inside FTL code. It does this by using a crazy lambda thingy (see FTLLowerDFGToLLVM.cpp's lazySlowPath() and its documentation). The result is quite natural to use. Even for straight slow path calls via something like vmCall(), the lazySlowPath offers the benefit that the call marshalling and the exception checking are not expressed using LLVM IR and do not require LLVM to think about it. It also has the benefit that we never generate the code if it never runs. That's great, since function calls usually involve ~10 instructions total (move arguments to argument registers, make the call, check exception, etc.). This patch adds the lazy slow path abstraction and uses it for some slow paths in the FTL. The code we generate with lazy slow paths is worse than the code that LLVM would have generated. Therefore, a lazy slow path only makes sense when we have strong evidence that the slow path will execute infrequently relative to the fast path. This completely precludes the use of lazy slow paths for out-of-line Nodes that unconditionally call a C++ function. It also precludes their use for the GetByVal out-of-bounds handler, since when we generate a GetByVal with an out-of-bounds handler it means that we only know that the out-of-bounds case executed at least once. So, for all we know, it may actually be the common case. So, this patch just deployed the lazy slow path for GC slow paths and masquerades-as-undefined slow paths. It makes sense for GC slow paths because those have a statistical guarantee of slow path frequency - probably bounded at less than 1/10. It makes sense for masquerades-as- undefined because we can say quite confidently that this is an uncommon scenario on the modern Web. Something that's always been challenging about abstractions involving the MacroAssembler is that linking is a separate phase, and there is no way for someone who is just given access to the MacroAssembler& to emit code that requires linking, since linking happens once we have emitted all code and we are creating the LinkBuffer. Moreover, the FTL requires that the final parts of linking happen on the main thread. This patch ran into this issue, and solved it comprehensively, by introducing MacroAssembler::addLinkTask(). This takes a lambda and runs it at the bitter end of linking - when performFinalization() is called. This ensure that the task added by addLinkTask() runs on the main thread. This patch doesn't replace all of the previously existing idioms for dealing with this issue; we can do that later. This shows small speed-ups on a bunch of things. No big win on any benchmark aggregate. But mainly this is done for https://bugs.webkit.org/show_bug.cgi?id=149852, where we found that outlining the slow path in this way was a significant speed boost. * CMakeLists.txt: * JavaScriptCore.vcxproj/JavaScriptCore.vcxproj: * JavaScriptCore.xcodeproj/project.pbxproj: * assembler/AbstractMacroAssembler.h: (JSC::AbstractMacroAssembler::replaceWithAddressComputation): (JSC::AbstractMacroAssembler::addLinkTask): (JSC::AbstractMacroAssembler::AbstractMacroAssembler): * assembler/LinkBuffer.cpp: (JSC::LinkBuffer::linkCode): (JSC::LinkBuffer::allocate): (JSC::LinkBuffer::performFinalization): * assembler/LinkBuffer.h: (JSC::LinkBuffer::wasAlreadyDisassembled): (JSC::LinkBuffer::didAlreadyDisassemble): (JSC::LinkBuffer::vm): (JSC::LinkBuffer::executableOffsetFor): * bytecode/CodeOrigin.h: (JSC::CodeOrigin::CodeOrigin): (JSC::CodeOrigin::isSet): (JSC::CodeOrigin::operator bool): (JSC::CodeOrigin::isHashTableDeletedValue): (JSC::CodeOrigin::operator!): Deleted. * ftl/FTLCompile.cpp: (JSC::FTL::mmAllocateDataSection): * ftl/FTLInlineCacheDescriptor.h: (JSC::FTL::InlineCacheDescriptor::InlineCacheDescriptor): (JSC::FTL::CheckInDescriptor::CheckInDescriptor): (JSC::FTL::LazySlowPathDescriptor::LazySlowPathDescriptor): * ftl/FTLJITCode.h: * ftl/FTLJITFinalizer.cpp: (JSC::FTL::JITFinalizer::finalizeFunction): * ftl/FTLJITFinalizer.h: * ftl/FTLLazySlowPath.cpp: Added. (JSC::FTL::LazySlowPath::LazySlowPath): (JSC::FTL::LazySlowPath::~LazySlowPath): (JSC::FTL::LazySlowPath::generate): * ftl/FTLLazySlowPath.h: Added. (JSC::FTL::LazySlowPath::createGenerator): (JSC::FTL::LazySlowPath::patchpoint): (JSC::FTL::LazySlowPath::usedRegisters): (JSC::FTL::LazySlowPath::callSiteIndex): (JSC::FTL::LazySlowPath::stub): * ftl/FTLLazySlowPathCall.h: Added. (JSC::FTL::createLazyCallGenerator): * ftl/FTLLowerDFGToLLVM.cpp: (JSC::FTL::DFG::LowerDFGToLLVM::compileCreateActivation): (JSC::FTL::DFG::LowerDFGToLLVM::compileNewFunction): (JSC::FTL::DFG::LowerDFGToLLVM::compileCreateDirectArguments): (JSC::FTL::DFG::LowerDFGToLLVM::compileNewArrayWithSize): (JSC::FTL::DFG::LowerDFGToLLVM::compileMakeRope): (JSC::FTL::DFG::LowerDFGToLLVM::compileNotifyWrite): (JSC::FTL::DFG::LowerDFGToLLVM::compileIsObjectOrNull): (JSC::FTL::DFG::LowerDFGToLLVM::compileIsFunction): (JSC::FTL::DFG::LowerDFGToLLVM::compileIn): (JSC::FTL::DFG::LowerDFGToLLVM::compileMaterializeNewObject): (JSC::FTL::DFG::LowerDFGToLLVM::compileMaterializeCreateActivation): (JSC::FTL::DFG::LowerDFGToLLVM::compileCheckWatchdogTimer): (JSC::FTL::DFG::LowerDFGToLLVM::allocatePropertyStorageWithSizeImpl): (JSC::FTL::DFG::LowerDFGToLLVM::allocateObject): (JSC::FTL::DFG::LowerDFGToLLVM::allocateJSArray): (JSC::FTL::DFG::LowerDFGToLLVM::buildTypeOf): (JSC::FTL::DFG::LowerDFGToLLVM::sensibleDoubleToInt32): (JSC::FTL::DFG::LowerDFGToLLVM::lazySlowPath): (JSC::FTL::DFG::LowerDFGToLLVM::speculate): (JSC::FTL::DFG::LowerDFGToLLVM::emitStoreBarrier): * ftl/FTLOperations.cpp: (JSC::FTL::operationMaterializeObjectInOSR): (JSC::FTL::compileFTLLazySlowPath): * ftl/FTLOperations.h: * ftl/FTLSlowPathCall.cpp: (JSC::FTL::SlowPathCallContext::SlowPathCallContext): (JSC::FTL::SlowPathCallContext::~SlowPathCallContext): (JSC::FTL::SlowPathCallContext::keyWithTarget): (JSC::FTL::SlowPathCallContext::makeCall): (JSC::FTL::callSiteIndexForCodeOrigin): (JSC::FTL::storeCodeOrigin): Deleted. (JSC::FTL::callOperation): Deleted. * ftl/FTLSlowPathCall.h: (JSC::FTL::callOperation): * ftl/FTLState.h: * ftl/FTLThunks.cpp: (JSC::FTL::genericGenerationThunkGenerator): (JSC::FTL::osrExitGenerationThunkGenerator): (JSC::FTL::lazySlowPathGenerationThunkGenerator): (JSC::FTL::registerClobberCheck): * ftl/FTLThunks.h: * interpreter/CallFrame.h: (JSC::CallSiteIndex::CallSiteIndex): (JSC::CallSiteIndex::operator bool): (JSC::CallSiteIndex::bits): * jit/CCallHelpers.h: (JSC::CCallHelpers::setupArgument): (JSC::CCallHelpers::setupArgumentsWithExecState): * jit/JITOperations.cpp: Source/WTF: Enables SharedTask to handle any function type, not just void(). It's probably better to use SharedTask instead of std::function in performance-sensitive code. std::function uses the system malloc and has copy semantics. SharedTask uses FastMalloc and has aliasing semantics. So, you can just trust that it will have sensible performance characteristics. * wtf/ParallelHelperPool.cpp: (WTF::ParallelHelperClient::~ParallelHelperClient): (WTF::ParallelHelperClient::setTask): (WTF::ParallelHelperClient::doSomeHelping): (WTF::ParallelHelperClient::runTaskInParallel): (WTF::ParallelHelperClient::finish): (WTF::ParallelHelperClient::claimTask): (WTF::ParallelHelperClient::runTask): (WTF::ParallelHelperPool::doSomeHelping): (WTF::ParallelHelperPool::helperThreadBody): * wtf/ParallelHelperPool.h: (WTF::ParallelHelperClient::setFunction): (WTF::ParallelHelperClient::runFunctionInParallel): (WTF::ParallelHelperClient::pool): * wtf/SharedTask.h: (WTF::createSharedTask): (WTF::SharedTask::SharedTask): Deleted. (WTF::SharedTask::~SharedTask): Deleted. (WTF::SharedTaskFunctor::SharedTaskFunctor): Deleted. Canonical link: https://commits.webkit.org/168154@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@190860 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2015-10-12 17:56:26 +00:00
m_jit.addPtr(CCallHelpers::TrustedImm32(m_stackBytesNeeded), CCallHelpers::stackPointerRegister);
}
Templatize CodePtr/Refs/FunctionPtrs with PtrTags. https://bugs.webkit.org/show_bug.cgi?id=184702 <rdar://problem/35391681> Reviewed by Filip Pizlo and Saam Barati. Source/JavaScriptCore: 1. Templatized MacroAssemblerCodePtr/Ref, FunctionPtr, and CodeLocation variants to take a PtrTag template argument. 2. Replaced some uses of raw pointers with the equivalent CodePtr / FunctionPtr. * assembler/AbstractMacroAssembler.h: (JSC::AbstractMacroAssembler::differenceBetweenCodePtr): (JSC::AbstractMacroAssembler::linkJump): (JSC::AbstractMacroAssembler::linkPointer): (JSC::AbstractMacroAssembler::getLinkerAddress): (JSC::AbstractMacroAssembler::repatchJump): (JSC::AbstractMacroAssembler::repatchJumpToNop): (JSC::AbstractMacroAssembler::repatchNearCall): (JSC::AbstractMacroAssembler::repatchCompact): (JSC::AbstractMacroAssembler::repatchInt32): (JSC::AbstractMacroAssembler::repatchPointer): (JSC::AbstractMacroAssembler::readPointer): (JSC::AbstractMacroAssembler::replaceWithLoad): (JSC::AbstractMacroAssembler::replaceWithAddressComputation): * assembler/CodeLocation.h: (JSC::CodeLocationCommon:: const): (JSC::CodeLocationCommon::CodeLocationCommon): (JSC::CodeLocationInstruction::CodeLocationInstruction): (JSC::CodeLocationLabel::CodeLocationLabel): (JSC::CodeLocationLabel::retagged): (JSC::CodeLocationLabel:: const): (JSC::CodeLocationJump::CodeLocationJump): (JSC::CodeLocationJump::retagged): (JSC::CodeLocationCall::CodeLocationCall): (JSC::CodeLocationCall::retagged): (JSC::CodeLocationNearCall::CodeLocationNearCall): (JSC::CodeLocationDataLabel32::CodeLocationDataLabel32): (JSC::CodeLocationDataLabelCompact::CodeLocationDataLabelCompact): (JSC::CodeLocationDataLabelPtr::CodeLocationDataLabelPtr): (JSC::CodeLocationConvertibleLoad::CodeLocationConvertibleLoad): (JSC::CodeLocationCommon<tag>::instructionAtOffset): (JSC::CodeLocationCommon<tag>::labelAtOffset): (JSC::CodeLocationCommon<tag>::jumpAtOffset): (JSC::CodeLocationCommon<tag>::callAtOffset): (JSC::CodeLocationCommon<tag>::nearCallAtOffset): (JSC::CodeLocationCommon<tag>::dataLabelPtrAtOffset): (JSC::CodeLocationCommon<tag>::dataLabel32AtOffset): (JSC::CodeLocationCommon<tag>::dataLabelCompactAtOffset): (JSC::CodeLocationCommon<tag>::convertibleLoadAtOffset): (JSC::CodeLocationCommon::instructionAtOffset): Deleted. (JSC::CodeLocationCommon::labelAtOffset): Deleted. (JSC::CodeLocationCommon::jumpAtOffset): Deleted. (JSC::CodeLocationCommon::callAtOffset): Deleted. (JSC::CodeLocationCommon::nearCallAtOffset): Deleted. (JSC::CodeLocationCommon::dataLabelPtrAtOffset): Deleted. (JSC::CodeLocationCommon::dataLabel32AtOffset): Deleted. (JSC::CodeLocationCommon::dataLabelCompactAtOffset): Deleted. (JSC::CodeLocationCommon::convertibleLoadAtOffset): Deleted. * assembler/LinkBuffer.cpp: (JSC::LinkBuffer::finalizeCodeWithoutDisassemblyImpl): (JSC::LinkBuffer::finalizeCodeWithDisassemblyImpl): (JSC::LinkBuffer::finalizeCodeWithoutDisassembly): Deleted. (JSC::LinkBuffer::finalizeCodeWithDisassembly): Deleted. * assembler/LinkBuffer.h: (JSC::LinkBuffer::link): (JSC::LinkBuffer::patch): (JSC::LinkBuffer::entrypoint): (JSC::LinkBuffer::locationOf): (JSC::LinkBuffer::locationOfNearCall): (JSC::LinkBuffer::finalizeCodeWithoutDisassembly): (JSC::LinkBuffer::finalizeCodeWithDisassembly): (JSC::LinkBuffer::trampolineAt): * assembler/MacroAssemblerARM.h: (JSC::MacroAssemblerARM::readCallTarget): (JSC::MacroAssemblerARM::replaceWithJump): (JSC::MacroAssemblerARM::startOfPatchableBranch32WithPatchOnAddress): (JSC::MacroAssemblerARM::startOfPatchableBranchPtrWithPatchOnAddress): (JSC::MacroAssemblerARM::startOfBranchPtrWithPatchOnRegister): (JSC::MacroAssemblerARM::revertJumpReplacementToBranchPtrWithPatch): (JSC::MacroAssemblerARM::revertJumpReplacementToPatchableBranch32WithPatch): (JSC::MacroAssemblerARM::revertJumpReplacementToPatchableBranchPtrWithPatch): (JSC::MacroAssemblerARM::repatchCall): (JSC::MacroAssemblerARM::linkCall): * assembler/MacroAssemblerARM64.h: (JSC::MacroAssemblerARM64::readCallTarget): (JSC::MacroAssemblerARM64::replaceWithVMHalt): (JSC::MacroAssemblerARM64::replaceWithJump): (JSC::MacroAssemblerARM64::startOfBranchPtrWithPatchOnRegister): (JSC::MacroAssemblerARM64::startOfPatchableBranchPtrWithPatchOnAddress): (JSC::MacroAssemblerARM64::startOfPatchableBranch32WithPatchOnAddress): (JSC::MacroAssemblerARM64::revertJumpReplacementToBranchPtrWithPatch): (JSC::MacroAssemblerARM64::revertJumpReplacementToPatchableBranchPtrWithPatch): (JSC::MacroAssemblerARM64::revertJumpReplacementToPatchableBranch32WithPatch): (JSC::MacroAssemblerARM64::repatchCall): (JSC::MacroAssemblerARM64::linkCall): * assembler/MacroAssemblerARMv7.h: (JSC::MacroAssemblerARMv7::replaceWithJump): (JSC::MacroAssemblerARMv7::readCallTarget): (JSC::MacroAssemblerARMv7::startOfBranchPtrWithPatchOnRegister): (JSC::MacroAssemblerARMv7::revertJumpReplacementToBranchPtrWithPatch): (JSC::MacroAssemblerARMv7::startOfPatchableBranchPtrWithPatchOnAddress): (JSC::MacroAssemblerARMv7::startOfPatchableBranch32WithPatchOnAddress): (JSC::MacroAssemblerARMv7::revertJumpReplacementToPatchableBranchPtrWithPatch): (JSC::MacroAssemblerARMv7::revertJumpReplacementToPatchableBranch32WithPatch): (JSC::MacroAssemblerARMv7::repatchCall): (JSC::MacroAssemblerARMv7::linkCall): * assembler/MacroAssemblerCodeRef.cpp: (JSC::MacroAssemblerCodePtrBase::dumpWithName): (JSC::MacroAssemblerCodeRefBase::tryToDisassemble): (JSC::MacroAssemblerCodeRefBase::disassembly): (JSC::MacroAssemblerCodePtr::createLLIntCodePtr): Deleted. (JSC::MacroAssemblerCodePtr::dumpWithName const): Deleted. (JSC::MacroAssemblerCodePtr::dump const): Deleted. (JSC::MacroAssemblerCodeRef::createLLIntCodeRef): Deleted. (JSC::MacroAssemblerCodeRef::tryToDisassemble const): Deleted. (JSC::MacroAssemblerCodeRef::disassembly const): Deleted. (JSC::MacroAssemblerCodeRef::dump const): Deleted. * assembler/MacroAssemblerCodeRef.h: (JSC::FunctionPtr::FunctionPtr): (JSC::FunctionPtr::retagged const): (JSC::FunctionPtr::retaggedExecutableAddress const): (JSC::FunctionPtr::operator== const): (JSC::FunctionPtr::operator!= const): (JSC::ReturnAddressPtr::ReturnAddressPtr): (JSC::MacroAssemblerCodePtr::MacroAssemblerCodePtr): (JSC::MacroAssemblerCodePtr::createFromExecutableAddress): (JSC::MacroAssemblerCodePtr::retagged const): (JSC::MacroAssemblerCodePtr:: const): (JSC::MacroAssemblerCodePtr::dumpWithName const): (JSC::MacroAssemblerCodePtr::dump const): (JSC::MacroAssemblerCodePtrHash::hash): (JSC::MacroAssemblerCodePtrHash::equal): (JSC::MacroAssemblerCodeRef::MacroAssemblerCodeRef): (JSC::MacroAssemblerCodeRef::createSelfManagedCodeRef): (JSC::MacroAssemblerCodeRef::code const): (JSC::MacroAssemblerCodeRef::retaggedCode const): (JSC::MacroAssemblerCodeRef::retagged const): (JSC::MacroAssemblerCodeRef::tryToDisassemble const): (JSC::MacroAssemblerCodeRef::disassembly const): (JSC::MacroAssemblerCodeRef::dump const): (JSC::FunctionPtr<tag>::FunctionPtr): * assembler/MacroAssemblerMIPS.h: (JSC::MacroAssemblerMIPS::readCallTarget): (JSC::MacroAssemblerMIPS::replaceWithJump): (JSC::MacroAssemblerMIPS::startOfPatchableBranch32WithPatchOnAddress): (JSC::MacroAssemblerMIPS::startOfBranchPtrWithPatchOnRegister): (JSC::MacroAssemblerMIPS::revertJumpReplacementToBranchPtrWithPatch): (JSC::MacroAssemblerMIPS::startOfPatchableBranchPtrWithPatchOnAddress): (JSC::MacroAssemblerMIPS::revertJumpReplacementToPatchableBranch32WithPatch): (JSC::MacroAssemblerMIPS::revertJumpReplacementToPatchableBranchPtrWithPatch): (JSC::MacroAssemblerMIPS::repatchCall): (JSC::MacroAssemblerMIPS::linkCall): * assembler/MacroAssemblerX86.h: (JSC::MacroAssemblerX86::readCallTarget): (JSC::MacroAssemblerX86::startOfBranchPtrWithPatchOnRegister): (JSC::MacroAssemblerX86::startOfPatchableBranchPtrWithPatchOnAddress): (JSC::MacroAssemblerX86::startOfPatchableBranch32WithPatchOnAddress): (JSC::MacroAssemblerX86::revertJumpReplacementToBranchPtrWithPatch): (JSC::MacroAssemblerX86::revertJumpReplacementToPatchableBranchPtrWithPatch): (JSC::MacroAssemblerX86::revertJumpReplacementToPatchableBranch32WithPatch): (JSC::MacroAssemblerX86::repatchCall): (JSC::MacroAssemblerX86::linkCall): * assembler/MacroAssemblerX86Common.h: (JSC::MacroAssemblerX86Common::repatchCompact): (JSC::MacroAssemblerX86Common::replaceWithVMHalt): (JSC::MacroAssemblerX86Common::replaceWithJump): * assembler/MacroAssemblerX86_64.h: (JSC::MacroAssemblerX86_64::readCallTarget): (JSC::MacroAssemblerX86_64::startOfBranchPtrWithPatchOnRegister): (JSC::MacroAssemblerX86_64::startOfBranch32WithPatchOnRegister): (JSC::MacroAssemblerX86_64::startOfPatchableBranchPtrWithPatchOnAddress): (JSC::MacroAssemblerX86_64::startOfPatchableBranch32WithPatchOnAddress): (JSC::MacroAssemblerX86_64::revertJumpReplacementToPatchableBranchPtrWithPatch): (JSC::MacroAssemblerX86_64::revertJumpReplacementToPatchableBranch32WithPatch): (JSC::MacroAssemblerX86_64::revertJumpReplacementToBranchPtrWithPatch): (JSC::MacroAssemblerX86_64::repatchCall): (JSC::MacroAssemblerX86_64::linkCall): * assembler/testmasm.cpp: (JSC::compile): (JSC::invoke): (JSC::testProbeModifiesProgramCounter): * b3/B3Compilation.cpp: (JSC::B3::Compilation::Compilation): * b3/B3Compilation.h: (JSC::B3::Compilation::code const): (JSC::B3::Compilation::codeRef const): * b3/B3Compile.cpp: (JSC::B3::compile): * b3/B3LowerMacros.cpp: * b3/air/AirDisassembler.cpp: (JSC::B3::Air::Disassembler::dump): * b3/air/testair.cpp: * b3/testb3.cpp: (JSC::B3::invoke): (JSC::B3::testInterpreter): (JSC::B3::testEntrySwitchSimple): (JSC::B3::testEntrySwitchNoEntrySwitch): (JSC::B3::testEntrySwitchWithCommonPaths): (JSC::B3::testEntrySwitchWithCommonPathsAndNonTrivialEntrypoint): (JSC::B3::testEntrySwitchLoop): * bytecode/AccessCase.cpp: (JSC::AccessCase::generateImpl): * bytecode/AccessCaseSnippetParams.cpp: (JSC::SlowPathCallGeneratorWithArguments::generateImpl): * bytecode/ByValInfo.h: (JSC::ByValInfo::ByValInfo): * bytecode/CallLinkInfo.cpp: (JSC::CallLinkInfo::callReturnLocation): (JSC::CallLinkInfo::patchableJump): (JSC::CallLinkInfo::hotPathBegin): (JSC::CallLinkInfo::slowPathStart): * bytecode/CallLinkInfo.h: (JSC::CallLinkInfo::setCallLocations): (JSC::CallLinkInfo::hotPathOther): * bytecode/CodeBlock.cpp: (JSC::CodeBlock::finishCreation): * bytecode/GetByIdStatus.cpp: (JSC::GetByIdStatus::computeForStubInfoWithoutExitSiteFeedback): * bytecode/GetByIdVariant.cpp: (JSC::GetByIdVariant::GetByIdVariant): (JSC::GetByIdVariant::dumpInContext const): * bytecode/GetByIdVariant.h: (JSC::GetByIdVariant::customAccessorGetter const): * bytecode/GetterSetterAccessCase.cpp: (JSC::GetterSetterAccessCase::create): (JSC::GetterSetterAccessCase::GetterSetterAccessCase): (JSC::GetterSetterAccessCase::dumpImpl const): * bytecode/GetterSetterAccessCase.h: (JSC::GetterSetterAccessCase::customAccessor const): (): Deleted. * bytecode/HandlerInfo.h: (JSC::HandlerInfo::initialize): * bytecode/InlineAccess.cpp: (JSC::linkCodeInline): (JSC::InlineAccess::rewireStubAsJump): * bytecode/InlineAccess.h: * bytecode/JumpTable.h: (JSC::StringJumpTable::ctiForValue): (JSC::SimpleJumpTable::ctiForValue): * bytecode/LLIntCallLinkInfo.h: (JSC::LLIntCallLinkInfo::unlink): * bytecode/PolymorphicAccess.cpp: (JSC::AccessGenerationState::emitExplicitExceptionHandler): (JSC::PolymorphicAccess::regenerate): * bytecode/PolymorphicAccess.h: (JSC::AccessGenerationResult::AccessGenerationResult): (JSC::AccessGenerationResult::code const): * bytecode/StructureStubInfo.h: (JSC::StructureStubInfo::slowPathCallLocation): (JSC::StructureStubInfo::doneLocation): (JSC::StructureStubInfo::slowPathStartLocation): (JSC::StructureStubInfo::patchableJumpForIn): * dfg/DFGCommonData.h: (JSC::DFG::CommonData::appendCatchEntrypoint): * dfg/DFGDisassembler.cpp: (JSC::DFG::Disassembler::dumpDisassembly): * dfg/DFGDriver.h: * dfg/DFGJITCompiler.cpp: (JSC::DFG::JITCompiler::linkOSRExits): (JSC::DFG::JITCompiler::compileExceptionHandlers): (JSC::DFG::JITCompiler::link): (JSC::DFG::JITCompiler::compileFunction): (JSC::DFG::JITCompiler::noticeCatchEntrypoint): * dfg/DFGJITCompiler.h: (JSC::DFG::CallLinkRecord::CallLinkRecord): (JSC::DFG::JITCompiler::appendCall): (JSC::DFG::JITCompiler::JSCallRecord::JSCallRecord): (JSC::DFG::JITCompiler::JSDirectCallRecord::JSDirectCallRecord): (JSC::DFG::JITCompiler::JSDirectTailCallRecord::JSDirectTailCallRecord): * dfg/DFGJITFinalizer.cpp: (JSC::DFG::JITFinalizer::JITFinalizer): (JSC::DFG::JITFinalizer::finalize): (JSC::DFG::JITFinalizer::finalizeFunction): * dfg/DFGJITFinalizer.h: * dfg/DFGJumpReplacement.h: (JSC::DFG::JumpReplacement::JumpReplacement): * dfg/DFGNode.h: * dfg/DFGOSREntry.cpp: (JSC::DFG::prepareOSREntry): (JSC::DFG::prepareCatchOSREntry): * dfg/DFGOSREntry.h: (JSC::DFG::prepareOSREntry): * dfg/DFGOSRExit.cpp: (JSC::DFG::OSRExit::executeOSRExit): (JSC::DFG::reifyInlinedCallFrames): (JSC::DFG::adjustAndJumpToTarget): (JSC::DFG::OSRExit::codeLocationForRepatch const): (JSC::DFG::OSRExit::emitRestoreArguments): (JSC::DFG::OSRExit::compileOSRExit): * dfg/DFGOSRExit.h: * dfg/DFGOSRExitCompilerCommon.cpp: (JSC::DFG::handleExitCounts): (JSC::DFG::reifyInlinedCallFrames): (JSC::DFG::osrWriteBarrier): (JSC::DFG::adjustAndJumpToTarget): * dfg/DFGOperations.cpp: * dfg/DFGSlowPathGenerator.h: (JSC::DFG::CallResultAndArgumentsSlowPathGenerator::CallResultAndArgumentsSlowPathGenerator): (JSC::DFG::CallResultAndArgumentsSlowPathGenerator::unpackAndGenerate): (JSC::DFG::slowPathCall): * dfg/DFGSpeculativeJIT.cpp: (JSC::DFG::SpeculativeJIT::compileMathIC): (JSC::DFG::SpeculativeJIT::compileCallDOM): (JSC::DFG::SpeculativeJIT::compileCallDOMGetter): (JSC::DFG::SpeculativeJIT::emitSwitchIntJump): (JSC::DFG::SpeculativeJIT::emitSwitchImm): (JSC::DFG::SpeculativeJIT::emitSwitchStringOnString): (JSC::DFG::SpeculativeJIT::compileHasIndexedProperty): (JSC::DFG::SpeculativeJIT::compileGetDirectPname): (JSC::DFG::SpeculativeJIT::cachedPutById): * dfg/DFGSpeculativeJIT.h: (JSC::DFG::SpeculativeJIT::callOperation): (JSC::DFG::SpeculativeJIT::appendCall): (JSC::DFG::SpeculativeJIT::appendCallWithCallFrameRollbackOnException): (JSC::DFG::SpeculativeJIT::appendCallWithCallFrameRollbackOnExceptionSetResult): (JSC::DFG::SpeculativeJIT::appendCallSetResult): * dfg/DFGSpeculativeJIT64.cpp: (JSC::DFG::SpeculativeJIT::cachedGetById): (JSC::DFG::SpeculativeJIT::cachedGetByIdWithThis): (JSC::DFG::SpeculativeJIT::compile): * dfg/DFGThunks.cpp: (JSC::DFG::osrExitThunkGenerator): (JSC::DFG::osrExitGenerationThunkGenerator): (JSC::DFG::osrEntryThunkGenerator): * dfg/DFGThunks.h: * disassembler/ARM64Disassembler.cpp: (JSC::tryToDisassemble): * disassembler/ARMv7Disassembler.cpp: (JSC::tryToDisassemble): * disassembler/Disassembler.cpp: (JSC::disassemble): (JSC::disassembleAsynchronously): * disassembler/Disassembler.h: (JSC::tryToDisassemble): * disassembler/UDis86Disassembler.cpp: (JSC::tryToDisassembleWithUDis86): * disassembler/UDis86Disassembler.h: (JSC::tryToDisassembleWithUDis86): * disassembler/X86Disassembler.cpp: (JSC::tryToDisassemble): * ftl/FTLCompile.cpp: (JSC::FTL::compile): * ftl/FTLExceptionTarget.cpp: (JSC::FTL::ExceptionTarget::label): (JSC::FTL::ExceptionTarget::jumps): * ftl/FTLExceptionTarget.h: * ftl/FTLGeneratedFunction.h: * ftl/FTLJITCode.cpp: (JSC::FTL::JITCode::initializeB3Code): (JSC::FTL::JITCode::initializeAddressForCall): (JSC::FTL::JITCode::initializeArityCheckEntrypoint): (JSC::FTL::JITCode::addressForCall): (JSC::FTL::JITCode::executableAddressAtOffset): * ftl/FTLJITCode.h: (JSC::FTL::JITCode::b3Code const): * ftl/FTLJITFinalizer.cpp: (JSC::FTL::JITFinalizer::finalizeCommon): * ftl/FTLLazySlowPath.cpp: (JSC::FTL::LazySlowPath::initialize): (JSC::FTL::LazySlowPath::generate): * ftl/FTLLazySlowPath.h: (JSC::FTL::LazySlowPath::patchableJump const): (JSC::FTL::LazySlowPath::done const): (JSC::FTL::LazySlowPath::stub const): * ftl/FTLLazySlowPathCall.h: (JSC::FTL::createLazyCallGenerator): * ftl/FTLLink.cpp: (JSC::FTL::link): * ftl/FTLLowerDFGToB3.cpp: (JSC::FTL::DFG::LowerDFGToB3::lower): (JSC::FTL::DFG::LowerDFGToB3::compileCallOrConstruct): (JSC::FTL::DFG::LowerDFGToB3::compileDirectCallOrConstruct): (JSC::FTL::DFG::LowerDFGToB3::compileTailCall): (JSC::FTL::DFG::LowerDFGToB3::compileCallOrConstructVarargsSpread): (JSC::FTL::DFG::LowerDFGToB3::compileCallOrConstructVarargs): (JSC::FTL::DFG::LowerDFGToB3::compileCallEval): (JSC::FTL::DFG::LowerDFGToB3::compileInvalidationPoint): (JSC::FTL::DFG::LowerDFGToB3::compileIn): (JSC::FTL::DFG::LowerDFGToB3::compileCheckSubClass): (JSC::FTL::DFG::LowerDFGToB3::compileCallDOM): (JSC::FTL::DFG::LowerDFGToB3::compileCallDOMGetter): (JSC::FTL::DFG::LowerDFGToB3::lazySlowPath): * ftl/FTLOSRExit.cpp: (JSC::FTL::OSRExit::codeLocationForRepatch const): * ftl/FTLOSRExit.h: * ftl/FTLOSRExitCompiler.cpp: (JSC::FTL::compileStub): (JSC::FTL::compileFTLOSRExit): * ftl/FTLOSRExitHandle.cpp: (JSC::FTL::OSRExitHandle::emitExitThunk): * ftl/FTLOperations.cpp: (JSC::FTL::compileFTLLazySlowPath): * ftl/FTLPatchpointExceptionHandle.cpp: (JSC::FTL::PatchpointExceptionHandle::scheduleExitCreationForUnwind): * ftl/FTLSlowPathCall.cpp: (JSC::FTL::SlowPathCallContext::keyWithTarget const): (JSC::FTL::SlowPathCallContext::makeCall): * ftl/FTLSlowPathCall.h: (JSC::FTL::callOperation): * ftl/FTLSlowPathCallKey.cpp: (JSC::FTL::SlowPathCallKey::dump const): * ftl/FTLSlowPathCallKey.h: (JSC::FTL::SlowPathCallKey::SlowPathCallKey): (JSC::FTL::SlowPathCallKey::callTarget const): (JSC::FTL::SlowPathCallKey::withCallTarget): (JSC::FTL::SlowPathCallKey::hash const): (JSC::FTL::SlowPathCallKey::callPtrTag const): Deleted. * ftl/FTLState.cpp: (JSC::FTL::State::State): * ftl/FTLThunks.cpp: (JSC::FTL::genericGenerationThunkGenerator): (JSC::FTL::osrExitGenerationThunkGenerator): (JSC::FTL::lazySlowPathGenerationThunkGenerator): (JSC::FTL::slowPathCallThunkGenerator): * ftl/FTLThunks.h: (JSC::FTL::generateIfNecessary): (JSC::FTL::keyForThunk): (JSC::FTL::Thunks::getSlowPathCallThunk): (JSC::FTL::Thunks::keyForSlowPathCallThunk): * interpreter/InterpreterInlines.h: (JSC::Interpreter::getOpcodeID): * jit/AssemblyHelpers.cpp: (JSC::AssemblyHelpers::callExceptionFuzz): (JSC::AssemblyHelpers::emitDumbVirtualCall): (JSC::AssemblyHelpers::debugCall): * jit/CCallHelpers.cpp: (JSC::CCallHelpers::ensureShadowChickenPacket): * jit/ExecutableAllocator.cpp: (JSC::FixedVMPoolExecutableAllocator::initializeSeparatedWXHeaps): (JSC::FixedVMPoolExecutableAllocator::jitWriteThunkGenerator): * jit/ExecutableAllocator.h: (JSC::performJITMemcpy): * jit/GCAwareJITStubRoutine.cpp: (JSC::GCAwareJITStubRoutine::GCAwareJITStubRoutine): (JSC::MarkingGCAwareJITStubRoutine::MarkingGCAwareJITStubRoutine): (JSC::GCAwareJITStubRoutineWithExceptionHandler::GCAwareJITStubRoutineWithExceptionHandler): (JSC::createJITStubRoutine): * jit/GCAwareJITStubRoutine.h: (JSC::createJITStubRoutine): * jit/JIT.cpp: (JSC::ctiPatchCallByReturnAddress): (JSC::JIT::compileWithoutLinking): (JSC::JIT::link): (JSC::JIT::privateCompileExceptionHandlers): * jit/JIT.h: (JSC::CallRecord::CallRecord): * jit/JITArithmetic.cpp: (JSC::JIT::emitMathICFast): (JSC::JIT::emitMathICSlow): * jit/JITCall.cpp: (JSC::JIT::compileOpCallSlowCase): * jit/JITCall32_64.cpp: (JSC::JIT::compileOpCallSlowCase): * jit/JITCode.cpp: (JSC::JITCodeWithCodeRef::JITCodeWithCodeRef): (JSC::JITCodeWithCodeRef::executableAddressAtOffset): (JSC::DirectJITCode::DirectJITCode): (JSC::DirectJITCode::initializeCodeRef): (JSC::DirectJITCode::addressForCall): (JSC::NativeJITCode::NativeJITCode): (JSC::NativeJITCode::initializeCodeRef): (JSC::NativeJITCode::addressForCall): * jit/JITCode.h: * jit/JITCodeMap.h: (JSC::JITCodeMap::Entry::Entry): (JSC::JITCodeMap::Entry::codeLocation): (JSC::JITCodeMap::append): (JSC::JITCodeMap::find const): * jit/JITDisassembler.cpp: (JSC::JITDisassembler::dumpDisassembly): * jit/JITExceptions.cpp: (JSC::genericUnwind): * jit/JITInlineCacheGenerator.cpp: (JSC::JITByIdGenerator::finalize): * jit/JITInlines.h: (JSC::JIT::emitNakedCall): (JSC::JIT::emitNakedTailCall): (JSC::JIT::appendCallWithExceptionCheck): (JSC::JIT::appendCallWithExceptionCheckAndSlowPathReturnType): (JSC::JIT::appendCallWithCallFrameRollbackOnException): (JSC::JIT::appendCallWithExceptionCheckSetJSValueResult): (JSC::JIT::appendCallWithExceptionCheckSetJSValueResultWithProfile): * jit/JITMathIC.h: (JSC::isProfileEmpty): * jit/JITOpcodes.cpp: (JSC::JIT::emit_op_catch): (JSC::JIT::emit_op_switch_imm): (JSC::JIT::emit_op_switch_char): (JSC::JIT::emit_op_switch_string): (JSC::JIT::privateCompileHasIndexedProperty): (JSC::JIT::emitSlow_op_has_indexed_property): * jit/JITOpcodes32_64.cpp: (JSC::JIT::privateCompileHasIndexedProperty): * jit/JITOperations.cpp: (JSC::getByVal): * jit/JITPropertyAccess.cpp: (JSC::JIT::stringGetByValStubGenerator): (JSC::JIT::emitGetByValWithCachedId): (JSC::JIT::emitSlow_op_get_by_val): (JSC::JIT::emitPutByValWithCachedId): (JSC::JIT::emitSlow_op_put_by_val): (JSC::JIT::emitSlow_op_try_get_by_id): (JSC::JIT::emitSlow_op_get_by_id_direct): (JSC::JIT::emitSlow_op_get_by_id): (JSC::JIT::emitSlow_op_get_by_id_with_this): (JSC::JIT::emitSlow_op_put_by_id): (JSC::JIT::privateCompileGetByVal): (JSC::JIT::privateCompileGetByValWithCachedId): (JSC::JIT::privateCompilePutByVal): (JSC::JIT::privateCompilePutByValWithCachedId): * jit/JITPropertyAccess32_64.cpp: (JSC::JIT::stringGetByValStubGenerator): (JSC::JIT::emitSlow_op_get_by_val): (JSC::JIT::emitSlow_op_put_by_val): * jit/JITStubRoutine.h: (JSC::JITStubRoutine::JITStubRoutine): (JSC::JITStubRoutine::createSelfManagedRoutine): (JSC::JITStubRoutine::code const): (JSC::JITStubRoutine::asCodePtr): * jit/JITThunks.cpp: (JSC::JITThunks::ctiNativeCall): (JSC::JITThunks::ctiNativeConstruct): (JSC::JITThunks::ctiNativeTailCall): (JSC::JITThunks::ctiNativeTailCallWithoutSavedTags): (JSC::JITThunks::ctiInternalFunctionCall): (JSC::JITThunks::ctiInternalFunctionConstruct): (JSC::JITThunks::ctiStub): (JSC::JITThunks::existingCTIStub): (JSC::JITThunks::hostFunctionStub): * jit/JITThunks.h: * jit/PCToCodeOriginMap.cpp: (JSC::PCToCodeOriginMap::PCToCodeOriginMap): * jit/PCToCodeOriginMap.h: * jit/PolymorphicCallStubRoutine.cpp: (JSC::PolymorphicCallStubRoutine::PolymorphicCallStubRoutine): * jit/PolymorphicCallStubRoutine.h: * jit/Repatch.cpp: (JSC::readPutICCallTarget): (JSC::ftlThunkAwareRepatchCall): (JSC::appropriateOptimizingGetByIdFunction): (JSC::appropriateGetByIdFunction): (JSC::tryCacheGetByID): (JSC::repatchGetByID): (JSC::tryCachePutByID): (JSC::repatchPutByID): (JSC::tryCacheIn): (JSC::repatchIn): (JSC::linkSlowFor): (JSC::linkFor): (JSC::linkDirectFor): (JSC::revertCall): (JSC::unlinkFor): (JSC::linkVirtualFor): (JSC::linkPolymorphicCall): (JSC::resetGetByID): (JSC::resetPutByID): * jit/Repatch.h: * jit/SlowPathCall.h: (JSC::JITSlowPathCall::call): * jit/SpecializedThunkJIT.h: (JSC::SpecializedThunkJIT::finalize): (JSC::SpecializedThunkJIT::callDoubleToDouble): (JSC::SpecializedThunkJIT::callDoubleToDoublePreservingReturn): * jit/ThunkGenerator.h: * jit/ThunkGenerators.cpp: (JSC::throwExceptionFromCallSlowPathGenerator): (JSC::slowPathFor): (JSC::linkCallThunkGenerator): (JSC::linkPolymorphicCallThunkGenerator): (JSC::virtualThunkFor): (JSC::nativeForGenerator): (JSC::nativeCallGenerator): (JSC::nativeTailCallGenerator): (JSC::nativeTailCallWithoutSavedTagsGenerator): (JSC::nativeConstructGenerator): (JSC::internalFunctionCallGenerator): (JSC::internalFunctionConstructGenerator): (JSC::arityFixupGenerator): (JSC::unreachableGenerator): (JSC::charCodeAtThunkGenerator): (JSC::charAtThunkGenerator): (JSC::fromCharCodeThunkGenerator): (JSC::clz32ThunkGenerator): (JSC::sqrtThunkGenerator): (JSC::floorThunkGenerator): (JSC::ceilThunkGenerator): (JSC::truncThunkGenerator): (JSC::roundThunkGenerator): (JSC::expThunkGenerator): (JSC::logThunkGenerator): (JSC::absThunkGenerator): (JSC::imulThunkGenerator): (JSC::randomThunkGenerator): (JSC::boundThisNoArgsFunctionCallGenerator): * jit/ThunkGenerators.h: * llint/LLIntData.cpp: (JSC::LLInt::initialize): * llint/LLIntData.h: (JSC::LLInt::getExecutableAddress): (JSC::LLInt::getCodePtr): (JSC::LLInt::getCodeRef): (JSC::LLInt::getCodeFunctionPtr): * llint/LLIntEntrypoint.cpp: (JSC::LLInt::setFunctionEntrypoint): (JSC::LLInt::setEvalEntrypoint): (JSC::LLInt::setProgramEntrypoint): (JSC::LLInt::setModuleProgramEntrypoint): * llint/LLIntExceptions.cpp: (JSC::LLInt::callToThrow): * llint/LLIntSlowPaths.cpp: (JSC::LLInt::LLINT_SLOW_PATH_DECL): (JSC::LLInt::setUpCall): * llint/LLIntThunks.cpp: (JSC::vmEntryToWasm): (JSC::LLInt::generateThunkWithJumpTo): (JSC::LLInt::functionForCallEntryThunkGenerator): (JSC::LLInt::functionForConstructEntryThunkGenerator): (JSC::LLInt::functionForCallArityCheckThunkGenerator): (JSC::LLInt::functionForConstructArityCheckThunkGenerator): (JSC::LLInt::evalEntryThunkGenerator): (JSC::LLInt::programEntryThunkGenerator): (JSC::LLInt::moduleProgramEntryThunkGenerator): * llint/LLIntThunks.h: * llint/LowLevelInterpreter.asm: * llint/LowLevelInterpreter32_64.asm: * llint/LowLevelInterpreter64.asm: * profiler/ProfilerCompilation.cpp: (JSC::Profiler::Compilation::addOSRExitSite): * profiler/ProfilerCompilation.h: * profiler/ProfilerOSRExitSite.cpp: (JSC::Profiler::OSRExitSite::toJS const): * profiler/ProfilerOSRExitSite.h: (JSC::Profiler::OSRExitSite::OSRExitSite): (JSC::Profiler::OSRExitSite::codeAddress const): (JSC::Profiler::OSRExitSite:: const): Deleted. * runtime/ExecutableBase.cpp: (JSC::ExecutableBase::clearCode): * runtime/ExecutableBase.h: (JSC::ExecutableBase::entrypointFor): * runtime/NativeExecutable.cpp: (JSC::NativeExecutable::finishCreation): * runtime/NativeFunction.h: (JSC::TaggedNativeFunction::TaggedNativeFunction): (JSC::TaggedNativeFunction::operator NativeFunction): * runtime/PtrTag.h: (JSC::tagCodePtr): (JSC::untagCodePtr): (JSC::retagCodePtr): (JSC::tagCFunctionPtr): (JSC::untagCFunctionPtr): (JSC::nextPtrTagID): Deleted. * runtime/PutPropertySlot.h: (JSC::PutPropertySlot::PutPropertySlot): (JSC::PutPropertySlot::setCustomValue): (JSC::PutPropertySlot::setCustomAccessor): (JSC::PutPropertySlot::customSetter const): * runtime/ScriptExecutable.cpp: (JSC::ScriptExecutable::installCode): * runtime/VM.cpp: (JSC::VM::getHostFunction): (JSC::VM::getCTIInternalFunctionTrampolineFor): * runtime/VM.h: (JSC::VM::getCTIStub): * wasm/WasmB3IRGenerator.cpp: (JSC::Wasm::B3IRGenerator::B3IRGenerator): (JSC::Wasm::B3IRGenerator::emitExceptionCheck): (JSC::Wasm::B3IRGenerator::emitTierUpCheck): (JSC::Wasm::B3IRGenerator::addCall): (JSC::Wasm::B3IRGenerator::addCallIndirect): * wasm/WasmBBQPlan.cpp: (JSC::Wasm::BBQPlan::prepare): (JSC::Wasm::BBQPlan::complete): * wasm/WasmBBQPlan.h: * wasm/WasmBinding.cpp: (JSC::Wasm::wasmToWasm): * wasm/WasmBinding.h: * wasm/WasmCallee.h: (JSC::Wasm::Callee::entrypoint const): * wasm/WasmCallingConvention.h: (JSC::Wasm::CallingConvention::setupFrameInPrologue const): * wasm/WasmCodeBlock.h: (JSC::Wasm::CodeBlock::entrypointLoadLocationFromFunctionIndexSpace): * wasm/WasmFaultSignalHandler.cpp: (JSC::Wasm::trapHandler): * wasm/WasmFormat.h: * wasm/WasmInstance.h: * wasm/WasmOMGPlan.cpp: (JSC::Wasm::OMGPlan::work): * wasm/WasmThunks.cpp: (JSC::Wasm::throwExceptionFromWasmThunkGenerator): (JSC::Wasm::throwStackOverflowFromWasmThunkGenerator): (JSC::Wasm::triggerOMGTierUpThunkGenerator): (JSC::Wasm::Thunks::stub): (JSC::Wasm::Thunks::existingStub): * wasm/WasmThunks.h: * wasm/js/JSToWasm.cpp: (JSC::Wasm::createJSToWasmWrapper): * wasm/js/JSWebAssemblyCodeBlock.h: * wasm/js/WasmToJS.cpp: (JSC::Wasm::handleBadI64Use): (JSC::Wasm::wasmToJS): * wasm/js/WasmToJS.h: * wasm/js/WebAssemblyFunction.h: * yarr/YarrJIT.cpp: (JSC::Yarr::YarrGenerator::loadFromFrameAndJump): (JSC::Yarr::YarrGenerator::BacktrackingState::linkDataLabels): (JSC::Yarr::YarrGenerator::compile): * yarr/YarrJIT.h: (JSC::Yarr::YarrCodeBlock::set8BitCode): (JSC::Yarr::YarrCodeBlock::set16BitCode): (JSC::Yarr::YarrCodeBlock::set8BitCodeMatchOnly): (JSC::Yarr::YarrCodeBlock::set16BitCodeMatchOnly): (JSC::Yarr::YarrCodeBlock::execute): (JSC::Yarr::YarrCodeBlock::clear): Source/WebCore: No new tests. This is covered by existing tests. * WebCore.xcodeproj/project.pbxproj: * css/ElementRuleCollector.cpp: (WebCore::ElementRuleCollector::ruleMatches): * cssjit/CSSPtrTag.h: Added. * cssjit/CompiledSelector.h: * cssjit/FunctionCall.h: (WebCore::FunctionCall::FunctionCall): (WebCore::FunctionCall::setFunctionAddress): (WebCore::FunctionCall::prepareAndCall): * cssjit/SelectorCompiler.cpp: (WebCore::SelectorCompiler::compileSelector): (WebCore::SelectorCompiler::SelectorFragment::appendUnoptimizedPseudoClassWithContext): (WebCore::SelectorCompiler::addPseudoClassType): (WebCore::SelectorCompiler::SelectorCodeGenerator::compile): (WebCore::SelectorCompiler::SelectorCodeGenerator::generateElementAttributeFunctionCallValueMatching): (WebCore::SelectorCompiler::SelectorCodeGenerator::generateElementFunctionCallTest): (WebCore::SelectorCompiler::SelectorCodeGenerator::generateContextFunctionCallTest): * cssjit/SelectorCompiler.h: (WebCore::SelectorCompiler::ruleCollectorSimpleSelectorCheckerFunction): (WebCore::SelectorCompiler::querySelectorSimpleSelectorCheckerFunction): (WebCore::SelectorCompiler::ruleCollectorSelectorCheckerFunctionWithCheckingContext): (WebCore::SelectorCompiler::querySelectorSelectorCheckerFunctionWithCheckingContext): * dom/SelectorQuery.cpp: (WebCore::SelectorDataList::executeCompiledSingleMultiSelectorData const): (WebCore::SelectorDataList::execute const): * dom/SelectorQuery.h: Canonical link: https://commits.webkit.org/200234@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@230748 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2018-04-18 03:31:09 +00:00
SlowPathCallKey SlowPathCallContext::keyWithTarget(FunctionPtr<CFunctionPtrTag> callTarget) const
Merge r170564, r170571, r170604, r170628, r170672, r170680, r170724, r170728, r170729, r170819, r170821, r170836, r170855, r170860, r170890, r170907, r170929, r171052, r171106, r171152, r171153, r171214 from ftlopt. Source/JavaScriptCore: This part of the merge delivers roughly a 2% across-the-board performance improvement, mostly due to immutable property inference and DFG-side GCSE. It also almost completely resolves accessor performance issues; in the common case the DFG will compile a getter/setter access into code that is just as efficient as a normal property access. Another major highlight of this part of the merge is the work to add a type profiler to the inspector. This work is still on-going but this greatly increases coverage. Note that this merge fixes a minor bug in the GetterSetter refactoring from http://trac.webkit.org/changeset/170729 (https://bugs.webkit.org/show_bug.cgi?id=134518). It also adds a new tests to tests/stress to cover that bug. That bug was previously only covered by layout tests. 2014-07-17 Filip Pizlo <fpizlo@apple.com> [ftlopt] DFG Flush(SetLocal) store elimination is overzealous for captured variables in the presence of nodes that have no effects but may throw (merge trunk r171190) https://bugs.webkit.org/show_bug.cgi?id=135019 Reviewed by Oliver Hunt. Behaviorally, this is just a merge of trunk r171190, except that the relevant functionality has moved to StrengthReductionPhase and is written in a different style. Same algorithm, different code. * dfg/DFGNodeType.h: * dfg/DFGStrengthReductionPhase.cpp: (JSC::DFG::StrengthReductionPhase::handleNode): * tests/stress/capture-escape-and-throw.js: Added. (foo.f): (foo): * tests/stress/new-array-with-size-throw-exception-and-tear-off-arguments.js: Added. (foo): (bar): 2014-07-15 Filip Pizlo <fpizlo@apple.com> [ftlopt] Constant fold GetGetter and GetSetter if the GetterSetter is a constant https://bugs.webkit.org/show_bug.cgi?id=134962 Reviewed by Oliver Hunt. This removes yet another steady-state-throughput implication of using getters and setters: if your accessor call is monomorphic then you'll just get a structure check, nothing more. No more loads to get to the GetterSetter object or the accessor function object. * dfg/DFGAbstractInterpreterInlines.h: (JSC::DFG::AbstractInterpreter<AbstractStateType>::executeEffects): * runtime/GetterSetter.h: (JSC::GetterSetter::getterConcurrently): (JSC::GetterSetter::setGetter): (JSC::GetterSetter::setterConcurrently): (JSC::GetterSetter::setSetter): 2014-07-15 Filip Pizlo <fpizlo@apple.com> [ftlopt] Identity replacement in CSE shouldn't create a Phantom over the Identity's children https://bugs.webkit.org/show_bug.cgi?id=134893 Reviewed by Oliver Hunt. Replace Identity with Check instead of Phantom. Phantom means that the child of the Identity should be unconditionally live. The liveness semantics of Identity are such that if the parents of Identity are live then the child is live. Removing the Identity entirely preserves such liveness semantics. So, the only thing that should be left behind is the type check on the child, which is what Check means: do the check but don't keep the child alive if the check isn't needed. * dfg/DFGCSEPhase.cpp: * dfg/DFGNode.h: (JSC::DFG::Node::convertToCheck): 2014-07-13 Filip Pizlo <fpizlo@apple.com> [ftlopt] DFG should be able to do GCSE in SSA and this should be unified with the CSE in CPS, and both of these things should use abstract heaps for reasoning about effects https://bugs.webkit.org/show_bug.cgi?id=134677 Reviewed by Sam Weinig. This removes the old local CSE phase, which was based on manually written backward-search rules for all of the different kinds of things we cared about, and adds a new local/global CSE (local for CPS and global for SSA) that leaves the node semantics almost entirely up to clobberize(). Thus, the CSE phase itself just worries about the algorithms and data structures used for storing sets of available values. This results in a large reduction in code size in CSEPhase.cpp while greatly increasing the phase's power (since it now does global CSE) and reducing compile time (since local CSE is now rewritten to use smarter data structures). Even though LLVM was already running GVN, the extra GCSE at DFG IR level means that this is a significant (~0.7%) throughput improvement. This work is based on the concept of "def" to clobberize(). If clobberize() calls def(), it means that the node being analyzed makes available some value in some DFG node, and that future attempts to compute that value can simply use that node. In other words, it establishes an available value mapping of the form value=>node. There are two kinds of values that can be passed to def(): PureValue. This captures everything needed to determine whether two pure nodes - nodes that neither read nor write, and produce a value that is a CSE candidate - are identical. It carries the NodeType, an AdjacencyList, and one word of meta-data. The meta-data is usually used for things like the arithmetic mode or constant pointer. Passing a PureValue to def() means that the node produces a value that is valid anywhere that the node dominates. HeapLocation. This describes a location in the heap that could be written to or read from. Both stores and loads can def() a HeapLocation. HeapLocation carries around an abstract heap that both serves as part of the "name" of the heap location (together with the other fields of HeapLocation) and also tells us what write()'s to watch for. If someone write()'s to an abstract heap that overlaps the heap associated with the HeapLocation, then it means that the values for that location are no longer available. This approach is sufficiently clever that the CSEPhase itself can focus on the mechanism of tracking the PureValue=>node and HeapLocation=>node maps, without having to worry about interpreting the semantics of different DFG node types - that is now almost entirely in clobberize(). The only things we special-case inside CSEPhase are the Identity node, which CSE is traditionally responsible for eliminating even though it has nothing to do with CSE, and the LocalCSE rule for turning PutByVal into PutByValAlias. This is a slight Octane, SunSpider, and Kraken speed-up - all somewhere arond 0.7% . It's not a bigger win because LLVM was already giving us most of what we needed in its GVN. Also, the SunSpider speed-up isn't from GCSE as much as it's a clean-up of local CSE - that is no longer O(n^2). Basically this is purely good: it reduces the amount of LLVM IR we generate, it removes the old CSE's heap modeling (which was a constant source of bugs), and it improves both the quality of the code we generate and the speed with which we generate it. Also, any future optimizations that depend on GCSE will now be easier to implement. During the development of this patch I also rationalized some other stuff, like Graph's ordered traversals - we now have preorder and postorder rather than just "depth first". * CMakeLists.txt: * JavaScriptCore.vcxproj/JavaScriptCore.vcxproj: * JavaScriptCore.xcodeproj/project.pbxproj: * dfg/DFGAbstractHeap.h: * dfg/DFGAdjacencyList.h: (JSC::DFG::AdjacencyList::hash): (JSC::DFG::AdjacencyList::operator==): * dfg/DFGBasicBlock.h: * dfg/DFGCSEPhase.cpp: (JSC::DFG::performLocalCSE): (JSC::DFG::performGlobalCSE): (JSC::DFG::CSEPhase::CSEPhase): Deleted. (JSC::DFG::CSEPhase::run): Deleted. (JSC::DFG::CSEPhase::endIndexForPureCSE): Deleted. (JSC::DFG::CSEPhase::pureCSE): Deleted. (JSC::DFG::CSEPhase::constantCSE): Deleted. (JSC::DFG::CSEPhase::constantStoragePointerCSE): Deleted. (JSC::DFG::CSEPhase::getCalleeLoadElimination): Deleted. (JSC::DFG::CSEPhase::getArrayLengthElimination): Deleted. (JSC::DFG::CSEPhase::globalVarLoadElimination): Deleted. (JSC::DFG::CSEPhase::scopedVarLoadElimination): Deleted. (JSC::DFG::CSEPhase::varInjectionWatchpointElimination): Deleted. (JSC::DFG::CSEPhase::getByValLoadElimination): Deleted. (JSC::DFG::CSEPhase::checkFunctionElimination): Deleted. (JSC::DFG::CSEPhase::checkExecutableElimination): Deleted. (JSC::DFG::CSEPhase::checkStructureElimination): Deleted. (JSC::DFG::CSEPhase::structureTransitionWatchpointElimination): Deleted. (JSC::DFG::CSEPhase::getByOffsetLoadElimination): Deleted. (JSC::DFG::CSEPhase::getGetterSetterByOffsetLoadElimination): Deleted. (JSC::DFG::CSEPhase::getPropertyStorageLoadElimination): Deleted. (JSC::DFG::CSEPhase::checkArrayElimination): Deleted. (JSC::DFG::CSEPhase::getIndexedPropertyStorageLoadElimination): Deleted. (JSC::DFG::CSEPhase::getInternalFieldLoadElimination): Deleted. (JSC::DFG::CSEPhase::getMyScopeLoadElimination): Deleted. (JSC::DFG::CSEPhase::getLocalLoadElimination): Deleted. (JSC::DFG::CSEPhase::invalidationPointElimination): Deleted. (JSC::DFG::CSEPhase::setReplacement): Deleted. (JSC::DFG::CSEPhase::eliminate): Deleted. (JSC::DFG::CSEPhase::performNodeCSE): Deleted. (JSC::DFG::CSEPhase::performBlockCSE): Deleted. (JSC::DFG::performCSE): Deleted. * dfg/DFGCSEPhase.h: * dfg/DFGClobberSet.cpp: (JSC::DFG::addReads): (JSC::DFG::addWrites): (JSC::DFG::addReadsAndWrites): (JSC::DFG::readsOverlap): (JSC::DFG::writesOverlap): * dfg/DFGClobberize.cpp: (JSC::DFG::doesWrites): (JSC::DFG::accessesOverlap): (JSC::DFG::writesOverlap): * dfg/DFGClobberize.h: (JSC::DFG::clobberize): (JSC::DFG::NoOpClobberize::operator()): (JSC::DFG::CheckClobberize::operator()): (JSC::DFG::ReadMethodClobberize::ReadMethodClobberize): (JSC::DFG::ReadMethodClobberize::operator()): (JSC::DFG::WriteMethodClobberize::WriteMethodClobberize): (JSC::DFG::WriteMethodClobberize::operator()): (JSC::DFG::DefMethodClobberize::DefMethodClobberize): (JSC::DFG::DefMethodClobberize::operator()): * dfg/DFGDCEPhase.cpp: (JSC::DFG::DCEPhase::run): (JSC::DFG::DCEPhase::fixupBlock): * dfg/DFGGraph.cpp: (JSC::DFG::Graph::getBlocksInPreOrder): (JSC::DFG::Graph::getBlocksInPostOrder): (JSC::DFG::Graph::addForDepthFirstSort): Deleted. (JSC::DFG::Graph::getBlocksInDepthFirstOrder): Deleted. * dfg/DFGGraph.h: * dfg/DFGHeapLocation.cpp: Added. (JSC::DFG::HeapLocation::dump): (WTF::printInternal): * dfg/DFGHeapLocation.h: Added. (JSC::DFG::HeapLocation::HeapLocation): (JSC::DFG::HeapLocation::operator!): (JSC::DFG::HeapLocation::kind): (JSC::DFG::HeapLocation::heap): (JSC::DFG::HeapLocation::base): (JSC::DFG::HeapLocation::index): (JSC::DFG::HeapLocation::hash): (JSC::DFG::HeapLocation::operator==): (JSC::DFG::HeapLocation::isHashTableDeletedValue): (JSC::DFG::HeapLocationHash::hash): (JSC::DFG::HeapLocationHash::equal): * dfg/DFGLICMPhase.cpp: (JSC::DFG::LICMPhase::run): * dfg/DFGNode.h: (JSC::DFG::Node::replaceWith): (JSC::DFG::Node::convertToPhantomUnchecked): Deleted. * dfg/DFGPlan.cpp: (JSC::DFG::Plan::compileInThreadImpl): * dfg/DFGPureValue.cpp: Added. (JSC::DFG::PureValue::dump): * dfg/DFGPureValue.h: Added. (JSC::DFG::PureValue::PureValue): (JSC::DFG::PureValue::operator!): (JSC::DFG::PureValue::op): (JSC::DFG::PureValue::children): (JSC::DFG::PureValue::info): (JSC::DFG::PureValue::hash): (JSC::DFG::PureValue::operator==): (JSC::DFG::PureValue::isHashTableDeletedValue): (JSC::DFG::PureValueHash::hash): (JSC::DFG::PureValueHash::equal): * dfg/DFGSSAConversionPhase.cpp: (JSC::DFG::SSAConversionPhase::run): * ftl/FTLLowerDFGToLLVM.cpp: (JSC::FTL::LowerDFGToLLVM::lower): 2014-07-13 Filip Pizlo <fpizlo@apple.com> Unreviewed, revert unintended change in r171051. * dfg/DFGCSEPhase.cpp: 2014-07-08 Filip Pizlo <fpizlo@apple.com> [ftlopt] Move Flush(SetLocal) store elimination to StrengthReductionPhase https://bugs.webkit.org/show_bug.cgi?id=134739 Reviewed by Mark Hahnenberg. I'm going to streamline CSE around clobberize() as part of https://bugs.webkit.org/show_bug.cgi?id=134677, and so Flush(SetLocal) store elimination wouldn't belong in CSE anymore. It doesn't quite belong anywhere, which means that it belongs in StrengthReductionPhase, since that's intended to be our dumping ground. To do this I had to add some missing smarts to clobberize(). Previously clobberize() could play a bit loose with reads of Variables because it wasn't used for store elimination. The main client of read() was LICM, but it would only use it to determine hoistability and anything that did a write() was not hoistable - so, we had benign (but still wrong) missing read() calls in places that did write()s. This fixes a bunch of those cases. * dfg/DFGCSEPhase.cpp: (JSC::DFG::CSEPhase::performNodeCSE): (JSC::DFG::CSEPhase::setLocalStoreElimination): Deleted. * dfg/DFGClobberize.cpp: (JSC::DFG::accessesOverlap): * dfg/DFGClobberize.h: (JSC::DFG::clobberize): Make clobberize() smart enough for detecting when this store elimination would be sound. * dfg/DFGStrengthReductionPhase.cpp: (JSC::DFG::StrengthReductionPhase::handleNode): Implement the store elimination in terms of clobberize(). 2014-07-08 Filip Pizlo <fpizlo@apple.com> [ftlopt] Phantom simplification should be in its own phase https://bugs.webkit.org/show_bug.cgi?id=134742 Reviewed by Geoffrey Garen. This moves Phantom simplification out of CSE, which greatly simplifies CSE and gives it more focus. Also this finally adds a phase that removes empty Phantoms. We sort of had this in CPSRethreading, but that phase runs too infrequently and doesn't run at all for SSA. * CMakeLists.txt: * JavaScriptCore.vcxproj/JavaScriptCore.vcxproj: * JavaScriptCore.xcodeproj/project.pbxproj: * dfg/DFGAdjacencyList.h: * dfg/DFGCSEPhase.cpp: (JSC::DFG::CSEPhase::run): (JSC::DFG::CSEPhase::setReplacement): (JSC::DFG::CSEPhase::eliminate): (JSC::DFG::CSEPhase::performNodeCSE): (JSC::DFG::CSEPhase::eliminateIrrelevantPhantomChildren): Deleted. * dfg/DFGPhantomRemovalPhase.cpp: Added. (JSC::DFG::PhantomRemovalPhase::PhantomRemovalPhase): (JSC::DFG::PhantomRemovalPhase::run): (JSC::DFG::performCleanUp): * dfg/DFGPhantomRemovalPhase.h: Added. * dfg/DFGPlan.cpp: (JSC::DFG::Plan::compileInThreadImpl): 2014-07-08 Filip Pizlo <fpizlo@apple.com> [ftlopt] Get rid of Node::misc by moving the fields out of the union so that you can use replacement and owner simultaneously https://bugs.webkit.org/show_bug.cgi?id=134730 Reviewed by Mark Lam. This will allow for a better GCSE implementation. * dfg/DFGCPSRethreadingPhase.cpp: (JSC::DFG::CPSRethreadingPhase::canonicalizeGetLocalFor): * dfg/DFGCSEPhase.cpp: (JSC::DFG::CSEPhase::setReplacement): * dfg/DFGEdgeDominates.h: (JSC::DFG::EdgeDominates::operator()): * dfg/DFGGraph.cpp: (JSC::DFG::Graph::clearReplacements): (JSC::DFG::Graph::initializeNodeOwners): * dfg/DFGGraph.h: (JSC::DFG::Graph::performSubstitutionForEdge): * dfg/DFGLICMPhase.cpp: (JSC::DFG::LICMPhase::attemptHoist): * dfg/DFGNode.h: (JSC::DFG::Node::Node): * dfg/DFGSSAConversionPhase.cpp: (JSC::DFG::SSAConversionPhase::run): 2014-07-04 Filip Pizlo <fpizlo@apple.com> [ftlopt] Infer immutable object properties https://bugs.webkit.org/show_bug.cgi?id=134567 Reviewed by Mark Hahnenberg. This introduces a new way of inferring immutable object properties. A property is said to be immutable if after its creation (i.e. the transition that creates it), we never overwrite it (i.e. replace it) or delete it. Immutability is a property of an "own property" - so if we say that "f" is immutable at "o" then we are implying that "o" has "f" directly and not on a prototype. More specifically, the immutability inference will prove that a property on some structure is immutable. This means that, for example, we may have a structure S1 with property "f" where we claim that "f" at S1 is immutable, but S1 has a transition to S2 that adds a new property "g" and we may claim that "f" at S2 is actually mutable. This is mainly for convenience; it allows us to decouple immutability logic from transition logic. Immutability can be used to constant-fold accesses to objects at DFG-time. The DFG needs to prove the following to constant-fold the access: - The base of the access must be a constant object pointer. We prove that a property at a structure is immutable, but that says nothing of its value; each actual instance of that property may have a different value. So, a constant object pointer is needed to get an actual constant instance of the immutable value. - A check (or watchpoint) must have been emitted proving that the object has a structure that allows loading the property in question. - The replacement watchpoint set of the property in the structure that we've proven the object to have is still valid and we add a watchpoint to it lazily. The replacement watchpoint set is the key new mechanism that this change adds. It's possible that we have proven that the object has one of many structures, in which case each of those structures needs a valid replacement watchpoint set. The replacement watchpoint set is created the first time that any access to the property is cached. A put replace cache will create, and immediately invalidate, the watchpoint set. A get cache will create the watchpoint set and make it start watching. Any non-cached put access will invalidate the watchpoint set if one had been created; the underlying algorithm ensures that checking for the existence of a replacement watchpoint set is very fast in the common case. This algorithm ensures that no cached access needs to ever do any work to invalidate, or check the validity of, any replacement watchpoint sets. It also has some other nice properties: - It's very robust in its definition of immutability. The strictest that it will ever be is that for any instance of the object, the property must be written to only once, specifically at the time that the property is created. But it's looser than this in practice. For example, the property may be written to any number of times before we add the final property that the object will have before anyone reads the property; this works since for optimization purposes we only care if we detect immutability on the structure that the object will have when it is most frequently read from, not any previous structure that the object had. Also, we may write to the property any number of times before anyone caches accesses to it. - It is mostly orthogonal to structure transitions. No new structures need to be created to track the immutability of a property. Hence, there is no risk from this feature causing more polymorphism. This is different from the previous "specificValue" constant inference, which did cause additional structures to be created and sometimes those structures led to fake polymorphism. This feature does leverage existing transitions to do some of the watchpointing: property deletions don't fire the replacement watchpoint set because that would cause a new structure and so the mandatory structure check would fail. Also, this feature is guaranteed to never kick in for uncacheable dictionaries because those wouldn't allow for cacheable accesses - and it takes a cacheable access for this feature to be enabled. - No memory overhead is incurred except when accesses to the property are cached. Dictionary properties will typically have no meta-data for immutability. The number of replacement watchpoint sets we allocate is proportional to the number of inline caches in the program, which is typically must smaller than the number of structures or even the number of objects. This inference is far more powerful than the previous "specificValue" inference, so this change also removes all of that code. It's interesting that the amount of code that is changed to remove that feature is almost as big as the amount of code added to support the new inference - and that's if you include the new tests in the tally. Without new tests, it appears that the new feature actually touches less code! There is one corner case where the previous "specificValue" inference was more powerful. You can imagine someone creating objects with functions as self properties on those objects, such that each object instance had the same function pointers - essentially, someone might be trying to create a vtable but failing at the whole "one vtable for many instances" concept. The "specificValue" inference would do very well for such programs, because a structure check would be sufficient to prove a constant value for all of the function properties. This new inference will fail because it doesn't track the constant values of constant properties; instead it detects the immutability of otherwise variable properties (in the sense that each instance of the property may have a different value). So, the new inference requires having a particular object instance to actually get the constant value. I think it's OK to lose this antifeature. It took a lot of code to support and was a constant source of grief in our transition logic, and there doesn't appear to be any real evidence that programs benefited from that particular kind of inference since usually it's the singleton prototype instance that has all of the functions. This change is a speed-up on everything. date-format-xparb and both SunSpider/raytrace and V8/raytrace seem to be the biggest winners among the macrobenchmarks; they see >5% speed-ups. Many of our microbenchmarks see very large performance improvements, even 80% in one case. * bytecode/ComplexGetStatus.cpp: (JSC::ComplexGetStatus::computeFor): * bytecode/GetByIdStatus.cpp: (JSC::GetByIdStatus::computeFromLLInt): (JSC::GetByIdStatus::computeForStubInfo): (JSC::GetByIdStatus::computeFor): * bytecode/GetByIdVariant.cpp: (JSC::GetByIdVariant::GetByIdVariant): (JSC::GetByIdVariant::operator=): (JSC::GetByIdVariant::attemptToMerge): (JSC::GetByIdVariant::dumpInContext): * bytecode/GetByIdVariant.h: (JSC::GetByIdVariant::alternateBase): (JSC::GetByIdVariant::specificValue): Deleted. * bytecode/PutByIdStatus.cpp: (JSC::PutByIdStatus::computeForStubInfo): (JSC::PutByIdStatus::computeFor): * bytecode/PutByIdVariant.cpp: (JSC::PutByIdVariant::operator=): (JSC::PutByIdVariant::setter): (JSC::PutByIdVariant::dumpInContext): * bytecode/PutByIdVariant.h: (JSC::PutByIdVariant::specificValue): Deleted. * bytecode/Watchpoint.cpp: (JSC::WatchpointSet::fireAllSlow): (JSC::WatchpointSet::fireAll): Deleted. * bytecode/Watchpoint.h: (JSC::WatchpointSet::fireAll): * dfg/DFGAbstractInterpreterInlines.h: (JSC::DFG::AbstractInterpreter<AbstractStateType>::executeEffects): * dfg/DFGByteCodeParser.cpp: (JSC::DFG::ByteCodeParser::handleGetByOffset): (JSC::DFG::ByteCodeParser::handleGetById): (JSC::DFG::ByteCodeParser::handlePutById): (JSC::DFG::ByteCodeParser::parseBlock): * dfg/DFGConstantFoldingPhase.cpp: (JSC::DFG::ConstantFoldingPhase::emitGetByOffset): * dfg/DFGFixupPhase.cpp: (JSC::DFG::FixupPhase::isStringPrototypeMethodSane): (JSC::DFG::FixupPhase::canOptimizeStringObjectAccess): * dfg/DFGGraph.cpp: (JSC::DFG::Graph::tryGetConstantProperty): (JSC::DFG::Graph::visitChildren): * dfg/DFGGraph.h: * dfg/DFGWatchableStructureWatchingPhase.cpp: (JSC::DFG::WatchableStructureWatchingPhase::run): * ftl/FTLLowerDFGToLLVM.cpp: (JSC::FTL::LowerDFGToLLVM::compileMultiGetByOffset): * jit/JITOperations.cpp: * jit/Repatch.cpp: (JSC::repatchByIdSelfAccess): (JSC::generateByIdStub): (JSC::tryCacheGetByID): (JSC::tryCachePutByID): (JSC::tryBuildPutByIdList): * llint/LLIntSlowPaths.cpp: (JSC::LLInt::LLINT_SLOW_PATH_DECL): (JSC::LLInt::putToScopeCommon): * runtime/CommonSlowPaths.h: (JSC::CommonSlowPaths::tryCachePutToScopeGlobal): * runtime/IntendedStructureChain.cpp: (JSC::IntendedStructureChain::mayInterceptStoreTo): * runtime/JSCJSValue.cpp: (JSC::JSValue::putToPrimitive): * runtime/JSGlobalObject.cpp: (JSC::JSGlobalObject::reset): * runtime/JSObject.cpp: (JSC::JSObject::put): (JSC::JSObject::putDirectNonIndexAccessor): (JSC::JSObject::deleteProperty): (JSC::JSObject::defaultValue): (JSC::getCallableObjectSlow): Deleted. (JSC::JSObject::getPropertySpecificValue): Deleted. * runtime/JSObject.h: (JSC::JSObject::getDirect): (JSC::JSObject::getDirectOffset): (JSC::JSObject::inlineGetOwnPropertySlot): (JSC::JSObject::putDirectInternal): (JSC::JSObject::putOwnDataProperty): (JSC::JSObject::putDirect): (JSC::JSObject::putDirectWithoutTransition): (JSC::getCallableObject): Deleted. * runtime/JSScope.cpp: (JSC::abstractAccess): * runtime/PropertyMapHashTable.h: (JSC::PropertyMapEntry::PropertyMapEntry): (JSC::PropertyTable::copy): * runtime/PropertyTable.cpp: (JSC::PropertyTable::clone): (JSC::PropertyTable::PropertyTable): (JSC::PropertyTable::visitChildren): Deleted. * runtime/Structure.cpp: (JSC::Structure::Structure): (JSC::Structure::materializePropertyMap): (JSC::Structure::addPropertyTransitionToExistingStructureImpl): (JSC::Structure::addPropertyTransitionToExistingStructure): (JSC::Structure::addPropertyTransitionToExistingStructureConcurrently): (JSC::Structure::addPropertyTransition): (JSC::Structure::changePrototypeTransition): (JSC::Structure::attributeChangeTransition): (JSC::Structure::toDictionaryTransition): (JSC::Structure::preventExtensionsTransition): (JSC::Structure::takePropertyTableOrCloneIfPinned): (JSC::Structure::nonPropertyTransition): (JSC::Structure::addPropertyWithoutTransition): (JSC::Structure::allocateRareData): (JSC::Structure::ensurePropertyReplacementWatchpointSet): (JSC::Structure::startWatchingPropertyForReplacements): (JSC::Structure::didCachePropertyReplacement): (JSC::Structure::startWatchingInternalProperties): (JSC::Structure::copyPropertyTable): (JSC::Structure::copyPropertyTableForPinning): (JSC::Structure::getConcurrently): (JSC::Structure::get): (JSC::Structure::add): (JSC::Structure::visitChildren): (JSC::Structure::prototypeChainMayInterceptStoreTo): (JSC::Structure::dump): (JSC::Structure::despecifyDictionaryFunction): Deleted. (JSC::Structure::despecifyFunctionTransition): Deleted. (JSC::Structure::despecifyFunction): Deleted. (JSC::Structure::despecifyAllFunctions): Deleted. (JSC::Structure::putSpecificValue): Deleted. * runtime/Structure.h: (JSC::Structure::startWatchingPropertyForReplacements): (JSC::Structure::startWatchingInternalPropertiesIfNecessary): (JSC::Structure::startWatchingInternalPropertiesIfNecessaryForEntireChain): (JSC::Structure::transitionDidInvolveSpecificValue): Deleted. (JSC::Structure::disableSpecificFunctionTracking): Deleted. * runtime/StructureInlines.h: (JSC::Structure::getConcurrently): (JSC::Structure::didReplaceProperty): (JSC::Structure::propertyReplacementWatchpointSet): * runtime/StructureRareData.cpp: (JSC::StructureRareData::destroy): * runtime/StructureRareData.h: * tests/stress/infer-constant-global-property.js: Added. (foo.Math.sin): (foo): * tests/stress/infer-constant-property.js: Added. (foo): * tests/stress/jit-cache-poly-replace-then-cache-get-and-fold-then-invalidate.js: Added. (foo): (bar): * tests/stress/jit-cache-replace-then-cache-get-and-fold-then-invalidate.js: Added. (foo): (bar): * tests/stress/jit-put-to-scope-global-cache-watchpoint-invalidate.js: Added. (foo): (bar): * tests/stress/llint-cache-replace-then-cache-get-and-fold-then-invalidate.js: Added. (foo): (bar): * tests/stress/llint-put-to-scope-global-cache-watchpoint-invalidate.js: Added. (foo): (bar): * tests/stress/repeat-put-to-scope-global-with-same-value-watchpoint-invalidate.js: Added. (foo): (bar): 2014-07-03 Saam Barati <sbarati@apple.com> Add more coverage for the profile_types_with_high_fidelity op code. https://bugs.webkit.org/show_bug.cgi?id=134616 Reviewed by Filip Pizlo. More operations are now being recorded by the profile_types_with_high_fidelity opcode. Specifically: function parameters, function return values, function 'this' value, get_by_id, get_by_value, resolve nodes, function return values at the call site. Added more flags to the profile_types_with_high_fidelity opcode so more focused tasks can take place when the instruction is being linked in CodeBlock. Re-worked the type profiler to search through character offset ranges when asked for the type of an expression at a given offset. Removed redundant calls to Structure::toStructureShape in HighFidelityLog and TypeSet by caching calls based on StructureID. * bytecode/BytecodeList.json: * bytecode/BytecodeUseDef.h: (JSC::computeUsesForBytecodeOffset): (JSC::computeDefsForBytecodeOffset): * bytecode/CodeBlock.cpp: (JSC::CodeBlock::CodeBlock): (JSC::CodeBlock::finalizeUnconditionally): (JSC::CodeBlock::scopeDependentProfile): * bytecode/CodeBlock.h: (JSC::CodeBlock::returnStatementTypeSet): * bytecode/TypeLocation.h: * bytecode/UnlinkedCodeBlock.cpp: (JSC::UnlinkedCodeBlock::highFidelityTypeProfileExpressionInfoForBytecodeOffset): (JSC::UnlinkedCodeBlock::addHighFidelityTypeProfileExpressionInfo): * bytecode/UnlinkedCodeBlock.h: * bytecompiler/BytecodeGenerator.cpp: (JSC::BytecodeGenerator::emitMove): (JSC::BytecodeGenerator::emitProfileTypesWithHighFidelity): (JSC::BytecodeGenerator::emitGetFromScopeWithProfile): (JSC::BytecodeGenerator::emitPutToScope): (JSC::BytecodeGenerator::emitPutToScopeWithProfile): (JSC::BytecodeGenerator::emitPutById): (JSC::BytecodeGenerator::emitPutByVal): * bytecompiler/BytecodeGenerator.h: (JSC::BytecodeGenerator::emitHighFidelityTypeProfilingExpressionInfo): * bytecompiler/NodesCodegen.cpp: (JSC::ResolveNode::emitBytecode): (JSC::BracketAccessorNode::emitBytecode): (JSC::DotAccessorNode::emitBytecode): (JSC::FunctionCallValueNode::emitBytecode): (JSC::FunctionCallResolveNode::emitBytecode): (JSC::FunctionCallBracketNode::emitBytecode): (JSC::FunctionCallDotNode::emitBytecode): (JSC::CallFunctionCallDotNode::emitBytecode): (JSC::ApplyFunctionCallDotNode::emitBytecode): (JSC::PostfixNode::emitResolve): (JSC::PostfixNode::emitBracket): (JSC::PostfixNode::emitDot): (JSC::PrefixNode::emitResolve): (JSC::PrefixNode::emitBracket): (JSC::PrefixNode::emitDot): (JSC::ReadModifyResolveNode::emitBytecode): (JSC::AssignResolveNode::emitBytecode): (JSC::AssignDotNode::emitBytecode): (JSC::ReadModifyDotNode::emitBytecode): (JSC::AssignBracketNode::emitBytecode): (JSC::ReadModifyBracketNode::emitBytecode): (JSC::ReturnNode::emitBytecode): (JSC::FunctionBodyNode::emitBytecode): * inspector/agents/InspectorRuntimeAgent.cpp: (Inspector::InspectorRuntimeAgent::getRuntimeTypeForVariableAtOffset): (Inspector::InspectorRuntimeAgent::getRuntimeTypeForVariableInTextRange): Deleted. * inspector/agents/InspectorRuntimeAgent.h: * inspector/protocol/Runtime.json: * llint/LLIntSlowPaths.cpp: (JSC::LLInt::getFromScopeCommon): (JSC::LLInt::LLINT_SLOW_PATH_DECL): * llint/LLIntSlowPaths.h: * llint/LowLevelInterpreter.asm: * runtime/HighFidelityLog.cpp: (JSC::HighFidelityLog::processHighFidelityLog): (JSC::HighFidelityLog::actuallyProcessLogThreadFunction): (JSC::HighFidelityLog::recordTypeInformationForLocation): Deleted. * runtime/HighFidelityLog.h: (JSC::HighFidelityLog::recordTypeInformationForLocation): * runtime/HighFidelityTypeProfiler.cpp: (JSC::HighFidelityTypeProfiler::getTypesForVariableInAtOffset): (JSC::HighFidelityTypeProfiler::getGlobalTypesForVariableAtOffset): (JSC::HighFidelityTypeProfiler::getLocalTypesForVariableAtOffset): (JSC::HighFidelityTypeProfiler::insertNewLocation): (JSC::HighFidelityTypeProfiler::findLocation): (JSC::HighFidelityTypeProfiler::getTypesForVariableInRange): Deleted. (JSC::HighFidelityTypeProfiler::getGlobalTypesForVariableInRange): Deleted. (JSC::HighFidelityTypeProfiler::getLocalTypesForVariableInRange): Deleted. (JSC::HighFidelityTypeProfiler::getLocationBasedHash): Deleted. * runtime/HighFidelityTypeProfiler.h: (JSC::LocationKey::LocationKey): Deleted. (JSC::LocationKey::hash): Deleted. (JSC::LocationKey::operator==): Deleted. * runtime/Structure.cpp: (JSC::Structure::toStructureShape): * runtime/Structure.h: * runtime/TypeSet.cpp: (JSC::TypeSet::TypeSet): (JSC::TypeSet::addTypeForValue): (JSC::TypeSet::seenTypes): (JSC::TypeSet::removeDuplicatesInStructureHistory): Deleted. * runtime/TypeSet.h: (JSC::StructureShape::setConstructorName): * runtime/VM.cpp: (JSC::VM::getTypesForVariableAtOffset): (JSC::VM::dumpHighFidelityProfilingTypes): (JSC::VM::getTypesForVariableInRange): Deleted. * runtime/VM.h: 2014-07-04 Filip Pizlo <fpizlo@apple.com> [ftlopt][REGRESSION] debug tests fail because PutByIdDirect is now implemented in terms of In https://bugs.webkit.org/show_bug.cgi?id=134642 Rubber stamped by Andreas Kling. * ftl/FTLLowerDFGToLLVM.cpp: (JSC::FTL::LowerDFGToLLVM::compileNode): 2014-07-01 Filip Pizlo <fpizlo@apple.com> [ftlopt] Allocate a new GetterSetter if we change the value of any of its entries other than when they were previously null, so that if we constant-infer an accessor slot then we immediately get the function constant for free https://bugs.webkit.org/show_bug.cgi?id=134518 Reviewed by Mark Hahnenberg. This has no real effect right now, particularly since almost all uses of setSetter/setGetter were already allocating a branch new GetterSetter. But once we start doing more aggressive constant property inference, this change will allow us to remove all runtime checks from getter/setter calls. * runtime/GetterSetter.cpp: (JSC::GetterSetter::withGetter): (JSC::GetterSetter::withSetter): * runtime/GetterSetter.h: (JSC::GetterSetter::setGetter): (JSC::GetterSetter::setSetter): * runtime/JSObject.cpp: (JSC::JSObject::defineOwnNonIndexProperty): 2014-07-02 Filip Pizlo <fpizlo@apple.com> [ftlopt] Rename notifyTransitionFromThisStructure to didTransitionFromThisStructure Rubber stamped by Mark Hahnenberg. * runtime/Structure.cpp: (JSC::Structure::Structure): (JSC::Structure::nonPropertyTransition): (JSC::Structure::didTransitionFromThisStructure): (JSC::Structure::notifyTransitionFromThisStructure): Deleted. * runtime/Structure.h: 2014-07-02 Filip Pizlo <fpizlo@apple.com> [ftlopt] Remove the functionality for cloning StructureRareData since we never do that anymore. Rubber stamped by Mark Hahnenberg. * runtime/Structure.cpp: (JSC::Structure::Structure): (JSC::Structure::cloneRareDataFrom): Deleted. * runtime/Structure.h: * runtime/StructureRareData.cpp: (JSC::StructureRareData::clone): Deleted. (JSC::StructureRareData::StructureRareData): Deleted. * runtime/StructureRareData.h: (JSC::StructureRareData::needsCloning): Deleted. 2014-07-01 Mark Lam <mark.lam@apple.com> [ftlopt] DebuggerCallFrame::scope() should return a DebuggerScope. <https://webkit.org/b/134420> Reviewed by Geoffrey Garen. Previously, DebuggerCallFrame::scope() returns a JSActivation (and relevant peers) which the WebInspector will use to introspect CallFrame variables. Instead, we should be returning a DebuggerScope as an abstraction layer that provides the introspection functionality that the WebInspector needs. This is the first step towards not forcing every frame to have a JSActivation object just because the debugger is enabled. 1. Instantiate the debuggerScopeStructure as a member of the JSGlobalObject instead of the VM. This allows JSObject::globalObject() to be able to return the global object for the DebuggerScope. 2. On the DebuggerScope's life-cycle management: The DebuggerCallFrame is designed to be "valid" only during a debugging session (while the debugger is broken) through the use of a DebuggerCallFrameScope in Debugger::pauseIfNeeded(). Once the debugger resumes from the break, the DebuggerCallFrameScope destructs, and the DebuggerCallFrame will be invalidated. We can't guarantee (from this code alone) that the Inspector code isn't still holding a ref to the DebuggerCallFrame (though they shouldn't), but by contract, the frame will be invalidated, and any attempt to query it will return null values. This is pre-existing behavior. Now, we're adding the DebuggerScope into the picture. While a single debugger pause session is in progress, the Inspector may request the scope from the DebuggerCallFrame. While the DebuggerCallFrame is still valid, we want DebuggerCallFrame::scope() to always return the same DebuggerScope object. This is why we hold on to the DebuggerScope with a strong ref. If we use a weak ref instead, the following cooky behavior can manifest: 1. The Inspector calls Debugger::scope() to get the top scope. 2. The Inspector iterates down the scope chain and is now only holding a reference to a parent scope. It is no longer referencing the top scope. 3. A GC occurs, and the DebuggerCallFrame's weak m_scope ref to the top scope gets cleared. 4. The Inspector calls DebuggerCallFrame::scope() to get the top scope again but gets a different DebuggerScope instance. 5. The Inspector iterates down the scope chain but never sees the parent scope instance that retained a ref to in step 2 above. This is because when iterating this new DebuggerScope instance (which has no knowledge of the previous parent DebuggerScope instance), a new DebuggerScope instance will get created for the same parent scope. Since the DebuggerScope is a JSObject, it's liveness is determined by its reachability. However, it's "validity" is determined by the life-cycle of its owner DebuggerCallFrame. When the owner DebuggerCallFrame gets invalidated, its debugger scope chain (if instantiated) will also get invalidated. This is why we need the DebuggerScope::invalidateChain() method. The Inspector should not be using the DebuggerScope instance after its owner DebuggerCallFrame is invalidated. If it does, those methods will do nothing or returned a failed status. * debugger/Debugger.h: * debugger/DebuggerCallFrame.cpp: (JSC::DebuggerCallFrame::scope): (JSC::DebuggerCallFrame::evaluate): (JSC::DebuggerCallFrame::invalidate): (JSC::DebuggerCallFrame::vm): (JSC::DebuggerCallFrame::lexicalGlobalObject): * debugger/DebuggerCallFrame.h: * debugger/DebuggerScope.cpp: (JSC::DebuggerScope::DebuggerScope): (JSC::DebuggerScope::finishCreation): (JSC::DebuggerScope::visitChildren): (JSC::DebuggerScope::className): (JSC::DebuggerScope::getOwnPropertySlot): (JSC::DebuggerScope::put): (JSC::DebuggerScope::deleteProperty): (JSC::DebuggerScope::getOwnPropertyNames): (JSC::DebuggerScope::defineOwnProperty): (JSC::DebuggerScope::next): (JSC::DebuggerScope::invalidateChain): (JSC::DebuggerScope::isWithScope): (JSC::DebuggerScope::isGlobalScope): (JSC::DebuggerScope::isFunctionScope): * debugger/DebuggerScope.h: (JSC::DebuggerScope::create): (JSC::DebuggerScope::Iterator::Iterator): (JSC::DebuggerScope::Iterator::get): (JSC::DebuggerScope::Iterator::operator++): (JSC::DebuggerScope::Iterator::operator==): (JSC::DebuggerScope::Iterator::operator!=): (JSC::DebuggerScope::isValid): (JSC::DebuggerScope::jsScope): (JSC::DebuggerScope::begin): (JSC::DebuggerScope::end): * inspector/JSJavaScriptCallFrame.cpp: (Inspector::JSJavaScriptCallFrame::scopeType): (Inspector::JSJavaScriptCallFrame::scopeChain): * inspector/JavaScriptCallFrame.h: (Inspector::JavaScriptCallFrame::scopeChain): * inspector/ScriptDebugServer.cpp: * runtime/JSGlobalObject.cpp: (JSC::JSGlobalObject::reset): (JSC::JSGlobalObject::visitChildren): * runtime/JSGlobalObject.h: (JSC::JSGlobalObject::debuggerScopeStructure): * runtime/JSObject.h: (JSC::JSObject::isWithScope): * runtime/JSScope.h: * runtime/VM.cpp: (JSC::VM::VM): * runtime/VM.h: 2014-07-01 Filip Pizlo <fpizlo@apple.com> [ftlopt] DFG bytecode parser should turn PutById with nothing but a Setter stub as stuff+handleCall, and handleCall should be allowed to inline if it wants to https://bugs.webkit.org/show_bug.cgi?id=130756 Reviewed by Oliver Hunt. The enables exposing the call to setters in the DFG, and then inlining it. Previously we already supproted inlined-cached calls to setters from within put_by_id inline caches, and the DFG could certainly emit such IC's. Now, if an IC had a setter call, then the DFG will either emit the GetGetterSetterByOffset/GetSetter/Call combo, or it will do one better and inline the call. A lot of the core functionality was already available from the previous work to inline getters. So, there are some refactorings in this patch that move preexisting functionality around. For example, the work to figure out how the DFG should go about getting to what we call the "loaded value" - i.e. the GetterSetter object reference in the case of accessors - is now shared in ComplexGetStatus, and both GetByIdStatus and PutByIdStatus use it. This means that we can keep the safety checks common. This patch also does additional refactorings in DFG::ByteCodeParser so that we can continue to reuse handleCall() for all of the various kinds of calls we can now emit. 83% speed-up on getter-richards, 2% speed-up on box2d. * CMakeLists.txt: * JavaScriptCore.vcxproj/JavaScriptCore.vcxproj: * JavaScriptCore.xcodeproj/project.pbxproj: * bytecode/ComplexGetStatus.cpp: Added. (JSC::ComplexGetStatus::computeFor): * bytecode/ComplexGetStatus.h: Added. (JSC::ComplexGetStatus::ComplexGetStatus): (JSC::ComplexGetStatus::skip): (JSC::ComplexGetStatus::takesSlowPath): (JSC::ComplexGetStatus::kind): (JSC::ComplexGetStatus::attributes): (JSC::ComplexGetStatus::specificValue): (JSC::ComplexGetStatus::offset): (JSC::ComplexGetStatus::chain): * bytecode/GetByIdStatus.cpp: (JSC::GetByIdStatus::computeForStubInfo): * bytecode/GetByIdVariant.cpp: (JSC::GetByIdVariant::GetByIdVariant): * bytecode/PolymorphicPutByIdList.h: (JSC::PutByIdAccess::PutByIdAccess): (JSC::PutByIdAccess::setter): (JSC::PutByIdAccess::structure): (JSC::PutByIdAccess::chainCount): * bytecode/PutByIdStatus.cpp: (JSC::PutByIdStatus::computeFromLLInt): (JSC::PutByIdStatus::computeFor): (JSC::PutByIdStatus::computeForStubInfo): (JSC::PutByIdStatus::makesCalls): * bytecode/PutByIdStatus.h: (JSC::PutByIdStatus::makesCalls): Deleted. * bytecode/PutByIdVariant.cpp: (JSC::PutByIdVariant::PutByIdVariant): (JSC::PutByIdVariant::operator=): (JSC::PutByIdVariant::replace): (JSC::PutByIdVariant::transition): (JSC::PutByIdVariant::setter): (JSC::PutByIdVariant::writesStructures): (JSC::PutByIdVariant::reallocatesStorage): (JSC::PutByIdVariant::makesCalls): (JSC::PutByIdVariant::dumpInContext): * bytecode/PutByIdVariant.h: (JSC::PutByIdVariant::PutByIdVariant): (JSC::PutByIdVariant::structure): (JSC::PutByIdVariant::oldStructure): (JSC::PutByIdVariant::alternateBase): (JSC::PutByIdVariant::specificValue): (JSC::PutByIdVariant::callLinkStatus): (JSC::PutByIdVariant::replace): Deleted. (JSC::PutByIdVariant::transition): Deleted. * dfg/DFGByteCodeParser.cpp: (JSC::DFG::ByteCodeParser::addCallWithoutSettingResult): (JSC::DFG::ByteCodeParser::addCall): (JSC::DFG::ByteCodeParser::handleCall): (JSC::DFG::ByteCodeParser::handleInlining): (JSC::DFG::ByteCodeParser::handleGetById): (JSC::DFG::ByteCodeParser::handlePutById): (JSC::DFG::ByteCodeParser::parseBlock): * jit/Repatch.cpp: (JSC::tryCachePutByID): (JSC::tryBuildPutByIdList): * runtime/IntendedStructureChain.cpp: (JSC::IntendedStructureChain::takesSlowPathInDFGForImpureProperty): * runtime/IntendedStructureChain.h: * tests/stress/exit-from-setter.js: Added. * tests/stress/poly-chain-setter.js: Added. (Cons): (foo): (test): * tests/stress/poly-chain-then-setter.js: Added. (Cons1): (Cons2): (foo): (test): * tests/stress/poly-setter-combo.js: Added. (Cons1): (Cons2): (foo): (test): (.test): * tests/stress/poly-setter-then-self.js: Added. (foo): (test): (.test): * tests/stress/weird-setter-counter.js: Added. (foo): (test): * tests/stress/weird-setter-counter-syntactic.js: Added. (foo): (test): 2014-07-01 Matthew Mirman <mmirman@apple.com> Added an implementation of the "in" check to FTL. https://bugs.webkit.org/show_bug.cgi?id=134508 Reviewed by Filip Pizlo. * ftl/FTLCapabilities.cpp: enabled compilation for "in" (JSC::FTL::canCompile): ditto * ftl/FTLCompile.cpp: (JSC::FTL::generateCheckInICFastPath): added. (JSC::FTL::fixFunctionBasedOnStackMaps): added case for CheckIn descriptors. * ftl/FTLInlineCacheDescriptor.h: (JSC::FTL::CheckInGenerator::CheckInGenerator): added. (JSC::FTL::CheckInDescriptor::CheckInDescriptor): added. * ftl/FTLInlineCacheSize.cpp: (JSC::FTL::sizeOfCheckIn): added. Currently larger than necessary. * ftl/FTLInlineCacheSize.h: ditto * ftl/FTLIntrinsicRepository.h: Added function type for operationInGeneric * ftl/FTLLowerDFGToLLVM.cpp: (JSC::FTL::LowerDFGToLLVM::compileNode): added case for In. (JSC::FTL::LowerDFGToLLVM::compileIn): added. * ftl/FTLSlowPathCall.cpp: Added a callOperation for operationIn (JSC::FTL::callOperation): ditto * ftl/FTLSlowPathCall.h: ditto * ftl/FTLState.h: Added a vector to hold CheckIn descriptors. * jit/JITOperations.h: made operationIns internal. * tests/stress/ftl-checkin.js: Added. * tests/stress/ftl-checkin-variable.js: Added. 2014-06-30 Mark Hahnenberg <mhahnenberg@apple.com> CodeBlock::stronglyVisitWeakReferences should mark DFG::CommonData::weakStructureReferences https://bugs.webkit.org/show_bug.cgi?id=134455 Reviewed by Geoffrey Garen. Otherwise we get hanging pointers which can cause us to die later. * bytecode/CodeBlock.cpp: (JSC::CodeBlock::stronglyVisitWeakReferences): 2014-06-27 Filip Pizlo <fpizlo@apple.com> [ftlopt] Reduce the GC's influence on optimization decisions https://bugs.webkit.org/show_bug.cgi?id=134427 Reviewed by Oliver Hunt. This is a slight speed-up on some platforms, that arises from a bunch of fixes that I made while trying to make the GC keep more structures alive (https://bugs.webkit.org/show_bug.cgi?id=128072). The fixes are, roughly: - If the GC clears an inline cache, then this no longer causes the IC to be forever polymorphic. - If we exit in inlined code into a function that tries to OSR enter, then we jettison sooner. - Some variables being uninitialized led to rage-recompilations. This is a pretty strong step in the direction of keeping more Structures alive and not blowing away code just because a Structure died. But, it seems like there is still a slight speed-up to be had from blowing away code that references dead Structures. * bytecode/CodeBlock.cpp: (JSC::CodeBlock::dumpAssumingJITType): (JSC::shouldMarkTransition): (JSC::CodeBlock::propagateTransitions): (JSC::CodeBlock::determineLiveness): * bytecode/GetByIdStatus.cpp: (JSC::GetByIdStatus::computeForStubInfo): * bytecode/PutByIdStatus.cpp: (JSC::PutByIdStatus::computeForStubInfo): * dfg/DFGCapabilities.cpp: (JSC::DFG::isSupportedForInlining): (JSC::DFG::mightInlineFunctionForCall): (JSC::DFG::mightInlineFunctionForClosureCall): (JSC::DFG::mightInlineFunctionForConstruct): * dfg/DFGCapabilities.h: * dfg/DFGCommonData.h: * dfg/DFGDesiredWeakReferences.cpp: (JSC::DFG::DesiredWeakReferences::reallyAdd): * dfg/DFGOSREntry.cpp: (JSC::DFG::prepareOSREntry): * dfg/DFGOSRExitCompilerCommon.cpp: (JSC::DFG::handleExitCounts): * dfg/DFGOperations.cpp: * dfg/DFGOperations.h: * ftl/FTLForOSREntryJITCode.cpp: (JSC::FTL::ForOSREntryJITCode::ForOSREntryJITCode): These variables being uninitialized is benign in terms of correctness but can sometimes cause rage-recompilations. For some reason it took this patch to reveal this. * ftl/FTLOSREntry.cpp: (JSC::FTL::prepareOSREntry): * runtime/Executable.cpp: (JSC::ExecutableBase::destroy): (JSC::NativeExecutable::destroy): (JSC::ScriptExecutable::ScriptExecutable): (JSC::ScriptExecutable::destroy): (JSC::ScriptExecutable::installCode): (JSC::EvalExecutable::EvalExecutable): (JSC::ProgramExecutable::ProgramExecutable): * runtime/Executable.h: (JSC::ScriptExecutable::setDidTryToEnterInLoop): (JSC::ScriptExecutable::didTryToEnterInLoop): (JSC::ScriptExecutable::addressOfDidTryToEnterInLoop): (JSC::ScriptExecutable::ScriptExecutable): Deleted. * runtime/StructureInlines.h: (JSC::Structure::storedPrototypeObject): (JSC::Structure::storedPrototypeStructure): 2014-06-25 Filip Pizlo <fpizlo@apple.com> [ftlopt] If a CodeBlock is jettisoned due to a watchpoint then it should be possible to figure out something about that watchpoint https://bugs.webkit.org/show_bug.cgi?id=134333 Reviewed by Geoffrey Garen. This is engineered to provide loads of information to the profiler without incurring any costs when the profiler is disabled. It's the oldest trick in the book: the thing that fires the watchpoint doesn't actually create anything to describe the reason why it was fired; instead it creates a stack-allocated FireDetail subclass instance. Only if the FireDetail::dump() virtual method is called does anything happen. Currently we use this to produce very fine-grained data for Structure watchpoints and some cases of variable watchpoints. For all other situations, the given reason is just a string constant, by using StringFireDetail. If we find a situation where that string constant is insufficient to diagnose an issue then we can change it to provide more fine-grained information. * JavaScriptCore.xcodeproj/project.pbxproj: * bytecode/CodeBlock.cpp: (JSC::CodeBlock::CodeBlock): (JSC::CodeBlock::jettison): * bytecode/CodeBlock.h: * bytecode/CodeBlockJettisoningWatchpoint.cpp: (JSC::CodeBlockJettisoningWatchpoint::fireInternal): * bytecode/CodeBlockJettisoningWatchpoint.h: * bytecode/ProfiledCodeBlockJettisoningWatchpoint.cpp: Removed. * bytecode/ProfiledCodeBlockJettisoningWatchpoint.h: Removed. * bytecode/StructureStubClearingWatchpoint.cpp: (JSC::StructureStubClearingWatchpoint::fireInternal): * bytecode/StructureStubClearingWatchpoint.h: * bytecode/VariableWatchpointSet.h: (JSC::VariableWatchpointSet::invalidate): (JSC::VariableWatchpointSet::finalizeUnconditionally): * bytecode/VariableWatchpointSetInlines.h: (JSC::VariableWatchpointSet::notifyWrite): * bytecode/Watchpoint.cpp: (JSC::StringFireDetail::dump): (JSC::WatchpointSet::fireAll): (JSC::WatchpointSet::fireAllSlow): (JSC::WatchpointSet::fireAllWatchpoints): (JSC::InlineWatchpointSet::fireAll): * bytecode/Watchpoint.h: (JSC::FireDetail::FireDetail): (JSC::FireDetail::~FireDetail): (JSC::StringFireDetail::StringFireDetail): (JSC::Watchpoint::fire): (JSC::WatchpointSet::fireAll): (JSC::WatchpointSet::touch): (JSC::WatchpointSet::invalidate): (JSC::InlineWatchpointSet::fireAll): (JSC::InlineWatchpointSet::touch): * dfg/DFGCommonData.h: * dfg/DFGOperations.cpp: * interpreter/Interpreter.cpp: (JSC::Interpreter::execute): * jsc.cpp: (WTF::Masquerader::create): * profiler/ProfilerCompilation.cpp: (JSC::Profiler::Compilation::setJettisonReason): (JSC::Profiler::Compilation::toJS): * profiler/ProfilerCompilation.h: (JSC::Profiler::Compilation::setJettisonReason): Deleted. * runtime/ArrayBuffer.cpp: (JSC::ArrayBuffer::transfer): * runtime/ArrayBufferNeuteringWatchpoint.cpp: (JSC::ArrayBufferNeuteringWatchpoint::fireAll): * runtime/ArrayBufferNeuteringWatchpoint.h: * runtime/CommonIdentifiers.h: * runtime/CommonSlowPaths.cpp: (JSC::SLOW_PATH_DECL): * runtime/Identifier.cpp: (JSC::Identifier::dump): * runtime/Identifier.h: * runtime/JSFunction.cpp: (JSC::JSFunction::put): (JSC::JSFunction::defineOwnProperty): * runtime/JSGlobalObject.cpp: (JSC::JSGlobalObject::addFunction): (JSC::JSGlobalObject::haveABadTime): * runtime/JSSymbolTableObject.cpp: (JSC::VariableWriteFireDetail::dump): * runtime/JSSymbolTableObject.h: (JSC::VariableWriteFireDetail::VariableWriteFireDetail): (JSC::symbolTablePut): (JSC::symbolTablePutWithAttributes): * runtime/PropertyName.h: (JSC::PropertyName::dump): * runtime/Structure.cpp: (JSC::Structure::notifyTransitionFromThisStructure): * runtime/Structure.h: (JSC::Structure::notifyTransitionFromThisStructure): Deleted. * runtime/SymbolTable.cpp: (JSC::SymbolTableEntry::notifyWriteSlow): (JSC::SymbolTable::WatchpointCleanup::finalizeUnconditionally): * runtime/SymbolTable.h: (JSC::SymbolTableEntry::notifyWrite): * runtime/VM.cpp: (JSC::VM::addImpureProperty): Source/WebCore: 2014-07-01 Mark Lam <mark.lam@apple.com> [ftlopt] DebuggerCallFrame::scope() should return a DebuggerScope. <https://webkit.org/b/134420> Reviewed by Geoffrey Garen. No new tests. * ForwardingHeaders/debugger/DebuggerCallFrame.h: Removed. - This is not in use. Hence, we can remove it. * bindings/js/ScriptController.cpp: (WebCore::ScriptController::attachDebugger): - We should acquire the JSLock before modifying a JS global object. 2014-06-25 Filip Pizlo <fpizlo@apple.com> [ftlopt] If a CodeBlock is jettisoned due to a watchpoint then it should be possible to figure out something about that watchpoint https://bugs.webkit.org/show_bug.cgi?id=134333 Reviewed by Geoffrey Garen. No new tests because no change in behavior. * bindings/scripts/CodeGeneratorJS.pm: (GenerateHeader): Tools: 2014-06-25 Filip Pizlo <fpizlo@apple.com> [ftlopt] If a CodeBlock is jettisoned due to a watchpoint then it should be possible to figure out something about that watchpoint https://bugs.webkit.org/show_bug.cgi?id=134333 Reviewed by Geoffrey Garen. * Scripts/display-profiler-output: LayoutTests: 2014-07-16 Mark Hahnenberg <mhahnenberg@apple.com> sputnik/Implementation_Diagnostics/S12.6.4_D1.html depends on undefined behavior https://bugs.webkit.org/show_bug.cgi?id=135007 Reviewed by Filip Pizlo. EcmaScript 5.1 specifies that during for-in enumeration newly added properties may or may not be visited during the current enumeration. Specifically, in section 12.6.4 the spec states: "If new properties are added to the object being enumerated during enumeration, the newly added properties are not guaranteed to be visited in the active enumeration." The sputnik/Implementation_Diagnostics/S12.6.4_D1.html layout test is from before sputnik was added to the test262 suite. I believe it has since been removed, so it would probably be okay to remove it from our layout test suite. * sputnik/Implementation_Diagnostics/S12.6.4_D1-expected.txt: Removed. * sputnik/Implementation_Diagnostics/S12.6.4_D1.html: Removed. 2014-07-13 Filip Pizlo <fpizlo@apple.com> [ftlopt] DFG should be able to do GCSE in SSA and this should be unified with the CSE in CPS, and both of these things should use abstract heaps for reasoning about effects https://bugs.webkit.org/show_bug.cgi?id=134677 Reviewed by Sam Weinig. * js/regress/gcse-expected.txt: Added. * js/regress/gcse-poly-get-expected.txt: Added. * js/regress/gcse-poly-get-less-obvious-expected.txt: Added. * js/regress/gcse-poly-get-less-obvious.html: Added. * js/regress/gcse-poly-get.html: Added. * js/regress/gcse.html: Added. * js/regress/script-tests/gcse-poly-get-less-obvious.js: Added. * js/regress/script-tests/gcse-poly-get.js: Added. * js/regress/script-tests/gcse.js: Added. 2014-07-04 Filip Pizlo <fpizlo@apple.com> [ftlopt] Infer immutable object properties https://bugs.webkit.org/show_bug.cgi?id=134567 Reviewed by Mark Hahnenberg. * js/regress/infer-constant-global-property-expected.txt: Added. * js/regress/infer-constant-global-property.html: Added. * js/regress/infer-constant-property-expected.txt: Added. * js/regress/infer-constant-property.html: Added. * js/regress/script-tests/infer-constant-global-property.js: Added. * js/regress/script-tests/infer-constant-property.js: Added. Canonical link: https://commits.webkit.org/153499@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@172129 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2014-08-06 05:27:46 +00:00
{
[JSC] Use DataIC for AccessCase https://bugs.webkit.org/show_bug.cgi?id=226072 Reviewed by Saam Barati and Filip Pizlo. Source/JavaScriptCore: This patch adds non-repatching IC for Baseline JIT in ARM64. This does not work in non-ARM64 architectures (including X64) due to the use of link-register. 1. We add non-repatching IC, which is enabled only in Baseline due to performance reason. We are using the existing IC in DFG and FTL. Non-repatching includes fast-path, and slow-path's operation function. 2. We still keep InlineAccess in all tiers. Removing that causes 0.3 ~ 1.0% regression in Speedometer2. This means that we still need some repatching when we first introduce stubs. 3. We add a mechanism to share generated code stubs in non-repatching IC. Currently, getter / setter / custom accessors are excluded since their code relies on JSGlobalObject, CodeBlock etc. which are not included in AccessCase's data structure. 4. This patch still relies on that CodeBlock will be destroyed synchronously since we need to ensure that sharing-hash-table does not include already-dead JIT code stubs. We can fix it (e.g. annotating epoch to these stubs, bump them in finalizeUnconditionally), but we avoid doing that to prevent from further enlarging this patch. This patch is already significant in its size. 5. Added callOperation(Address) support in CCallHelpers, which can save the target in nonArgGPR0, and call it so that we can use Address including GPR which is also used for arguments. Performance is neutral in JetStream2 and Speedometer2. But it offers the way to remove some code generation in Baseline. * assembler/MacroAssemblerARM64E.h: (JSC::MacroAssemblerARM64E::call): * bytecode/AccessCase.cpp: (JSC::AccessCase::create): (JSC::AccessCase::createTransition): (JSC::AccessCase::createDelete): (JSC::AccessCase::createCheckPrivateBrand): (JSC::AccessCase::createSetPrivateBrand): (JSC::AccessCase::fromStructureStubInfo): (JSC::AccessCase::clone const): (JSC::AccessCase::generateWithGuard): (JSC::AccessCase::generateImpl): (JSC::AccessCase::canBeShared): * bytecode/AccessCase.h: (JSC::AccessCase::hash const): (JSC::AccessCase::AccessCase): (JSC::SharedJITStubSet::Hash::Key::Key): (JSC::SharedJITStubSet::Hash::Key::isHashTableDeletedValue const): (JSC::SharedJITStubSet::Hash::Key::operator==): (JSC::SharedJITStubSet::Hash::hash): (JSC::SharedJITStubSet::Hash::equal): (JSC::SharedJITStubSet::Searcher::Translator::hash): (JSC::SharedJITStubSet::Searcher::Translator::equal): (JSC::SharedJITStubSet::PointerTranslator::hash): (JSC::SharedJITStubSet::PointerTranslator::equal): (JSC::SharedJITStubSet::add): (JSC::SharedJITStubSet::remove): (JSC::SharedJITStubSet::find): * bytecode/ByValInfo.h: (JSC::ByValInfo::setUp): (JSC::ByValInfo::offsetOfSlowOperation): (JSC::ByValInfo::offsetOfNotIndexJumpTarget): (JSC::ByValInfo::offsetOfBadTypeJumpTarget): * bytecode/CallLinkInfo.cpp: (JSC::CallLinkInfo::initializeDirectCall): (JSC::CallLinkInfo::setDirectCallTarget): * bytecode/CodeBlock.h: (JSC::CodeBlock::useDataIC const): * bytecode/GetterSetterAccessCase.cpp: (JSC::GetterSetterAccessCase::create): (JSC::GetterSetterAccessCase::clone const): (JSC::GetterSetterAccessCase::emitDOMJITGetter): * bytecode/GetterSetterAccessCase.h: * bytecode/InlineAccess.cpp: (JSC::getScratchRegister): (JSC::InlineAccess::rewireStubAsJumpInAccessNotUsingInlineAccess): (JSC::InlineAccess::rewireStubAsJumpInAccess): (JSC::InlineAccess::resetStubAsJumpInAccess): (JSC::InlineAccess::resetStubAsJumpInAccessNotUsingInlineAccess): (JSC::InlineAccess::rewireStubAsJump): Deleted. * bytecode/InlineAccess.h: * bytecode/InstanceOfAccessCase.cpp: (JSC::InstanceOfAccessCase::create): (JSC::InstanceOfAccessCase::clone const): * bytecode/InstanceOfAccessCase.h: * bytecode/IntrinsicGetterAccessCase.cpp: (JSC::IntrinsicGetterAccessCase::create): (JSC::IntrinsicGetterAccessCase::clone const): * bytecode/IntrinsicGetterAccessCase.h: * bytecode/ModuleNamespaceAccessCase.cpp: (JSC::ModuleNamespaceAccessCase::create): (JSC::ModuleNamespaceAccessCase::clone const): * bytecode/ModuleNamespaceAccessCase.h: * bytecode/ObjectPropertyConditionSet.h: (JSC::ObjectPropertyConditionSet::hash const): (JSC::ObjectPropertyConditionSet::operator==): (JSC::ObjectPropertyConditionSet::operator!=): * bytecode/PolymorphicAccess.cpp: (JSC::AccessGenerationState::installWatchpoint): (JSC::AccessGenerationState::succeed): (JSC::AccessGenerationState::preserveLiveRegistersToStackForCallWithoutExceptions): (JSC::PolymorphicAccess::addCases): (JSC::PolymorphicAccess::addCase): (JSC::PolymorphicAccess::visitWeak const): (JSC::PolymorphicAccess::regenerate): * bytecode/PolymorphicAccess.h: (JSC::AccessGenerationState::AccessGenerationState): (JSC::AccessGenerationState::preserveLiveRegistersToStackForCallWithoutExceptions): * bytecode/ProxyableAccessCase.cpp: (JSC::ProxyableAccessCase::create): (JSC::ProxyableAccessCase::clone const): * bytecode/ProxyableAccessCase.h: * bytecode/StructureStubInfo.cpp: (JSC::StructureStubInfo::StructureStubInfo): (JSC::StructureStubInfo::addAccessCase): (JSC::StructureStubInfo::reset): * bytecode/StructureStubInfo.h: (JSC::StructureStubInfo::offsetOfCodePtr): (JSC::StructureStubInfo::offsetOfSlowPathStartLocation): (JSC::StructureStubInfo::offsetOfSlowOperation): (JSC::StructureStubInfo::patchableJump): Deleted. * dfg/DFGJITCompiler.h: (JSC::DFG::JITCompiler::appendCall): * dfg/DFGSlowPathGenerator.h: (JSC::DFG::slowPathICCall): * dfg/DFGSpeculativeJIT.cpp: (JSC::DFG::SpeculativeJIT::compileGetById): (JSC::DFG::SpeculativeJIT::compileGetByIdFlush): (JSC::DFG::SpeculativeJIT::compileDeleteById): (JSC::DFG::SpeculativeJIT::compileDeleteByVal): (JSC::DFG::SpeculativeJIT::compileInById): (JSC::DFG::SpeculativeJIT::compileInByVal): (JSC::DFG::SpeculativeJIT::compileGetPrivateNameByVal): (JSC::DFG::SpeculativeJIT::compileGetPrivateNameById): (JSC::DFG::SpeculativeJIT::compilePutPrivateNameById): (JSC::DFG::SpeculativeJIT::compileCheckPrivateBrand): (JSC::DFG::SpeculativeJIT::compileSetPrivateBrand): (JSC::DFG::SpeculativeJIT::compileInstanceOfForCells): (JSC::DFG::SpeculativeJIT::compileInstanceOf): (JSC::DFG::SpeculativeJIT::compilePutByIdFlush): (JSC::DFG::SpeculativeJIT::compilePutById): (JSC::DFG::SpeculativeJIT::compilePutByIdDirect): (JSC::DFG::SpeculativeJIT::cachedPutById): * dfg/DFGSpeculativeJIT.h: (JSC::DFG::SpeculativeJIT::callOperation): (JSC::DFG::SpeculativeJIT::appendCall): (JSC::DFG::SpeculativeJIT::appendCallSetResult): * dfg/DFGSpeculativeJIT32_64.cpp: (JSC::DFG::SpeculativeJIT::cachedGetById): (JSC::DFG::SpeculativeJIT::cachedGetByIdWithThis): (JSC::DFG::SpeculativeJIT::compile): * dfg/DFGSpeculativeJIT64.cpp: (JSC::DFG::SpeculativeJIT::cachedGetById): (JSC::DFG::SpeculativeJIT::cachedGetByIdWithThis): (JSC::DFG::SpeculativeJIT::compile): * ftl/FTLLowerDFGToB3.cpp: (JSC::FTL::DFG::LowerDFGToB3::getPrivateName): (JSC::FTL::DFG::LowerDFGToB3::compilePrivateBrandAccess): (JSC::FTL::DFG::LowerDFGToB3::cachedPutById): (JSC::FTL::DFG::LowerDFGToB3::compileGetByVal): (JSC::FTL::DFG::LowerDFGToB3::compileDelBy): (JSC::FTL::DFG::LowerDFGToB3::compileCompareStrictEq): * ftl/FTLSlowPathCall.cpp: (JSC::FTL::SlowPathCallContext::SlowPathCallContext): (JSC::FTL::SlowPathCallContext::keyWithTarget const): (JSC::FTL::SlowPathCallContext::makeCall): * ftl/FTLSlowPathCall.h: (JSC::FTL::callOperation): * ftl/FTLSlowPathCallKey.cpp: (JSC::FTL::SlowPathCallKey::dump const): * ftl/FTLSlowPathCallKey.h: (JSC::FTL::SlowPathCallKey::SlowPathCallKey): (JSC::FTL::SlowPathCallKey::indirectOffset const): (JSC::FTL::SlowPathCallKey::withCallTarget): (JSC::FTL::SlowPathCallKey::operator== const): (JSC::FTL::SlowPathCallKey::hash const): * ftl/FTLThunks.cpp: (JSC::FTL::genericGenerationThunkGenerator): (JSC::FTL::slowPathCallThunkGenerator): * jit/AssemblyHelpers.cpp: (JSC::AssemblyHelpers::emitVirtualCall): * jit/CCallHelpers.cpp: (JSC::CCallHelpers::emitJITCodeOver): * jit/CCallHelpers.h: (JSC::CCallHelpers::ArgCollection::ArgCollection): (JSC::CCallHelpers::ArgCollection::pushRegArg): (JSC::CCallHelpers::ArgCollection::pushExtraRegArg): (JSC::CCallHelpers::ArgCollection::pushNonArg): (JSC::CCallHelpers::ArgCollection::addGPRArg): (JSC::CCallHelpers::ArgCollection::addGPRExtraArg): (JSC::CCallHelpers::ArgCollection::addStackArg): (JSC::CCallHelpers::ArgCollection::addPoke): (JSC::CCallHelpers::calculatePokeOffset): (JSC::CCallHelpers::pokeForArgument): (JSC::CCallHelpers::stackAligned): (JSC::CCallHelpers::marshallArgumentRegister): (JSC::CCallHelpers::setupArgumentsImpl): (JSC::CCallHelpers::pokeArgumentsAligned): (JSC::CCallHelpers::std::is_integral<CURRENT_ARGUMENT_TYPE>::value): (JSC::CCallHelpers::std::is_pointer<CURRENT_ARGUMENT_TYPE>::value): (JSC::CCallHelpers::setupArgumentsEntryImpl): (JSC::CCallHelpers::setupArguments): (JSC::CCallHelpers::setupArgumentsForIndirectCall): * jit/GCAwareJITStubRoutine.cpp: (JSC::GCAwareJITStubRoutine::GCAwareJITStubRoutine): (JSC::GCAwareJITStubRoutine::makeGCAware): (JSC::GCAwareJITStubRoutine::observeZeroRefCount): (JSC::PolymorphicAccessJITStubRoutine::PolymorphicAccessJITStubRoutine): (JSC::PolymorphicAccessJITStubRoutine::observeZeroRefCount): (JSC::PolymorphicAccessJITStubRoutine::computeHash): (JSC::MarkingGCAwareJITStubRoutine::MarkingGCAwareJITStubRoutine): (JSC::GCAwareJITStubRoutineWithExceptionHandler::GCAwareJITStubRoutineWithExceptionHandler): (JSC::createICJITStubRoutine): (JSC::createJITStubRoutine): Deleted. * jit/GCAwareJITStubRoutine.h: (JSC::GCAwareJITStubRoutine::create): (JSC::PolymorphicAccessJITStubRoutine::cases const): (JSC::PolymorphicAccessJITStubRoutine::weakStructures const): (JSC::PolymorphicAccessJITStubRoutine::hash const): * jit/GPRInfo.h: * jit/JIT.cpp: (JSC::JIT::link): * jit/JIT.h: * jit/JITCall.cpp: (JSC::JIT::emit_op_iterator_open): (JSC::JIT::emitSlow_op_iterator_open): (JSC::JIT::emit_op_iterator_next): (JSC::JIT::emitSlow_op_iterator_next): * jit/JITCall32_64.cpp: (JSC::JIT::emit_op_iterator_open): (JSC::JIT::emit_op_iterator_next): * jit/JITCode.h: (JSC::JITCode::useDataIC): * jit/JITInlineCacheGenerator.cpp: (JSC::JITInlineCacheGenerator::JITInlineCacheGenerator): (JSC::JITInlineCacheGenerator::finalize): (JSC::JITByIdGenerator::JITByIdGenerator): (JSC::JITByIdGenerator::finalize): (JSC::JITByIdGenerator::generateFastCommon): (JSC::JITGetByIdGenerator::JITGetByIdGenerator): (JSC::JITGetByIdWithThisGenerator::JITGetByIdWithThisGenerator): (JSC::JITPutByIdGenerator::JITPutByIdGenerator): (JSC::JITDelByValGenerator::JITDelByValGenerator): (JSC::JITDelByValGenerator::generateFastPath): (JSC::JITDelByValGenerator::finalize): (JSC::JITDelByIdGenerator::JITDelByIdGenerator): (JSC::JITDelByIdGenerator::generateFastPath): (JSC::JITDelByIdGenerator::finalize): (JSC::JITInByValGenerator::JITInByValGenerator): (JSC::JITInByValGenerator::generateFastPath): (JSC::JITInByValGenerator::finalize): (JSC::JITInByIdGenerator::JITInByIdGenerator): (JSC::JITInstanceOfGenerator::JITInstanceOfGenerator): (JSC::JITInstanceOfGenerator::generateFastPath): (JSC::JITInstanceOfGenerator::finalize): (JSC::JITGetByValGenerator::JITGetByValGenerator): (JSC::JITGetByValGenerator::generateFastPath): (JSC::JITGetByValGenerator::finalize): (JSC::JITPrivateBrandAccessGenerator::JITPrivateBrandAccessGenerator): (JSC::JITPrivateBrandAccessGenerator::generateFastPath): (JSC::JITPrivateBrandAccessGenerator::finalize): * jit/JITInlineCacheGenerator.h: (JSC::JITGetByIdGenerator::JITGetByIdGenerator): Deleted. (JSC::JITGetByIdWithThisGenerator::JITGetByIdWithThisGenerator): Deleted. (JSC::JITPutByIdGenerator::JITPutByIdGenerator): Deleted. (JSC::JITDelByValGenerator::JITDelByValGenerator): Deleted. (JSC::JITDelByValGenerator::slowPathJump const): Deleted. (JSC::JITDelByIdGenerator::JITDelByIdGenerator): Deleted. (JSC::JITDelByIdGenerator::slowPathJump const): Deleted. (JSC::JITInByIdGenerator::JITInByIdGenerator): Deleted. (JSC::JITInstanceOfGenerator::JITInstanceOfGenerator): Deleted. (JSC::JITGetByValGenerator::JITGetByValGenerator): Deleted. (JSC::JITGetByValGenerator::slowPathJump const): Deleted. (JSC::JITPrivateBrandAccessGenerator::JITPrivateBrandAccessGenerator): Deleted. (JSC::JITPrivateBrandAccessGenerator::slowPathJump const): Deleted. * jit/JITInlines.h: (JSC::JIT::emitLoadForArrayMode): (JSC::JIT::appendCallWithExceptionCheck): (JSC::JIT::appendCallWithExceptionCheckSetJSValueResult): (JSC::JIT::appendCallWithExceptionCheckSetJSValueResultWithProfile): * jit/JITOpcodes.cpp: (JSC::JIT::emit_op_instanceof): (JSC::JIT::emitSlow_op_instanceof): (JSC::JIT::privateCompileHasIndexedProperty): (JSC::JIT::emit_op_has_enumerable_indexed_property): (JSC::JIT::emitSlow_op_has_enumerable_indexed_property): * jit/JITOpcodes32_64.cpp: (JSC::JIT::emit_op_instanceof): (JSC::JIT::privateCompileHasIndexedProperty): (JSC::JIT::emit_op_has_enumerable_indexed_property): * jit/JITOperations.cpp: (JSC::JSC_DEFINE_JIT_OPERATION): * jit/JITPropertyAccess.cpp: (JSC::JIT::emit_op_get_by_val): (JSC::JIT::emitSlow_op_get_by_val): (JSC::JIT::slow_op_get_by_val_prepareCallGenerator): (JSC::JIT::emit_op_get_private_name): (JSC::JIT::emitSlow_op_get_private_name): (JSC::JIT::slow_op_get_private_name_prepareCallGenerator): (JSC::JIT::emit_op_set_private_brand): (JSC::JIT::emitSlow_op_set_private_brand): (JSC::JIT::emit_op_check_private_brand): (JSC::JIT::emitSlow_op_check_private_brand): (JSC::JIT::emit_op_put_by_val): (JSC::JIT::emitGenericContiguousPutByVal): (JSC::JIT::emitArrayStoragePutByVal): (JSC::JIT::emitPutByValWithCachedId): (JSC::JIT::emitSlow_op_put_by_val): (JSC::JIT::slow_op_put_by_val_prepareCallGenerator): (JSC::JIT::emit_op_put_private_name): (JSC::JIT::emitSlow_op_put_private_name): (JSC::JIT::slow_op_put_private_name_prepareCallGenerator): (JSC::JIT::emit_op_del_by_id): (JSC::JIT::emitSlow_op_del_by_id): (JSC::JIT::slow_op_del_by_id_prepareCallGenerator): (JSC::JIT::emit_op_del_by_val): (JSC::JIT::emitSlow_op_del_by_val): (JSC::JIT::slow_op_del_by_val_prepareCallGenerator): (JSC::JIT::emit_op_try_get_by_id): (JSC::JIT::emitSlow_op_try_get_by_id): (JSC::JIT::emit_op_get_by_id_direct): (JSC::JIT::emitSlow_op_get_by_id_direct): (JSC::JIT::emit_op_get_by_id): (JSC::JIT::emit_op_get_by_id_with_this): (JSC::JIT::emitSlow_op_get_by_id): (JSC::JIT::slow_op_get_by_id_prepareCallGenerator): (JSC::JIT::emitSlow_op_get_by_id_with_this): (JSC::JIT::slow_op_get_by_id_with_this_prepareCallGenerator): (JSC::JIT::emit_op_put_by_id): (JSC::JIT::emitSlow_op_put_by_id): (JSC::JIT::slow_op_put_by_id_prepareCallGenerator): (JSC::JIT::emit_op_in_by_id): (JSC::JIT::emitSlow_op_in_by_id): (JSC::JIT::emit_op_in_by_val): (JSC::JIT::emitSlow_op_in_by_val): (JSC::JIT::privateCompilePutByVal): (JSC::JIT::privateCompilePutPrivateNameWithCachedId): (JSC::JIT::privateCompilePutByValWithCachedId): (JSC::JIT::emitDoubleLoad): (JSC::JIT::emitContiguousLoad): (JSC::JIT::emitArrayStorageLoad): (JSC::JIT::emitIntTypedArrayPutByVal): (JSC::JIT::emitFloatTypedArrayPutByVal): * jit/JITPropertyAccess32_64.cpp: (JSC::JIT::emit_op_del_by_id): (JSC::JIT::emit_op_del_by_val): (JSC::JIT::emit_op_get_by_val): (JSC::JIT::emit_op_get_private_name): (JSC::JIT::emit_op_set_private_brand): (JSC::JIT::emit_op_check_private_brand): (JSC::JIT::emit_op_put_by_val): (JSC::JIT::emitGenericContiguousPutByVal): (JSC::JIT::emitArrayStoragePutByVal): (JSC::JIT::emit_op_try_get_by_id): (JSC::JIT::emit_op_get_by_id_direct): (JSC::JIT::emit_op_get_by_id): (JSC::JIT::emit_op_get_by_id_with_this): (JSC::JIT::emit_op_put_by_id): (JSC::JIT::emit_op_in_by_id): (JSC::JIT::emit_op_in_by_val): * jit/JITStubRoutine.h: * jit/PolymorphicCallStubRoutine.cpp: (JSC::PolymorphicCallStubRoutine::PolymorphicCallStubRoutine): * jit/Repatch.cpp: (JSC::readPutICCallTarget): (JSC::repatchSlowPathCall): (JSC::tryCacheGetBy): (JSC::repatchGetBy): (JSC::tryCacheArrayGetByVal): (JSC::repatchArrayGetByVal): (JSC::tryCachePutByID): (JSC::repatchPutByID): (JSC::tryCacheDeleteBy): (JSC::repatchDeleteBy): (JSC::tryCacheInBy): (JSC::repatchInBy): (JSC::tryCacheCheckPrivateBrand): (JSC::repatchCheckPrivateBrand): (JSC::tryCacheSetPrivateBrand): (JSC::repatchSetPrivateBrand): (JSC::tryCacheInstanceOf): (JSC::repatchInstanceOf): (JSC::linkSlowFor): (JSC::linkVirtualFor): (JSC::resetGetBy): (JSC::resetPutByID): (JSC::resetDelBy): (JSC::resetInBy): (JSC::resetInstanceOf): (JSC::resetCheckPrivateBrand): (JSC::resetSetPrivateBrand): (JSC::resetPatchableJump): Deleted. * jit/Repatch.h: * runtime/Options.cpp: (JSC::Options::recomputeDependentOptions): * runtime/OptionsList.h: * runtime/StructureIDTable.h: * runtime/VM.cpp: (JSC::VM::VM): (JSC::VM::~VM): * runtime/VM.h: Tools: * Scripts/run-jsc-stress-tests: Canonical link: https://commits.webkit.org/238638@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@278656 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-06-09 11:17:36 +00:00
return SlowPathCallKey(m_thunkSaveSet, callTarget, m_argumentRegisters, m_offset, 0);
}
SlowPathCallKey SlowPathCallContext::keyWithTarget(CCallHelpers::Address address) const
{
return SlowPathCallKey(m_thunkSaveSet, nullptr, m_argumentRegisters, m_offset, address.offset);
Merge r170564, r170571, r170604, r170628, r170672, r170680, r170724, r170728, r170729, r170819, r170821, r170836, r170855, r170860, r170890, r170907, r170929, r171052, r171106, r171152, r171153, r171214 from ftlopt. Source/JavaScriptCore: This part of the merge delivers roughly a 2% across-the-board performance improvement, mostly due to immutable property inference and DFG-side GCSE. It also almost completely resolves accessor performance issues; in the common case the DFG will compile a getter/setter access into code that is just as efficient as a normal property access. Another major highlight of this part of the merge is the work to add a type profiler to the inspector. This work is still on-going but this greatly increases coverage. Note that this merge fixes a minor bug in the GetterSetter refactoring from http://trac.webkit.org/changeset/170729 (https://bugs.webkit.org/show_bug.cgi?id=134518). It also adds a new tests to tests/stress to cover that bug. That bug was previously only covered by layout tests. 2014-07-17 Filip Pizlo <fpizlo@apple.com> [ftlopt] DFG Flush(SetLocal) store elimination is overzealous for captured variables in the presence of nodes that have no effects but may throw (merge trunk r171190) https://bugs.webkit.org/show_bug.cgi?id=135019 Reviewed by Oliver Hunt. Behaviorally, this is just a merge of trunk r171190, except that the relevant functionality has moved to StrengthReductionPhase and is written in a different style. Same algorithm, different code. * dfg/DFGNodeType.h: * dfg/DFGStrengthReductionPhase.cpp: (JSC::DFG::StrengthReductionPhase::handleNode): * tests/stress/capture-escape-and-throw.js: Added. (foo.f): (foo): * tests/stress/new-array-with-size-throw-exception-and-tear-off-arguments.js: Added. (foo): (bar): 2014-07-15 Filip Pizlo <fpizlo@apple.com> [ftlopt] Constant fold GetGetter and GetSetter if the GetterSetter is a constant https://bugs.webkit.org/show_bug.cgi?id=134962 Reviewed by Oliver Hunt. This removes yet another steady-state-throughput implication of using getters and setters: if your accessor call is monomorphic then you'll just get a structure check, nothing more. No more loads to get to the GetterSetter object or the accessor function object. * dfg/DFGAbstractInterpreterInlines.h: (JSC::DFG::AbstractInterpreter<AbstractStateType>::executeEffects): * runtime/GetterSetter.h: (JSC::GetterSetter::getterConcurrently): (JSC::GetterSetter::setGetter): (JSC::GetterSetter::setterConcurrently): (JSC::GetterSetter::setSetter): 2014-07-15 Filip Pizlo <fpizlo@apple.com> [ftlopt] Identity replacement in CSE shouldn't create a Phantom over the Identity's children https://bugs.webkit.org/show_bug.cgi?id=134893 Reviewed by Oliver Hunt. Replace Identity with Check instead of Phantom. Phantom means that the child of the Identity should be unconditionally live. The liveness semantics of Identity are such that if the parents of Identity are live then the child is live. Removing the Identity entirely preserves such liveness semantics. So, the only thing that should be left behind is the type check on the child, which is what Check means: do the check but don't keep the child alive if the check isn't needed. * dfg/DFGCSEPhase.cpp: * dfg/DFGNode.h: (JSC::DFG::Node::convertToCheck): 2014-07-13 Filip Pizlo <fpizlo@apple.com> [ftlopt] DFG should be able to do GCSE in SSA and this should be unified with the CSE in CPS, and both of these things should use abstract heaps for reasoning about effects https://bugs.webkit.org/show_bug.cgi?id=134677 Reviewed by Sam Weinig. This removes the old local CSE phase, which was based on manually written backward-search rules for all of the different kinds of things we cared about, and adds a new local/global CSE (local for CPS and global for SSA) that leaves the node semantics almost entirely up to clobberize(). Thus, the CSE phase itself just worries about the algorithms and data structures used for storing sets of available values. This results in a large reduction in code size in CSEPhase.cpp while greatly increasing the phase's power (since it now does global CSE) and reducing compile time (since local CSE is now rewritten to use smarter data structures). Even though LLVM was already running GVN, the extra GCSE at DFG IR level means that this is a significant (~0.7%) throughput improvement. This work is based on the concept of "def" to clobberize(). If clobberize() calls def(), it means that the node being analyzed makes available some value in some DFG node, and that future attempts to compute that value can simply use that node. In other words, it establishes an available value mapping of the form value=>node. There are two kinds of values that can be passed to def(): PureValue. This captures everything needed to determine whether two pure nodes - nodes that neither read nor write, and produce a value that is a CSE candidate - are identical. It carries the NodeType, an AdjacencyList, and one word of meta-data. The meta-data is usually used for things like the arithmetic mode or constant pointer. Passing a PureValue to def() means that the node produces a value that is valid anywhere that the node dominates. HeapLocation. This describes a location in the heap that could be written to or read from. Both stores and loads can def() a HeapLocation. HeapLocation carries around an abstract heap that both serves as part of the "name" of the heap location (together with the other fields of HeapLocation) and also tells us what write()'s to watch for. If someone write()'s to an abstract heap that overlaps the heap associated with the HeapLocation, then it means that the values for that location are no longer available. This approach is sufficiently clever that the CSEPhase itself can focus on the mechanism of tracking the PureValue=>node and HeapLocation=>node maps, without having to worry about interpreting the semantics of different DFG node types - that is now almost entirely in clobberize(). The only things we special-case inside CSEPhase are the Identity node, which CSE is traditionally responsible for eliminating even though it has nothing to do with CSE, and the LocalCSE rule for turning PutByVal into PutByValAlias. This is a slight Octane, SunSpider, and Kraken speed-up - all somewhere arond 0.7% . It's not a bigger win because LLVM was already giving us most of what we needed in its GVN. Also, the SunSpider speed-up isn't from GCSE as much as it's a clean-up of local CSE - that is no longer O(n^2). Basically this is purely good: it reduces the amount of LLVM IR we generate, it removes the old CSE's heap modeling (which was a constant source of bugs), and it improves both the quality of the code we generate and the speed with which we generate it. Also, any future optimizations that depend on GCSE will now be easier to implement. During the development of this patch I also rationalized some other stuff, like Graph's ordered traversals - we now have preorder and postorder rather than just "depth first". * CMakeLists.txt: * JavaScriptCore.vcxproj/JavaScriptCore.vcxproj: * JavaScriptCore.xcodeproj/project.pbxproj: * dfg/DFGAbstractHeap.h: * dfg/DFGAdjacencyList.h: (JSC::DFG::AdjacencyList::hash): (JSC::DFG::AdjacencyList::operator==): * dfg/DFGBasicBlock.h: * dfg/DFGCSEPhase.cpp: (JSC::DFG::performLocalCSE): (JSC::DFG::performGlobalCSE): (JSC::DFG::CSEPhase::CSEPhase): Deleted. (JSC::DFG::CSEPhase::run): Deleted. (JSC::DFG::CSEPhase::endIndexForPureCSE): Deleted. (JSC::DFG::CSEPhase::pureCSE): Deleted. (JSC::DFG::CSEPhase::constantCSE): Deleted. (JSC::DFG::CSEPhase::constantStoragePointerCSE): Deleted. (JSC::DFG::CSEPhase::getCalleeLoadElimination): Deleted. (JSC::DFG::CSEPhase::getArrayLengthElimination): Deleted. (JSC::DFG::CSEPhase::globalVarLoadElimination): Deleted. (JSC::DFG::CSEPhase::scopedVarLoadElimination): Deleted. (JSC::DFG::CSEPhase::varInjectionWatchpointElimination): Deleted. (JSC::DFG::CSEPhase::getByValLoadElimination): Deleted. (JSC::DFG::CSEPhase::checkFunctionElimination): Deleted. (JSC::DFG::CSEPhase::checkExecutableElimination): Deleted. (JSC::DFG::CSEPhase::checkStructureElimination): Deleted. (JSC::DFG::CSEPhase::structureTransitionWatchpointElimination): Deleted. (JSC::DFG::CSEPhase::getByOffsetLoadElimination): Deleted. (JSC::DFG::CSEPhase::getGetterSetterByOffsetLoadElimination): Deleted. (JSC::DFG::CSEPhase::getPropertyStorageLoadElimination): Deleted. (JSC::DFG::CSEPhase::checkArrayElimination): Deleted. (JSC::DFG::CSEPhase::getIndexedPropertyStorageLoadElimination): Deleted. (JSC::DFG::CSEPhase::getInternalFieldLoadElimination): Deleted. (JSC::DFG::CSEPhase::getMyScopeLoadElimination): Deleted. (JSC::DFG::CSEPhase::getLocalLoadElimination): Deleted. (JSC::DFG::CSEPhase::invalidationPointElimination): Deleted. (JSC::DFG::CSEPhase::setReplacement): Deleted. (JSC::DFG::CSEPhase::eliminate): Deleted. (JSC::DFG::CSEPhase::performNodeCSE): Deleted. (JSC::DFG::CSEPhase::performBlockCSE): Deleted. (JSC::DFG::performCSE): Deleted. * dfg/DFGCSEPhase.h: * dfg/DFGClobberSet.cpp: (JSC::DFG::addReads): (JSC::DFG::addWrites): (JSC::DFG::addReadsAndWrites): (JSC::DFG::readsOverlap): (JSC::DFG::writesOverlap): * dfg/DFGClobberize.cpp: (JSC::DFG::doesWrites): (JSC::DFG::accessesOverlap): (JSC::DFG::writesOverlap): * dfg/DFGClobberize.h: (JSC::DFG::clobberize): (JSC::DFG::NoOpClobberize::operator()): (JSC::DFG::CheckClobberize::operator()): (JSC::DFG::ReadMethodClobberize::ReadMethodClobberize): (JSC::DFG::ReadMethodClobberize::operator()): (JSC::DFG::WriteMethodClobberize::WriteMethodClobberize): (JSC::DFG::WriteMethodClobberize::operator()): (JSC::DFG::DefMethodClobberize::DefMethodClobberize): (JSC::DFG::DefMethodClobberize::operator()): * dfg/DFGDCEPhase.cpp: (JSC::DFG::DCEPhase::run): (JSC::DFG::DCEPhase::fixupBlock): * dfg/DFGGraph.cpp: (JSC::DFG::Graph::getBlocksInPreOrder): (JSC::DFG::Graph::getBlocksInPostOrder): (JSC::DFG::Graph::addForDepthFirstSort): Deleted. (JSC::DFG::Graph::getBlocksInDepthFirstOrder): Deleted. * dfg/DFGGraph.h: * dfg/DFGHeapLocation.cpp: Added. (JSC::DFG::HeapLocation::dump): (WTF::printInternal): * dfg/DFGHeapLocation.h: Added. (JSC::DFG::HeapLocation::HeapLocation): (JSC::DFG::HeapLocation::operator!): (JSC::DFG::HeapLocation::kind): (JSC::DFG::HeapLocation::heap): (JSC::DFG::HeapLocation::base): (JSC::DFG::HeapLocation::index): (JSC::DFG::HeapLocation::hash): (JSC::DFG::HeapLocation::operator==): (JSC::DFG::HeapLocation::isHashTableDeletedValue): (JSC::DFG::HeapLocationHash::hash): (JSC::DFG::HeapLocationHash::equal): * dfg/DFGLICMPhase.cpp: (JSC::DFG::LICMPhase::run): * dfg/DFGNode.h: (JSC::DFG::Node::replaceWith): (JSC::DFG::Node::convertToPhantomUnchecked): Deleted. * dfg/DFGPlan.cpp: (JSC::DFG::Plan::compileInThreadImpl): * dfg/DFGPureValue.cpp: Added. (JSC::DFG::PureValue::dump): * dfg/DFGPureValue.h: Added. (JSC::DFG::PureValue::PureValue): (JSC::DFG::PureValue::operator!): (JSC::DFG::PureValue::op): (JSC::DFG::PureValue::children): (JSC::DFG::PureValue::info): (JSC::DFG::PureValue::hash): (JSC::DFG::PureValue::operator==): (JSC::DFG::PureValue::isHashTableDeletedValue): (JSC::DFG::PureValueHash::hash): (JSC::DFG::PureValueHash::equal): * dfg/DFGSSAConversionPhase.cpp: (JSC::DFG::SSAConversionPhase::run): * ftl/FTLLowerDFGToLLVM.cpp: (JSC::FTL::LowerDFGToLLVM::lower): 2014-07-13 Filip Pizlo <fpizlo@apple.com> Unreviewed, revert unintended change in r171051. * dfg/DFGCSEPhase.cpp: 2014-07-08 Filip Pizlo <fpizlo@apple.com> [ftlopt] Move Flush(SetLocal) store elimination to StrengthReductionPhase https://bugs.webkit.org/show_bug.cgi?id=134739 Reviewed by Mark Hahnenberg. I'm going to streamline CSE around clobberize() as part of https://bugs.webkit.org/show_bug.cgi?id=134677, and so Flush(SetLocal) store elimination wouldn't belong in CSE anymore. It doesn't quite belong anywhere, which means that it belongs in StrengthReductionPhase, since that's intended to be our dumping ground. To do this I had to add some missing smarts to clobberize(). Previously clobberize() could play a bit loose with reads of Variables because it wasn't used for store elimination. The main client of read() was LICM, but it would only use it to determine hoistability and anything that did a write() was not hoistable - so, we had benign (but still wrong) missing read() calls in places that did write()s. This fixes a bunch of those cases. * dfg/DFGCSEPhase.cpp: (JSC::DFG::CSEPhase::performNodeCSE): (JSC::DFG::CSEPhase::setLocalStoreElimination): Deleted. * dfg/DFGClobberize.cpp: (JSC::DFG::accessesOverlap): * dfg/DFGClobberize.h: (JSC::DFG::clobberize): Make clobberize() smart enough for detecting when this store elimination would be sound. * dfg/DFGStrengthReductionPhase.cpp: (JSC::DFG::StrengthReductionPhase::handleNode): Implement the store elimination in terms of clobberize(). 2014-07-08 Filip Pizlo <fpizlo@apple.com> [ftlopt] Phantom simplification should be in its own phase https://bugs.webkit.org/show_bug.cgi?id=134742 Reviewed by Geoffrey Garen. This moves Phantom simplification out of CSE, which greatly simplifies CSE and gives it more focus. Also this finally adds a phase that removes empty Phantoms. We sort of had this in CPSRethreading, but that phase runs too infrequently and doesn't run at all for SSA. * CMakeLists.txt: * JavaScriptCore.vcxproj/JavaScriptCore.vcxproj: * JavaScriptCore.xcodeproj/project.pbxproj: * dfg/DFGAdjacencyList.h: * dfg/DFGCSEPhase.cpp: (JSC::DFG::CSEPhase::run): (JSC::DFG::CSEPhase::setReplacement): (JSC::DFG::CSEPhase::eliminate): (JSC::DFG::CSEPhase::performNodeCSE): (JSC::DFG::CSEPhase::eliminateIrrelevantPhantomChildren): Deleted. * dfg/DFGPhantomRemovalPhase.cpp: Added. (JSC::DFG::PhantomRemovalPhase::PhantomRemovalPhase): (JSC::DFG::PhantomRemovalPhase::run): (JSC::DFG::performCleanUp): * dfg/DFGPhantomRemovalPhase.h: Added. * dfg/DFGPlan.cpp: (JSC::DFG::Plan::compileInThreadImpl): 2014-07-08 Filip Pizlo <fpizlo@apple.com> [ftlopt] Get rid of Node::misc by moving the fields out of the union so that you can use replacement and owner simultaneously https://bugs.webkit.org/show_bug.cgi?id=134730 Reviewed by Mark Lam. This will allow for a better GCSE implementation. * dfg/DFGCPSRethreadingPhase.cpp: (JSC::DFG::CPSRethreadingPhase::canonicalizeGetLocalFor): * dfg/DFGCSEPhase.cpp: (JSC::DFG::CSEPhase::setReplacement): * dfg/DFGEdgeDominates.h: (JSC::DFG::EdgeDominates::operator()): * dfg/DFGGraph.cpp: (JSC::DFG::Graph::clearReplacements): (JSC::DFG::Graph::initializeNodeOwners): * dfg/DFGGraph.h: (JSC::DFG::Graph::performSubstitutionForEdge): * dfg/DFGLICMPhase.cpp: (JSC::DFG::LICMPhase::attemptHoist): * dfg/DFGNode.h: (JSC::DFG::Node::Node): * dfg/DFGSSAConversionPhase.cpp: (JSC::DFG::SSAConversionPhase::run): 2014-07-04 Filip Pizlo <fpizlo@apple.com> [ftlopt] Infer immutable object properties https://bugs.webkit.org/show_bug.cgi?id=134567 Reviewed by Mark Hahnenberg. This introduces a new way of inferring immutable object properties. A property is said to be immutable if after its creation (i.e. the transition that creates it), we never overwrite it (i.e. replace it) or delete it. Immutability is a property of an "own property" - so if we say that "f" is immutable at "o" then we are implying that "o" has "f" directly and not on a prototype. More specifically, the immutability inference will prove that a property on some structure is immutable. This means that, for example, we may have a structure S1 with property "f" where we claim that "f" at S1 is immutable, but S1 has a transition to S2 that adds a new property "g" and we may claim that "f" at S2 is actually mutable. This is mainly for convenience; it allows us to decouple immutability logic from transition logic. Immutability can be used to constant-fold accesses to objects at DFG-time. The DFG needs to prove the following to constant-fold the access: - The base of the access must be a constant object pointer. We prove that a property at a structure is immutable, but that says nothing of its value; each actual instance of that property may have a different value. So, a constant object pointer is needed to get an actual constant instance of the immutable value. - A check (or watchpoint) must have been emitted proving that the object has a structure that allows loading the property in question. - The replacement watchpoint set of the property in the structure that we've proven the object to have is still valid and we add a watchpoint to it lazily. The replacement watchpoint set is the key new mechanism that this change adds. It's possible that we have proven that the object has one of many structures, in which case each of those structures needs a valid replacement watchpoint set. The replacement watchpoint set is created the first time that any access to the property is cached. A put replace cache will create, and immediately invalidate, the watchpoint set. A get cache will create the watchpoint set and make it start watching. Any non-cached put access will invalidate the watchpoint set if one had been created; the underlying algorithm ensures that checking for the existence of a replacement watchpoint set is very fast in the common case. This algorithm ensures that no cached access needs to ever do any work to invalidate, or check the validity of, any replacement watchpoint sets. It also has some other nice properties: - It's very robust in its definition of immutability. The strictest that it will ever be is that for any instance of the object, the property must be written to only once, specifically at the time that the property is created. But it's looser than this in practice. For example, the property may be written to any number of times before we add the final property that the object will have before anyone reads the property; this works since for optimization purposes we only care if we detect immutability on the structure that the object will have when it is most frequently read from, not any previous structure that the object had. Also, we may write to the property any number of times before anyone caches accesses to it. - It is mostly orthogonal to structure transitions. No new structures need to be created to track the immutability of a property. Hence, there is no risk from this feature causing more polymorphism. This is different from the previous "specificValue" constant inference, which did cause additional structures to be created and sometimes those structures led to fake polymorphism. This feature does leverage existing transitions to do some of the watchpointing: property deletions don't fire the replacement watchpoint set because that would cause a new structure and so the mandatory structure check would fail. Also, this feature is guaranteed to never kick in for uncacheable dictionaries because those wouldn't allow for cacheable accesses - and it takes a cacheable access for this feature to be enabled. - No memory overhead is incurred except when accesses to the property are cached. Dictionary properties will typically have no meta-data for immutability. The number of replacement watchpoint sets we allocate is proportional to the number of inline caches in the program, which is typically must smaller than the number of structures or even the number of objects. This inference is far more powerful than the previous "specificValue" inference, so this change also removes all of that code. It's interesting that the amount of code that is changed to remove that feature is almost as big as the amount of code added to support the new inference - and that's if you include the new tests in the tally. Without new tests, it appears that the new feature actually touches less code! There is one corner case where the previous "specificValue" inference was more powerful. You can imagine someone creating objects with functions as self properties on those objects, such that each object instance had the same function pointers - essentially, someone might be trying to create a vtable but failing at the whole "one vtable for many instances" concept. The "specificValue" inference would do very well for such programs, because a structure check would be sufficient to prove a constant value for all of the function properties. This new inference will fail because it doesn't track the constant values of constant properties; instead it detects the immutability of otherwise variable properties (in the sense that each instance of the property may have a different value). So, the new inference requires having a particular object instance to actually get the constant value. I think it's OK to lose this antifeature. It took a lot of code to support and was a constant source of grief in our transition logic, and there doesn't appear to be any real evidence that programs benefited from that particular kind of inference since usually it's the singleton prototype instance that has all of the functions. This change is a speed-up on everything. date-format-xparb and both SunSpider/raytrace and V8/raytrace seem to be the biggest winners among the macrobenchmarks; they see >5% speed-ups. Many of our microbenchmarks see very large performance improvements, even 80% in one case. * bytecode/ComplexGetStatus.cpp: (JSC::ComplexGetStatus::computeFor): * bytecode/GetByIdStatus.cpp: (JSC::GetByIdStatus::computeFromLLInt): (JSC::GetByIdStatus::computeForStubInfo): (JSC::GetByIdStatus::computeFor): * bytecode/GetByIdVariant.cpp: (JSC::GetByIdVariant::GetByIdVariant): (JSC::GetByIdVariant::operator=): (JSC::GetByIdVariant::attemptToMerge): (JSC::GetByIdVariant::dumpInContext): * bytecode/GetByIdVariant.h: (JSC::GetByIdVariant::alternateBase): (JSC::GetByIdVariant::specificValue): Deleted. * bytecode/PutByIdStatus.cpp: (JSC::PutByIdStatus::computeForStubInfo): (JSC::PutByIdStatus::computeFor): * bytecode/PutByIdVariant.cpp: (JSC::PutByIdVariant::operator=): (JSC::PutByIdVariant::setter): (JSC::PutByIdVariant::dumpInContext): * bytecode/PutByIdVariant.h: (JSC::PutByIdVariant::specificValue): Deleted. * bytecode/Watchpoint.cpp: (JSC::WatchpointSet::fireAllSlow): (JSC::WatchpointSet::fireAll): Deleted. * bytecode/Watchpoint.h: (JSC::WatchpointSet::fireAll): * dfg/DFGAbstractInterpreterInlines.h: (JSC::DFG::AbstractInterpreter<AbstractStateType>::executeEffects): * dfg/DFGByteCodeParser.cpp: (JSC::DFG::ByteCodeParser::handleGetByOffset): (JSC::DFG::ByteCodeParser::handleGetById): (JSC::DFG::ByteCodeParser::handlePutById): (JSC::DFG::ByteCodeParser::parseBlock): * dfg/DFGConstantFoldingPhase.cpp: (JSC::DFG::ConstantFoldingPhase::emitGetByOffset): * dfg/DFGFixupPhase.cpp: (JSC::DFG::FixupPhase::isStringPrototypeMethodSane): (JSC::DFG::FixupPhase::canOptimizeStringObjectAccess): * dfg/DFGGraph.cpp: (JSC::DFG::Graph::tryGetConstantProperty): (JSC::DFG::Graph::visitChildren): * dfg/DFGGraph.h: * dfg/DFGWatchableStructureWatchingPhase.cpp: (JSC::DFG::WatchableStructureWatchingPhase::run): * ftl/FTLLowerDFGToLLVM.cpp: (JSC::FTL::LowerDFGToLLVM::compileMultiGetByOffset): * jit/JITOperations.cpp: * jit/Repatch.cpp: (JSC::repatchByIdSelfAccess): (JSC::generateByIdStub): (JSC::tryCacheGetByID): (JSC::tryCachePutByID): (JSC::tryBuildPutByIdList): * llint/LLIntSlowPaths.cpp: (JSC::LLInt::LLINT_SLOW_PATH_DECL): (JSC::LLInt::putToScopeCommon): * runtime/CommonSlowPaths.h: (JSC::CommonSlowPaths::tryCachePutToScopeGlobal): * runtime/IntendedStructureChain.cpp: (JSC::IntendedStructureChain::mayInterceptStoreTo): * runtime/JSCJSValue.cpp: (JSC::JSValue::putToPrimitive): * runtime/JSGlobalObject.cpp: (JSC::JSGlobalObject::reset): * runtime/JSObject.cpp: (JSC::JSObject::put): (JSC::JSObject::putDirectNonIndexAccessor): (JSC::JSObject::deleteProperty): (JSC::JSObject::defaultValue): (JSC::getCallableObjectSlow): Deleted. (JSC::JSObject::getPropertySpecificValue): Deleted. * runtime/JSObject.h: (JSC::JSObject::getDirect): (JSC::JSObject::getDirectOffset): (JSC::JSObject::inlineGetOwnPropertySlot): (JSC::JSObject::putDirectInternal): (JSC::JSObject::putOwnDataProperty): (JSC::JSObject::putDirect): (JSC::JSObject::putDirectWithoutTransition): (JSC::getCallableObject): Deleted. * runtime/JSScope.cpp: (JSC::abstractAccess): * runtime/PropertyMapHashTable.h: (JSC::PropertyMapEntry::PropertyMapEntry): (JSC::PropertyTable::copy): * runtime/PropertyTable.cpp: (JSC::PropertyTable::clone): (JSC::PropertyTable::PropertyTable): (JSC::PropertyTable::visitChildren): Deleted. * runtime/Structure.cpp: (JSC::Structure::Structure): (JSC::Structure::materializePropertyMap): (JSC::Structure::addPropertyTransitionToExistingStructureImpl): (JSC::Structure::addPropertyTransitionToExistingStructure): (JSC::Structure::addPropertyTransitionToExistingStructureConcurrently): (JSC::Structure::addPropertyTransition): (JSC::Structure::changePrototypeTransition): (JSC::Structure::attributeChangeTransition): (JSC::Structure::toDictionaryTransition): (JSC::Structure::preventExtensionsTransition): (JSC::Structure::takePropertyTableOrCloneIfPinned): (JSC::Structure::nonPropertyTransition): (JSC::Structure::addPropertyWithoutTransition): (JSC::Structure::allocateRareData): (JSC::Structure::ensurePropertyReplacementWatchpointSet): (JSC::Structure::startWatchingPropertyForReplacements): (JSC::Structure::didCachePropertyReplacement): (JSC::Structure::startWatchingInternalProperties): (JSC::Structure::copyPropertyTable): (JSC::Structure::copyPropertyTableForPinning): (JSC::Structure::getConcurrently): (JSC::Structure::get): (JSC::Structure::add): (JSC::Structure::visitChildren): (JSC::Structure::prototypeChainMayInterceptStoreTo): (JSC::Structure::dump): (JSC::Structure::despecifyDictionaryFunction): Deleted. (JSC::Structure::despecifyFunctionTransition): Deleted. (JSC::Structure::despecifyFunction): Deleted. (JSC::Structure::despecifyAllFunctions): Deleted. (JSC::Structure::putSpecificValue): Deleted. * runtime/Structure.h: (JSC::Structure::startWatchingPropertyForReplacements): (JSC::Structure::startWatchingInternalPropertiesIfNecessary): (JSC::Structure::startWatchingInternalPropertiesIfNecessaryForEntireChain): (JSC::Structure::transitionDidInvolveSpecificValue): Deleted. (JSC::Structure::disableSpecificFunctionTracking): Deleted. * runtime/StructureInlines.h: (JSC::Structure::getConcurrently): (JSC::Structure::didReplaceProperty): (JSC::Structure::propertyReplacementWatchpointSet): * runtime/StructureRareData.cpp: (JSC::StructureRareData::destroy): * runtime/StructureRareData.h: * tests/stress/infer-constant-global-property.js: Added. (foo.Math.sin): (foo): * tests/stress/infer-constant-property.js: Added. (foo): * tests/stress/jit-cache-poly-replace-then-cache-get-and-fold-then-invalidate.js: Added. (foo): (bar): * tests/stress/jit-cache-replace-then-cache-get-and-fold-then-invalidate.js: Added. (foo): (bar): * tests/stress/jit-put-to-scope-global-cache-watchpoint-invalidate.js: Added. (foo): (bar): * tests/stress/llint-cache-replace-then-cache-get-and-fold-then-invalidate.js: Added. (foo): (bar): * tests/stress/llint-put-to-scope-global-cache-watchpoint-invalidate.js: Added. (foo): (bar): * tests/stress/repeat-put-to-scope-global-with-same-value-watchpoint-invalidate.js: Added. (foo): (bar): 2014-07-03 Saam Barati <sbarati@apple.com> Add more coverage for the profile_types_with_high_fidelity op code. https://bugs.webkit.org/show_bug.cgi?id=134616 Reviewed by Filip Pizlo. More operations are now being recorded by the profile_types_with_high_fidelity opcode. Specifically: function parameters, function return values, function 'this' value, get_by_id, get_by_value, resolve nodes, function return values at the call site. Added more flags to the profile_types_with_high_fidelity opcode so more focused tasks can take place when the instruction is being linked in CodeBlock. Re-worked the type profiler to search through character offset ranges when asked for the type of an expression at a given offset. Removed redundant calls to Structure::toStructureShape in HighFidelityLog and TypeSet by caching calls based on StructureID. * bytecode/BytecodeList.json: * bytecode/BytecodeUseDef.h: (JSC::computeUsesForBytecodeOffset): (JSC::computeDefsForBytecodeOffset): * bytecode/CodeBlock.cpp: (JSC::CodeBlock::CodeBlock): (JSC::CodeBlock::finalizeUnconditionally): (JSC::CodeBlock::scopeDependentProfile): * bytecode/CodeBlock.h: (JSC::CodeBlock::returnStatementTypeSet): * bytecode/TypeLocation.h: * bytecode/UnlinkedCodeBlock.cpp: (JSC::UnlinkedCodeBlock::highFidelityTypeProfileExpressionInfoForBytecodeOffset): (JSC::UnlinkedCodeBlock::addHighFidelityTypeProfileExpressionInfo): * bytecode/UnlinkedCodeBlock.h: * bytecompiler/BytecodeGenerator.cpp: (JSC::BytecodeGenerator::emitMove): (JSC::BytecodeGenerator::emitProfileTypesWithHighFidelity): (JSC::BytecodeGenerator::emitGetFromScopeWithProfile): (JSC::BytecodeGenerator::emitPutToScope): (JSC::BytecodeGenerator::emitPutToScopeWithProfile): (JSC::BytecodeGenerator::emitPutById): (JSC::BytecodeGenerator::emitPutByVal): * bytecompiler/BytecodeGenerator.h: (JSC::BytecodeGenerator::emitHighFidelityTypeProfilingExpressionInfo): * bytecompiler/NodesCodegen.cpp: (JSC::ResolveNode::emitBytecode): (JSC::BracketAccessorNode::emitBytecode): (JSC::DotAccessorNode::emitBytecode): (JSC::FunctionCallValueNode::emitBytecode): (JSC::FunctionCallResolveNode::emitBytecode): (JSC::FunctionCallBracketNode::emitBytecode): (JSC::FunctionCallDotNode::emitBytecode): (JSC::CallFunctionCallDotNode::emitBytecode): (JSC::ApplyFunctionCallDotNode::emitBytecode): (JSC::PostfixNode::emitResolve): (JSC::PostfixNode::emitBracket): (JSC::PostfixNode::emitDot): (JSC::PrefixNode::emitResolve): (JSC::PrefixNode::emitBracket): (JSC::PrefixNode::emitDot): (JSC::ReadModifyResolveNode::emitBytecode): (JSC::AssignResolveNode::emitBytecode): (JSC::AssignDotNode::emitBytecode): (JSC::ReadModifyDotNode::emitBytecode): (JSC::AssignBracketNode::emitBytecode): (JSC::ReadModifyBracketNode::emitBytecode): (JSC::ReturnNode::emitBytecode): (JSC::FunctionBodyNode::emitBytecode): * inspector/agents/InspectorRuntimeAgent.cpp: (Inspector::InspectorRuntimeAgent::getRuntimeTypeForVariableAtOffset): (Inspector::InspectorRuntimeAgent::getRuntimeTypeForVariableInTextRange): Deleted. * inspector/agents/InspectorRuntimeAgent.h: * inspector/protocol/Runtime.json: * llint/LLIntSlowPaths.cpp: (JSC::LLInt::getFromScopeCommon): (JSC::LLInt::LLINT_SLOW_PATH_DECL): * llint/LLIntSlowPaths.h: * llint/LowLevelInterpreter.asm: * runtime/HighFidelityLog.cpp: (JSC::HighFidelityLog::processHighFidelityLog): (JSC::HighFidelityLog::actuallyProcessLogThreadFunction): (JSC::HighFidelityLog::recordTypeInformationForLocation): Deleted. * runtime/HighFidelityLog.h: (JSC::HighFidelityLog::recordTypeInformationForLocation): * runtime/HighFidelityTypeProfiler.cpp: (JSC::HighFidelityTypeProfiler::getTypesForVariableInAtOffset): (JSC::HighFidelityTypeProfiler::getGlobalTypesForVariableAtOffset): (JSC::HighFidelityTypeProfiler::getLocalTypesForVariableAtOffset): (JSC::HighFidelityTypeProfiler::insertNewLocation): (JSC::HighFidelityTypeProfiler::findLocation): (JSC::HighFidelityTypeProfiler::getTypesForVariableInRange): Deleted. (JSC::HighFidelityTypeProfiler::getGlobalTypesForVariableInRange): Deleted. (JSC::HighFidelityTypeProfiler::getLocalTypesForVariableInRange): Deleted. (JSC::HighFidelityTypeProfiler::getLocationBasedHash): Deleted. * runtime/HighFidelityTypeProfiler.h: (JSC::LocationKey::LocationKey): Deleted. (JSC::LocationKey::hash): Deleted. (JSC::LocationKey::operator==): Deleted. * runtime/Structure.cpp: (JSC::Structure::toStructureShape): * runtime/Structure.h: * runtime/TypeSet.cpp: (JSC::TypeSet::TypeSet): (JSC::TypeSet::addTypeForValue): (JSC::TypeSet::seenTypes): (JSC::TypeSet::removeDuplicatesInStructureHistory): Deleted. * runtime/TypeSet.h: (JSC::StructureShape::setConstructorName): * runtime/VM.cpp: (JSC::VM::getTypesForVariableAtOffset): (JSC::VM::dumpHighFidelityProfilingTypes): (JSC::VM::getTypesForVariableInRange): Deleted. * runtime/VM.h: 2014-07-04 Filip Pizlo <fpizlo@apple.com> [ftlopt][REGRESSION] debug tests fail because PutByIdDirect is now implemented in terms of In https://bugs.webkit.org/show_bug.cgi?id=134642 Rubber stamped by Andreas Kling. * ftl/FTLLowerDFGToLLVM.cpp: (JSC::FTL::LowerDFGToLLVM::compileNode): 2014-07-01 Filip Pizlo <fpizlo@apple.com> [ftlopt] Allocate a new GetterSetter if we change the value of any of its entries other than when they were previously null, so that if we constant-infer an accessor slot then we immediately get the function constant for free https://bugs.webkit.org/show_bug.cgi?id=134518 Reviewed by Mark Hahnenberg. This has no real effect right now, particularly since almost all uses of setSetter/setGetter were already allocating a branch new GetterSetter. But once we start doing more aggressive constant property inference, this change will allow us to remove all runtime checks from getter/setter calls. * runtime/GetterSetter.cpp: (JSC::GetterSetter::withGetter): (JSC::GetterSetter::withSetter): * runtime/GetterSetter.h: (JSC::GetterSetter::setGetter): (JSC::GetterSetter::setSetter): * runtime/JSObject.cpp: (JSC::JSObject::defineOwnNonIndexProperty): 2014-07-02 Filip Pizlo <fpizlo@apple.com> [ftlopt] Rename notifyTransitionFromThisStructure to didTransitionFromThisStructure Rubber stamped by Mark Hahnenberg. * runtime/Structure.cpp: (JSC::Structure::Structure): (JSC::Structure::nonPropertyTransition): (JSC::Structure::didTransitionFromThisStructure): (JSC::Structure::notifyTransitionFromThisStructure): Deleted. * runtime/Structure.h: 2014-07-02 Filip Pizlo <fpizlo@apple.com> [ftlopt] Remove the functionality for cloning StructureRareData since we never do that anymore. Rubber stamped by Mark Hahnenberg. * runtime/Structure.cpp: (JSC::Structure::Structure): (JSC::Structure::cloneRareDataFrom): Deleted. * runtime/Structure.h: * runtime/StructureRareData.cpp: (JSC::StructureRareData::clone): Deleted. (JSC::StructureRareData::StructureRareData): Deleted. * runtime/StructureRareData.h: (JSC::StructureRareData::needsCloning): Deleted. 2014-07-01 Mark Lam <mark.lam@apple.com> [ftlopt] DebuggerCallFrame::scope() should return a DebuggerScope. <https://webkit.org/b/134420> Reviewed by Geoffrey Garen. Previously, DebuggerCallFrame::scope() returns a JSActivation (and relevant peers) which the WebInspector will use to introspect CallFrame variables. Instead, we should be returning a DebuggerScope as an abstraction layer that provides the introspection functionality that the WebInspector needs. This is the first step towards not forcing every frame to have a JSActivation object just because the debugger is enabled. 1. Instantiate the debuggerScopeStructure as a member of the JSGlobalObject instead of the VM. This allows JSObject::globalObject() to be able to return the global object for the DebuggerScope. 2. On the DebuggerScope's life-cycle management: The DebuggerCallFrame is designed to be "valid" only during a debugging session (while the debugger is broken) through the use of a DebuggerCallFrameScope in Debugger::pauseIfNeeded(). Once the debugger resumes from the break, the DebuggerCallFrameScope destructs, and the DebuggerCallFrame will be invalidated. We can't guarantee (from this code alone) that the Inspector code isn't still holding a ref to the DebuggerCallFrame (though they shouldn't), but by contract, the frame will be invalidated, and any attempt to query it will return null values. This is pre-existing behavior. Now, we're adding the DebuggerScope into the picture. While a single debugger pause session is in progress, the Inspector may request the scope from the DebuggerCallFrame. While the DebuggerCallFrame is still valid, we want DebuggerCallFrame::scope() to always return the same DebuggerScope object. This is why we hold on to the DebuggerScope with a strong ref. If we use a weak ref instead, the following cooky behavior can manifest: 1. The Inspector calls Debugger::scope() to get the top scope. 2. The Inspector iterates down the scope chain and is now only holding a reference to a parent scope. It is no longer referencing the top scope. 3. A GC occurs, and the DebuggerCallFrame's weak m_scope ref to the top scope gets cleared. 4. The Inspector calls DebuggerCallFrame::scope() to get the top scope again but gets a different DebuggerScope instance. 5. The Inspector iterates down the scope chain but never sees the parent scope instance that retained a ref to in step 2 above. This is because when iterating this new DebuggerScope instance (which has no knowledge of the previous parent DebuggerScope instance), a new DebuggerScope instance will get created for the same parent scope. Since the DebuggerScope is a JSObject, it's liveness is determined by its reachability. However, it's "validity" is determined by the life-cycle of its owner DebuggerCallFrame. When the owner DebuggerCallFrame gets invalidated, its debugger scope chain (if instantiated) will also get invalidated. This is why we need the DebuggerScope::invalidateChain() method. The Inspector should not be using the DebuggerScope instance after its owner DebuggerCallFrame is invalidated. If it does, those methods will do nothing or returned a failed status. * debugger/Debugger.h: * debugger/DebuggerCallFrame.cpp: (JSC::DebuggerCallFrame::scope): (JSC::DebuggerCallFrame::evaluate): (JSC::DebuggerCallFrame::invalidate): (JSC::DebuggerCallFrame::vm): (JSC::DebuggerCallFrame::lexicalGlobalObject): * debugger/DebuggerCallFrame.h: * debugger/DebuggerScope.cpp: (JSC::DebuggerScope::DebuggerScope): (JSC::DebuggerScope::finishCreation): (JSC::DebuggerScope::visitChildren): (JSC::DebuggerScope::className): (JSC::DebuggerScope::getOwnPropertySlot): (JSC::DebuggerScope::put): (JSC::DebuggerScope::deleteProperty): (JSC::DebuggerScope::getOwnPropertyNames): (JSC::DebuggerScope::defineOwnProperty): (JSC::DebuggerScope::next): (JSC::DebuggerScope::invalidateChain): (JSC::DebuggerScope::isWithScope): (JSC::DebuggerScope::isGlobalScope): (JSC::DebuggerScope::isFunctionScope): * debugger/DebuggerScope.h: (JSC::DebuggerScope::create): (JSC::DebuggerScope::Iterator::Iterator): (JSC::DebuggerScope::Iterator::get): (JSC::DebuggerScope::Iterator::operator++): (JSC::DebuggerScope::Iterator::operator==): (JSC::DebuggerScope::Iterator::operator!=): (JSC::DebuggerScope::isValid): (JSC::DebuggerScope::jsScope): (JSC::DebuggerScope::begin): (JSC::DebuggerScope::end): * inspector/JSJavaScriptCallFrame.cpp: (Inspector::JSJavaScriptCallFrame::scopeType): (Inspector::JSJavaScriptCallFrame::scopeChain): * inspector/JavaScriptCallFrame.h: (Inspector::JavaScriptCallFrame::scopeChain): * inspector/ScriptDebugServer.cpp: * runtime/JSGlobalObject.cpp: (JSC::JSGlobalObject::reset): (JSC::JSGlobalObject::visitChildren): * runtime/JSGlobalObject.h: (JSC::JSGlobalObject::debuggerScopeStructure): * runtime/JSObject.h: (JSC::JSObject::isWithScope): * runtime/JSScope.h: * runtime/VM.cpp: (JSC::VM::VM): * runtime/VM.h: 2014-07-01 Filip Pizlo <fpizlo@apple.com> [ftlopt] DFG bytecode parser should turn PutById with nothing but a Setter stub as stuff+handleCall, and handleCall should be allowed to inline if it wants to https://bugs.webkit.org/show_bug.cgi?id=130756 Reviewed by Oliver Hunt. The enables exposing the call to setters in the DFG, and then inlining it. Previously we already supproted inlined-cached calls to setters from within put_by_id inline caches, and the DFG could certainly emit such IC's. Now, if an IC had a setter call, then the DFG will either emit the GetGetterSetterByOffset/GetSetter/Call combo, or it will do one better and inline the call. A lot of the core functionality was already available from the previous work to inline getters. So, there are some refactorings in this patch that move preexisting functionality around. For example, the work to figure out how the DFG should go about getting to what we call the "loaded value" - i.e. the GetterSetter object reference in the case of accessors - is now shared in ComplexGetStatus, and both GetByIdStatus and PutByIdStatus use it. This means that we can keep the safety checks common. This patch also does additional refactorings in DFG::ByteCodeParser so that we can continue to reuse handleCall() for all of the various kinds of calls we can now emit. 83% speed-up on getter-richards, 2% speed-up on box2d. * CMakeLists.txt: * JavaScriptCore.vcxproj/JavaScriptCore.vcxproj: * JavaScriptCore.xcodeproj/project.pbxproj: * bytecode/ComplexGetStatus.cpp: Added. (JSC::ComplexGetStatus::computeFor): * bytecode/ComplexGetStatus.h: Added. (JSC::ComplexGetStatus::ComplexGetStatus): (JSC::ComplexGetStatus::skip): (JSC::ComplexGetStatus::takesSlowPath): (JSC::ComplexGetStatus::kind): (JSC::ComplexGetStatus::attributes): (JSC::ComplexGetStatus::specificValue): (JSC::ComplexGetStatus::offset): (JSC::ComplexGetStatus::chain): * bytecode/GetByIdStatus.cpp: (JSC::GetByIdStatus::computeForStubInfo): * bytecode/GetByIdVariant.cpp: (JSC::GetByIdVariant::GetByIdVariant): * bytecode/PolymorphicPutByIdList.h: (JSC::PutByIdAccess::PutByIdAccess): (JSC::PutByIdAccess::setter): (JSC::PutByIdAccess::structure): (JSC::PutByIdAccess::chainCount): * bytecode/PutByIdStatus.cpp: (JSC::PutByIdStatus::computeFromLLInt): (JSC::PutByIdStatus::computeFor): (JSC::PutByIdStatus::computeForStubInfo): (JSC::PutByIdStatus::makesCalls): * bytecode/PutByIdStatus.h: (JSC::PutByIdStatus::makesCalls): Deleted. * bytecode/PutByIdVariant.cpp: (JSC::PutByIdVariant::PutByIdVariant): (JSC::PutByIdVariant::operator=): (JSC::PutByIdVariant::replace): (JSC::PutByIdVariant::transition): (JSC::PutByIdVariant::setter): (JSC::PutByIdVariant::writesStructures): (JSC::PutByIdVariant::reallocatesStorage): (JSC::PutByIdVariant::makesCalls): (JSC::PutByIdVariant::dumpInContext): * bytecode/PutByIdVariant.h: (JSC::PutByIdVariant::PutByIdVariant): (JSC::PutByIdVariant::structure): (JSC::PutByIdVariant::oldStructure): (JSC::PutByIdVariant::alternateBase): (JSC::PutByIdVariant::specificValue): (JSC::PutByIdVariant::callLinkStatus): (JSC::PutByIdVariant::replace): Deleted. (JSC::PutByIdVariant::transition): Deleted. * dfg/DFGByteCodeParser.cpp: (JSC::DFG::ByteCodeParser::addCallWithoutSettingResult): (JSC::DFG::ByteCodeParser::addCall): (JSC::DFG::ByteCodeParser::handleCall): (JSC::DFG::ByteCodeParser::handleInlining): (JSC::DFG::ByteCodeParser::handleGetById): (JSC::DFG::ByteCodeParser::handlePutById): (JSC::DFG::ByteCodeParser::parseBlock): * jit/Repatch.cpp: (JSC::tryCachePutByID): (JSC::tryBuildPutByIdList): * runtime/IntendedStructureChain.cpp: (JSC::IntendedStructureChain::takesSlowPathInDFGForImpureProperty): * runtime/IntendedStructureChain.h: * tests/stress/exit-from-setter.js: Added. * tests/stress/poly-chain-setter.js: Added. (Cons): (foo): (test): * tests/stress/poly-chain-then-setter.js: Added. (Cons1): (Cons2): (foo): (test): * tests/stress/poly-setter-combo.js: Added. (Cons1): (Cons2): (foo): (test): (.test): * tests/stress/poly-setter-then-self.js: Added. (foo): (test): (.test): * tests/stress/weird-setter-counter.js: Added. (foo): (test): * tests/stress/weird-setter-counter-syntactic.js: Added. (foo): (test): 2014-07-01 Matthew Mirman <mmirman@apple.com> Added an implementation of the "in" check to FTL. https://bugs.webkit.org/show_bug.cgi?id=134508 Reviewed by Filip Pizlo. * ftl/FTLCapabilities.cpp: enabled compilation for "in" (JSC::FTL::canCompile): ditto * ftl/FTLCompile.cpp: (JSC::FTL::generateCheckInICFastPath): added. (JSC::FTL::fixFunctionBasedOnStackMaps): added case for CheckIn descriptors. * ftl/FTLInlineCacheDescriptor.h: (JSC::FTL::CheckInGenerator::CheckInGenerator): added. (JSC::FTL::CheckInDescriptor::CheckInDescriptor): added. * ftl/FTLInlineCacheSize.cpp: (JSC::FTL::sizeOfCheckIn): added. Currently larger than necessary. * ftl/FTLInlineCacheSize.h: ditto * ftl/FTLIntrinsicRepository.h: Added function type for operationInGeneric * ftl/FTLLowerDFGToLLVM.cpp: (JSC::FTL::LowerDFGToLLVM::compileNode): added case for In. (JSC::FTL::LowerDFGToLLVM::compileIn): added. * ftl/FTLSlowPathCall.cpp: Added a callOperation for operationIn (JSC::FTL::callOperation): ditto * ftl/FTLSlowPathCall.h: ditto * ftl/FTLState.h: Added a vector to hold CheckIn descriptors. * jit/JITOperations.h: made operationIns internal. * tests/stress/ftl-checkin.js: Added. * tests/stress/ftl-checkin-variable.js: Added. 2014-06-30 Mark Hahnenberg <mhahnenberg@apple.com> CodeBlock::stronglyVisitWeakReferences should mark DFG::CommonData::weakStructureReferences https://bugs.webkit.org/show_bug.cgi?id=134455 Reviewed by Geoffrey Garen. Otherwise we get hanging pointers which can cause us to die later. * bytecode/CodeBlock.cpp: (JSC::CodeBlock::stronglyVisitWeakReferences): 2014-06-27 Filip Pizlo <fpizlo@apple.com> [ftlopt] Reduce the GC's influence on optimization decisions https://bugs.webkit.org/show_bug.cgi?id=134427 Reviewed by Oliver Hunt. This is a slight speed-up on some platforms, that arises from a bunch of fixes that I made while trying to make the GC keep more structures alive (https://bugs.webkit.org/show_bug.cgi?id=128072). The fixes are, roughly: - If the GC clears an inline cache, then this no longer causes the IC to be forever polymorphic. - If we exit in inlined code into a function that tries to OSR enter, then we jettison sooner. - Some variables being uninitialized led to rage-recompilations. This is a pretty strong step in the direction of keeping more Structures alive and not blowing away code just because a Structure died. But, it seems like there is still a slight speed-up to be had from blowing away code that references dead Structures. * bytecode/CodeBlock.cpp: (JSC::CodeBlock::dumpAssumingJITType): (JSC::shouldMarkTransition): (JSC::CodeBlock::propagateTransitions): (JSC::CodeBlock::determineLiveness): * bytecode/GetByIdStatus.cpp: (JSC::GetByIdStatus::computeForStubInfo): * bytecode/PutByIdStatus.cpp: (JSC::PutByIdStatus::computeForStubInfo): * dfg/DFGCapabilities.cpp: (JSC::DFG::isSupportedForInlining): (JSC::DFG::mightInlineFunctionForCall): (JSC::DFG::mightInlineFunctionForClosureCall): (JSC::DFG::mightInlineFunctionForConstruct): * dfg/DFGCapabilities.h: * dfg/DFGCommonData.h: * dfg/DFGDesiredWeakReferences.cpp: (JSC::DFG::DesiredWeakReferences::reallyAdd): * dfg/DFGOSREntry.cpp: (JSC::DFG::prepareOSREntry): * dfg/DFGOSRExitCompilerCommon.cpp: (JSC::DFG::handleExitCounts): * dfg/DFGOperations.cpp: * dfg/DFGOperations.h: * ftl/FTLForOSREntryJITCode.cpp: (JSC::FTL::ForOSREntryJITCode::ForOSREntryJITCode): These variables being uninitialized is benign in terms of correctness but can sometimes cause rage-recompilations. For some reason it took this patch to reveal this. * ftl/FTLOSREntry.cpp: (JSC::FTL::prepareOSREntry): * runtime/Executable.cpp: (JSC::ExecutableBase::destroy): (JSC::NativeExecutable::destroy): (JSC::ScriptExecutable::ScriptExecutable): (JSC::ScriptExecutable::destroy): (JSC::ScriptExecutable::installCode): (JSC::EvalExecutable::EvalExecutable): (JSC::ProgramExecutable::ProgramExecutable): * runtime/Executable.h: (JSC::ScriptExecutable::setDidTryToEnterInLoop): (JSC::ScriptExecutable::didTryToEnterInLoop): (JSC::ScriptExecutable::addressOfDidTryToEnterInLoop): (JSC::ScriptExecutable::ScriptExecutable): Deleted. * runtime/StructureInlines.h: (JSC::Structure::storedPrototypeObject): (JSC::Structure::storedPrototypeStructure): 2014-06-25 Filip Pizlo <fpizlo@apple.com> [ftlopt] If a CodeBlock is jettisoned due to a watchpoint then it should be possible to figure out something about that watchpoint https://bugs.webkit.org/show_bug.cgi?id=134333 Reviewed by Geoffrey Garen. This is engineered to provide loads of information to the profiler without incurring any costs when the profiler is disabled. It's the oldest trick in the book: the thing that fires the watchpoint doesn't actually create anything to describe the reason why it was fired; instead it creates a stack-allocated FireDetail subclass instance. Only if the FireDetail::dump() virtual method is called does anything happen. Currently we use this to produce very fine-grained data for Structure watchpoints and some cases of variable watchpoints. For all other situations, the given reason is just a string constant, by using StringFireDetail. If we find a situation where that string constant is insufficient to diagnose an issue then we can change it to provide more fine-grained information. * JavaScriptCore.xcodeproj/project.pbxproj: * bytecode/CodeBlock.cpp: (JSC::CodeBlock::CodeBlock): (JSC::CodeBlock::jettison): * bytecode/CodeBlock.h: * bytecode/CodeBlockJettisoningWatchpoint.cpp: (JSC::CodeBlockJettisoningWatchpoint::fireInternal): * bytecode/CodeBlockJettisoningWatchpoint.h: * bytecode/ProfiledCodeBlockJettisoningWatchpoint.cpp: Removed. * bytecode/ProfiledCodeBlockJettisoningWatchpoint.h: Removed. * bytecode/StructureStubClearingWatchpoint.cpp: (JSC::StructureStubClearingWatchpoint::fireInternal): * bytecode/StructureStubClearingWatchpoint.h: * bytecode/VariableWatchpointSet.h: (JSC::VariableWatchpointSet::invalidate): (JSC::VariableWatchpointSet::finalizeUnconditionally): * bytecode/VariableWatchpointSetInlines.h: (JSC::VariableWatchpointSet::notifyWrite): * bytecode/Watchpoint.cpp: (JSC::StringFireDetail::dump): (JSC::WatchpointSet::fireAll): (JSC::WatchpointSet::fireAllSlow): (JSC::WatchpointSet::fireAllWatchpoints): (JSC::InlineWatchpointSet::fireAll): * bytecode/Watchpoint.h: (JSC::FireDetail::FireDetail): (JSC::FireDetail::~FireDetail): (JSC::StringFireDetail::StringFireDetail): (JSC::Watchpoint::fire): (JSC::WatchpointSet::fireAll): (JSC::WatchpointSet::touch): (JSC::WatchpointSet::invalidate): (JSC::InlineWatchpointSet::fireAll): (JSC::InlineWatchpointSet::touch): * dfg/DFGCommonData.h: * dfg/DFGOperations.cpp: * interpreter/Interpreter.cpp: (JSC::Interpreter::execute): * jsc.cpp: (WTF::Masquerader::create): * profiler/ProfilerCompilation.cpp: (JSC::Profiler::Compilation::setJettisonReason): (JSC::Profiler::Compilation::toJS): * profiler/ProfilerCompilation.h: (JSC::Profiler::Compilation::setJettisonReason): Deleted. * runtime/ArrayBuffer.cpp: (JSC::ArrayBuffer::transfer): * runtime/ArrayBufferNeuteringWatchpoint.cpp: (JSC::ArrayBufferNeuteringWatchpoint::fireAll): * runtime/ArrayBufferNeuteringWatchpoint.h: * runtime/CommonIdentifiers.h: * runtime/CommonSlowPaths.cpp: (JSC::SLOW_PATH_DECL): * runtime/Identifier.cpp: (JSC::Identifier::dump): * runtime/Identifier.h: * runtime/JSFunction.cpp: (JSC::JSFunction::put): (JSC::JSFunction::defineOwnProperty): * runtime/JSGlobalObject.cpp: (JSC::JSGlobalObject::addFunction): (JSC::JSGlobalObject::haveABadTime): * runtime/JSSymbolTableObject.cpp: (JSC::VariableWriteFireDetail::dump): * runtime/JSSymbolTableObject.h: (JSC::VariableWriteFireDetail::VariableWriteFireDetail): (JSC::symbolTablePut): (JSC::symbolTablePutWithAttributes): * runtime/PropertyName.h: (JSC::PropertyName::dump): * runtime/Structure.cpp: (JSC::Structure::notifyTransitionFromThisStructure): * runtime/Structure.h: (JSC::Structure::notifyTransitionFromThisStructure): Deleted. * runtime/SymbolTable.cpp: (JSC::SymbolTableEntry::notifyWriteSlow): (JSC::SymbolTable::WatchpointCleanup::finalizeUnconditionally): * runtime/SymbolTable.h: (JSC::SymbolTableEntry::notifyWrite): * runtime/VM.cpp: (JSC::VM::addImpureProperty): Source/WebCore: 2014-07-01 Mark Lam <mark.lam@apple.com> [ftlopt] DebuggerCallFrame::scope() should return a DebuggerScope. <https://webkit.org/b/134420> Reviewed by Geoffrey Garen. No new tests. * ForwardingHeaders/debugger/DebuggerCallFrame.h: Removed. - This is not in use. Hence, we can remove it. * bindings/js/ScriptController.cpp: (WebCore::ScriptController::attachDebugger): - We should acquire the JSLock before modifying a JS global object. 2014-06-25 Filip Pizlo <fpizlo@apple.com> [ftlopt] If a CodeBlock is jettisoned due to a watchpoint then it should be possible to figure out something about that watchpoint https://bugs.webkit.org/show_bug.cgi?id=134333 Reviewed by Geoffrey Garen. No new tests because no change in behavior. * bindings/scripts/CodeGeneratorJS.pm: (GenerateHeader): Tools: 2014-06-25 Filip Pizlo <fpizlo@apple.com> [ftlopt] If a CodeBlock is jettisoned due to a watchpoint then it should be possible to figure out something about that watchpoint https://bugs.webkit.org/show_bug.cgi?id=134333 Reviewed by Geoffrey Garen. * Scripts/display-profiler-output: LayoutTests: 2014-07-16 Mark Hahnenberg <mhahnenberg@apple.com> sputnik/Implementation_Diagnostics/S12.6.4_D1.html depends on undefined behavior https://bugs.webkit.org/show_bug.cgi?id=135007 Reviewed by Filip Pizlo. EcmaScript 5.1 specifies that during for-in enumeration newly added properties may or may not be visited during the current enumeration. Specifically, in section 12.6.4 the spec states: "If new properties are added to the object being enumerated during enumeration, the newly added properties are not guaranteed to be visited in the active enumeration." The sputnik/Implementation_Diagnostics/S12.6.4_D1.html layout test is from before sputnik was added to the test262 suite. I believe it has since been removed, so it would probably be okay to remove it from our layout test suite. * sputnik/Implementation_Diagnostics/S12.6.4_D1-expected.txt: Removed. * sputnik/Implementation_Diagnostics/S12.6.4_D1.html: Removed. 2014-07-13 Filip Pizlo <fpizlo@apple.com> [ftlopt] DFG should be able to do GCSE in SSA and this should be unified with the CSE in CPS, and both of these things should use abstract heaps for reasoning about effects https://bugs.webkit.org/show_bug.cgi?id=134677 Reviewed by Sam Weinig. * js/regress/gcse-expected.txt: Added. * js/regress/gcse-poly-get-expected.txt: Added. * js/regress/gcse-poly-get-less-obvious-expected.txt: Added. * js/regress/gcse-poly-get-less-obvious.html: Added. * js/regress/gcse-poly-get.html: Added. * js/regress/gcse.html: Added. * js/regress/script-tests/gcse-poly-get-less-obvious.js: Added. * js/regress/script-tests/gcse-poly-get.js: Added. * js/regress/script-tests/gcse.js: Added. 2014-07-04 Filip Pizlo <fpizlo@apple.com> [ftlopt] Infer immutable object properties https://bugs.webkit.org/show_bug.cgi?id=134567 Reviewed by Mark Hahnenberg. * js/regress/infer-constant-global-property-expected.txt: Added. * js/regress/infer-constant-global-property.html: Added. * js/regress/infer-constant-property-expected.txt: Added. * js/regress/infer-constant-property.html: Added. * js/regress/script-tests/infer-constant-global-property.js: Added. * js/regress/script-tests/infer-constant-property.js: Added. Canonical link: https://commits.webkit.org/153499@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@172129 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2014-08-06 05:27:46 +00:00
}
Templatize CodePtr/Refs/FunctionPtrs with PtrTags. https://bugs.webkit.org/show_bug.cgi?id=184702 <rdar://problem/35391681> Reviewed by Filip Pizlo and Saam Barati. Source/JavaScriptCore: 1. Templatized MacroAssemblerCodePtr/Ref, FunctionPtr, and CodeLocation variants to take a PtrTag template argument. 2. Replaced some uses of raw pointers with the equivalent CodePtr / FunctionPtr. * assembler/AbstractMacroAssembler.h: (JSC::AbstractMacroAssembler::differenceBetweenCodePtr): (JSC::AbstractMacroAssembler::linkJump): (JSC::AbstractMacroAssembler::linkPointer): (JSC::AbstractMacroAssembler::getLinkerAddress): (JSC::AbstractMacroAssembler::repatchJump): (JSC::AbstractMacroAssembler::repatchJumpToNop): (JSC::AbstractMacroAssembler::repatchNearCall): (JSC::AbstractMacroAssembler::repatchCompact): (JSC::AbstractMacroAssembler::repatchInt32): (JSC::AbstractMacroAssembler::repatchPointer): (JSC::AbstractMacroAssembler::readPointer): (JSC::AbstractMacroAssembler::replaceWithLoad): (JSC::AbstractMacroAssembler::replaceWithAddressComputation): * assembler/CodeLocation.h: (JSC::CodeLocationCommon:: const): (JSC::CodeLocationCommon::CodeLocationCommon): (JSC::CodeLocationInstruction::CodeLocationInstruction): (JSC::CodeLocationLabel::CodeLocationLabel): (JSC::CodeLocationLabel::retagged): (JSC::CodeLocationLabel:: const): (JSC::CodeLocationJump::CodeLocationJump): (JSC::CodeLocationJump::retagged): (JSC::CodeLocationCall::CodeLocationCall): (JSC::CodeLocationCall::retagged): (JSC::CodeLocationNearCall::CodeLocationNearCall): (JSC::CodeLocationDataLabel32::CodeLocationDataLabel32): (JSC::CodeLocationDataLabelCompact::CodeLocationDataLabelCompact): (JSC::CodeLocationDataLabelPtr::CodeLocationDataLabelPtr): (JSC::CodeLocationConvertibleLoad::CodeLocationConvertibleLoad): (JSC::CodeLocationCommon<tag>::instructionAtOffset): (JSC::CodeLocationCommon<tag>::labelAtOffset): (JSC::CodeLocationCommon<tag>::jumpAtOffset): (JSC::CodeLocationCommon<tag>::callAtOffset): (JSC::CodeLocationCommon<tag>::nearCallAtOffset): (JSC::CodeLocationCommon<tag>::dataLabelPtrAtOffset): (JSC::CodeLocationCommon<tag>::dataLabel32AtOffset): (JSC::CodeLocationCommon<tag>::dataLabelCompactAtOffset): (JSC::CodeLocationCommon<tag>::convertibleLoadAtOffset): (JSC::CodeLocationCommon::instructionAtOffset): Deleted. (JSC::CodeLocationCommon::labelAtOffset): Deleted. (JSC::CodeLocationCommon::jumpAtOffset): Deleted. (JSC::CodeLocationCommon::callAtOffset): Deleted. (JSC::CodeLocationCommon::nearCallAtOffset): Deleted. (JSC::CodeLocationCommon::dataLabelPtrAtOffset): Deleted. (JSC::CodeLocationCommon::dataLabel32AtOffset): Deleted. (JSC::CodeLocationCommon::dataLabelCompactAtOffset): Deleted. (JSC::CodeLocationCommon::convertibleLoadAtOffset): Deleted. * assembler/LinkBuffer.cpp: (JSC::LinkBuffer::finalizeCodeWithoutDisassemblyImpl): (JSC::LinkBuffer::finalizeCodeWithDisassemblyImpl): (JSC::LinkBuffer::finalizeCodeWithoutDisassembly): Deleted. (JSC::LinkBuffer::finalizeCodeWithDisassembly): Deleted. * assembler/LinkBuffer.h: (JSC::LinkBuffer::link): (JSC::LinkBuffer::patch): (JSC::LinkBuffer::entrypoint): (JSC::LinkBuffer::locationOf): (JSC::LinkBuffer::locationOfNearCall): (JSC::LinkBuffer::finalizeCodeWithoutDisassembly): (JSC::LinkBuffer::finalizeCodeWithDisassembly): (JSC::LinkBuffer::trampolineAt): * assembler/MacroAssemblerARM.h: (JSC::MacroAssemblerARM::readCallTarget): (JSC::MacroAssemblerARM::replaceWithJump): (JSC::MacroAssemblerARM::startOfPatchableBranch32WithPatchOnAddress): (JSC::MacroAssemblerARM::startOfPatchableBranchPtrWithPatchOnAddress): (JSC::MacroAssemblerARM::startOfBranchPtrWithPatchOnRegister): (JSC::MacroAssemblerARM::revertJumpReplacementToBranchPtrWithPatch): (JSC::MacroAssemblerARM::revertJumpReplacementToPatchableBranch32WithPatch): (JSC::MacroAssemblerARM::revertJumpReplacementToPatchableBranchPtrWithPatch): (JSC::MacroAssemblerARM::repatchCall): (JSC::MacroAssemblerARM::linkCall): * assembler/MacroAssemblerARM64.h: (JSC::MacroAssemblerARM64::readCallTarget): (JSC::MacroAssemblerARM64::replaceWithVMHalt): (JSC::MacroAssemblerARM64::replaceWithJump): (JSC::MacroAssemblerARM64::startOfBranchPtrWithPatchOnRegister): (JSC::MacroAssemblerARM64::startOfPatchableBranchPtrWithPatchOnAddress): (JSC::MacroAssemblerARM64::startOfPatchableBranch32WithPatchOnAddress): (JSC::MacroAssemblerARM64::revertJumpReplacementToBranchPtrWithPatch): (JSC::MacroAssemblerARM64::revertJumpReplacementToPatchableBranchPtrWithPatch): (JSC::MacroAssemblerARM64::revertJumpReplacementToPatchableBranch32WithPatch): (JSC::MacroAssemblerARM64::repatchCall): (JSC::MacroAssemblerARM64::linkCall): * assembler/MacroAssemblerARMv7.h: (JSC::MacroAssemblerARMv7::replaceWithJump): (JSC::MacroAssemblerARMv7::readCallTarget): (JSC::MacroAssemblerARMv7::startOfBranchPtrWithPatchOnRegister): (JSC::MacroAssemblerARMv7::revertJumpReplacementToBranchPtrWithPatch): (JSC::MacroAssemblerARMv7::startOfPatchableBranchPtrWithPatchOnAddress): (JSC::MacroAssemblerARMv7::startOfPatchableBranch32WithPatchOnAddress): (JSC::MacroAssemblerARMv7::revertJumpReplacementToPatchableBranchPtrWithPatch): (JSC::MacroAssemblerARMv7::revertJumpReplacementToPatchableBranch32WithPatch): (JSC::MacroAssemblerARMv7::repatchCall): (JSC::MacroAssemblerARMv7::linkCall): * assembler/MacroAssemblerCodeRef.cpp: (JSC::MacroAssemblerCodePtrBase::dumpWithName): (JSC::MacroAssemblerCodeRefBase::tryToDisassemble): (JSC::MacroAssemblerCodeRefBase::disassembly): (JSC::MacroAssemblerCodePtr::createLLIntCodePtr): Deleted. (JSC::MacroAssemblerCodePtr::dumpWithName const): Deleted. (JSC::MacroAssemblerCodePtr::dump const): Deleted. (JSC::MacroAssemblerCodeRef::createLLIntCodeRef): Deleted. (JSC::MacroAssemblerCodeRef::tryToDisassemble const): Deleted. (JSC::MacroAssemblerCodeRef::disassembly const): Deleted. (JSC::MacroAssemblerCodeRef::dump const): Deleted. * assembler/MacroAssemblerCodeRef.h: (JSC::FunctionPtr::FunctionPtr): (JSC::FunctionPtr::retagged const): (JSC::FunctionPtr::retaggedExecutableAddress const): (JSC::FunctionPtr::operator== const): (JSC::FunctionPtr::operator!= const): (JSC::ReturnAddressPtr::ReturnAddressPtr): (JSC::MacroAssemblerCodePtr::MacroAssemblerCodePtr): (JSC::MacroAssemblerCodePtr::createFromExecutableAddress): (JSC::MacroAssemblerCodePtr::retagged const): (JSC::MacroAssemblerCodePtr:: const): (JSC::MacroAssemblerCodePtr::dumpWithName const): (JSC::MacroAssemblerCodePtr::dump const): (JSC::MacroAssemblerCodePtrHash::hash): (JSC::MacroAssemblerCodePtrHash::equal): (JSC::MacroAssemblerCodeRef::MacroAssemblerCodeRef): (JSC::MacroAssemblerCodeRef::createSelfManagedCodeRef): (JSC::MacroAssemblerCodeRef::code const): (JSC::MacroAssemblerCodeRef::retaggedCode const): (JSC::MacroAssemblerCodeRef::retagged const): (JSC::MacroAssemblerCodeRef::tryToDisassemble const): (JSC::MacroAssemblerCodeRef::disassembly const): (JSC::MacroAssemblerCodeRef::dump const): (JSC::FunctionPtr<tag>::FunctionPtr): * assembler/MacroAssemblerMIPS.h: (JSC::MacroAssemblerMIPS::readCallTarget): (JSC::MacroAssemblerMIPS::replaceWithJump): (JSC::MacroAssemblerMIPS::startOfPatchableBranch32WithPatchOnAddress): (JSC::MacroAssemblerMIPS::startOfBranchPtrWithPatchOnRegister): (JSC::MacroAssemblerMIPS::revertJumpReplacementToBranchPtrWithPatch): (JSC::MacroAssemblerMIPS::startOfPatchableBranchPtrWithPatchOnAddress): (JSC::MacroAssemblerMIPS::revertJumpReplacementToPatchableBranch32WithPatch): (JSC::MacroAssemblerMIPS::revertJumpReplacementToPatchableBranchPtrWithPatch): (JSC::MacroAssemblerMIPS::repatchCall): (JSC::MacroAssemblerMIPS::linkCall): * assembler/MacroAssemblerX86.h: (JSC::MacroAssemblerX86::readCallTarget): (JSC::MacroAssemblerX86::startOfBranchPtrWithPatchOnRegister): (JSC::MacroAssemblerX86::startOfPatchableBranchPtrWithPatchOnAddress): (JSC::MacroAssemblerX86::startOfPatchableBranch32WithPatchOnAddress): (JSC::MacroAssemblerX86::revertJumpReplacementToBranchPtrWithPatch): (JSC::MacroAssemblerX86::revertJumpReplacementToPatchableBranchPtrWithPatch): (JSC::MacroAssemblerX86::revertJumpReplacementToPatchableBranch32WithPatch): (JSC::MacroAssemblerX86::repatchCall): (JSC::MacroAssemblerX86::linkCall): * assembler/MacroAssemblerX86Common.h: (JSC::MacroAssemblerX86Common::repatchCompact): (JSC::MacroAssemblerX86Common::replaceWithVMHalt): (JSC::MacroAssemblerX86Common::replaceWithJump): * assembler/MacroAssemblerX86_64.h: (JSC::MacroAssemblerX86_64::readCallTarget): (JSC::MacroAssemblerX86_64::startOfBranchPtrWithPatchOnRegister): (JSC::MacroAssemblerX86_64::startOfBranch32WithPatchOnRegister): (JSC::MacroAssemblerX86_64::startOfPatchableBranchPtrWithPatchOnAddress): (JSC::MacroAssemblerX86_64::startOfPatchableBranch32WithPatchOnAddress): (JSC::MacroAssemblerX86_64::revertJumpReplacementToPatchableBranchPtrWithPatch): (JSC::MacroAssemblerX86_64::revertJumpReplacementToPatchableBranch32WithPatch): (JSC::MacroAssemblerX86_64::revertJumpReplacementToBranchPtrWithPatch): (JSC::MacroAssemblerX86_64::repatchCall): (JSC::MacroAssemblerX86_64::linkCall): * assembler/testmasm.cpp: (JSC::compile): (JSC::invoke): (JSC::testProbeModifiesProgramCounter): * b3/B3Compilation.cpp: (JSC::B3::Compilation::Compilation): * b3/B3Compilation.h: (JSC::B3::Compilation::code const): (JSC::B3::Compilation::codeRef const): * b3/B3Compile.cpp: (JSC::B3::compile): * b3/B3LowerMacros.cpp: * b3/air/AirDisassembler.cpp: (JSC::B3::Air::Disassembler::dump): * b3/air/testair.cpp: * b3/testb3.cpp: (JSC::B3::invoke): (JSC::B3::testInterpreter): (JSC::B3::testEntrySwitchSimple): (JSC::B3::testEntrySwitchNoEntrySwitch): (JSC::B3::testEntrySwitchWithCommonPaths): (JSC::B3::testEntrySwitchWithCommonPathsAndNonTrivialEntrypoint): (JSC::B3::testEntrySwitchLoop): * bytecode/AccessCase.cpp: (JSC::AccessCase::generateImpl): * bytecode/AccessCaseSnippetParams.cpp: (JSC::SlowPathCallGeneratorWithArguments::generateImpl): * bytecode/ByValInfo.h: (JSC::ByValInfo::ByValInfo): * bytecode/CallLinkInfo.cpp: (JSC::CallLinkInfo::callReturnLocation): (JSC::CallLinkInfo::patchableJump): (JSC::CallLinkInfo::hotPathBegin): (JSC::CallLinkInfo::slowPathStart): * bytecode/CallLinkInfo.h: (JSC::CallLinkInfo::setCallLocations): (JSC::CallLinkInfo::hotPathOther): * bytecode/CodeBlock.cpp: (JSC::CodeBlock::finishCreation): * bytecode/GetByIdStatus.cpp: (JSC::GetByIdStatus::computeForStubInfoWithoutExitSiteFeedback): * bytecode/GetByIdVariant.cpp: (JSC::GetByIdVariant::GetByIdVariant): (JSC::GetByIdVariant::dumpInContext const): * bytecode/GetByIdVariant.h: (JSC::GetByIdVariant::customAccessorGetter const): * bytecode/GetterSetterAccessCase.cpp: (JSC::GetterSetterAccessCase::create): (JSC::GetterSetterAccessCase::GetterSetterAccessCase): (JSC::GetterSetterAccessCase::dumpImpl const): * bytecode/GetterSetterAccessCase.h: (JSC::GetterSetterAccessCase::customAccessor const): (): Deleted. * bytecode/HandlerInfo.h: (JSC::HandlerInfo::initialize): * bytecode/InlineAccess.cpp: (JSC::linkCodeInline): (JSC::InlineAccess::rewireStubAsJump): * bytecode/InlineAccess.h: * bytecode/JumpTable.h: (JSC::StringJumpTable::ctiForValue): (JSC::SimpleJumpTable::ctiForValue): * bytecode/LLIntCallLinkInfo.h: (JSC::LLIntCallLinkInfo::unlink): * bytecode/PolymorphicAccess.cpp: (JSC::AccessGenerationState::emitExplicitExceptionHandler): (JSC::PolymorphicAccess::regenerate): * bytecode/PolymorphicAccess.h: (JSC::AccessGenerationResult::AccessGenerationResult): (JSC::AccessGenerationResult::code const): * bytecode/StructureStubInfo.h: (JSC::StructureStubInfo::slowPathCallLocation): (JSC::StructureStubInfo::doneLocation): (JSC::StructureStubInfo::slowPathStartLocation): (JSC::StructureStubInfo::patchableJumpForIn): * dfg/DFGCommonData.h: (JSC::DFG::CommonData::appendCatchEntrypoint): * dfg/DFGDisassembler.cpp: (JSC::DFG::Disassembler::dumpDisassembly): * dfg/DFGDriver.h: * dfg/DFGJITCompiler.cpp: (JSC::DFG::JITCompiler::linkOSRExits): (JSC::DFG::JITCompiler::compileExceptionHandlers): (JSC::DFG::JITCompiler::link): (JSC::DFG::JITCompiler::compileFunction): (JSC::DFG::JITCompiler::noticeCatchEntrypoint): * dfg/DFGJITCompiler.h: (JSC::DFG::CallLinkRecord::CallLinkRecord): (JSC::DFG::JITCompiler::appendCall): (JSC::DFG::JITCompiler::JSCallRecord::JSCallRecord): (JSC::DFG::JITCompiler::JSDirectCallRecord::JSDirectCallRecord): (JSC::DFG::JITCompiler::JSDirectTailCallRecord::JSDirectTailCallRecord): * dfg/DFGJITFinalizer.cpp: (JSC::DFG::JITFinalizer::JITFinalizer): (JSC::DFG::JITFinalizer::finalize): (JSC::DFG::JITFinalizer::finalizeFunction): * dfg/DFGJITFinalizer.h: * dfg/DFGJumpReplacement.h: (JSC::DFG::JumpReplacement::JumpReplacement): * dfg/DFGNode.h: * dfg/DFGOSREntry.cpp: (JSC::DFG::prepareOSREntry): (JSC::DFG::prepareCatchOSREntry): * dfg/DFGOSREntry.h: (JSC::DFG::prepareOSREntry): * dfg/DFGOSRExit.cpp: (JSC::DFG::OSRExit::executeOSRExit): (JSC::DFG::reifyInlinedCallFrames): (JSC::DFG::adjustAndJumpToTarget): (JSC::DFG::OSRExit::codeLocationForRepatch const): (JSC::DFG::OSRExit::emitRestoreArguments): (JSC::DFG::OSRExit::compileOSRExit): * dfg/DFGOSRExit.h: * dfg/DFGOSRExitCompilerCommon.cpp: (JSC::DFG::handleExitCounts): (JSC::DFG::reifyInlinedCallFrames): (JSC::DFG::osrWriteBarrier): (JSC::DFG::adjustAndJumpToTarget): * dfg/DFGOperations.cpp: * dfg/DFGSlowPathGenerator.h: (JSC::DFG::CallResultAndArgumentsSlowPathGenerator::CallResultAndArgumentsSlowPathGenerator): (JSC::DFG::CallResultAndArgumentsSlowPathGenerator::unpackAndGenerate): (JSC::DFG::slowPathCall): * dfg/DFGSpeculativeJIT.cpp: (JSC::DFG::SpeculativeJIT::compileMathIC): (JSC::DFG::SpeculativeJIT::compileCallDOM): (JSC::DFG::SpeculativeJIT::compileCallDOMGetter): (JSC::DFG::SpeculativeJIT::emitSwitchIntJump): (JSC::DFG::SpeculativeJIT::emitSwitchImm): (JSC::DFG::SpeculativeJIT::emitSwitchStringOnString): (JSC::DFG::SpeculativeJIT::compileHasIndexedProperty): (JSC::DFG::SpeculativeJIT::compileGetDirectPname): (JSC::DFG::SpeculativeJIT::cachedPutById): * dfg/DFGSpeculativeJIT.h: (JSC::DFG::SpeculativeJIT::callOperation): (JSC::DFG::SpeculativeJIT::appendCall): (JSC::DFG::SpeculativeJIT::appendCallWithCallFrameRollbackOnException): (JSC::DFG::SpeculativeJIT::appendCallWithCallFrameRollbackOnExceptionSetResult): (JSC::DFG::SpeculativeJIT::appendCallSetResult): * dfg/DFGSpeculativeJIT64.cpp: (JSC::DFG::SpeculativeJIT::cachedGetById): (JSC::DFG::SpeculativeJIT::cachedGetByIdWithThis): (JSC::DFG::SpeculativeJIT::compile): * dfg/DFGThunks.cpp: (JSC::DFG::osrExitThunkGenerator): (JSC::DFG::osrExitGenerationThunkGenerator): (JSC::DFG::osrEntryThunkGenerator): * dfg/DFGThunks.h: * disassembler/ARM64Disassembler.cpp: (JSC::tryToDisassemble): * disassembler/ARMv7Disassembler.cpp: (JSC::tryToDisassemble): * disassembler/Disassembler.cpp: (JSC::disassemble): (JSC::disassembleAsynchronously): * disassembler/Disassembler.h: (JSC::tryToDisassemble): * disassembler/UDis86Disassembler.cpp: (JSC::tryToDisassembleWithUDis86): * disassembler/UDis86Disassembler.h: (JSC::tryToDisassembleWithUDis86): * disassembler/X86Disassembler.cpp: (JSC::tryToDisassemble): * ftl/FTLCompile.cpp: (JSC::FTL::compile): * ftl/FTLExceptionTarget.cpp: (JSC::FTL::ExceptionTarget::label): (JSC::FTL::ExceptionTarget::jumps): * ftl/FTLExceptionTarget.h: * ftl/FTLGeneratedFunction.h: * ftl/FTLJITCode.cpp: (JSC::FTL::JITCode::initializeB3Code): (JSC::FTL::JITCode::initializeAddressForCall): (JSC::FTL::JITCode::initializeArityCheckEntrypoint): (JSC::FTL::JITCode::addressForCall): (JSC::FTL::JITCode::executableAddressAtOffset): * ftl/FTLJITCode.h: (JSC::FTL::JITCode::b3Code const): * ftl/FTLJITFinalizer.cpp: (JSC::FTL::JITFinalizer::finalizeCommon): * ftl/FTLLazySlowPath.cpp: (JSC::FTL::LazySlowPath::initialize): (JSC::FTL::LazySlowPath::generate): * ftl/FTLLazySlowPath.h: (JSC::FTL::LazySlowPath::patchableJump const): (JSC::FTL::LazySlowPath::done const): (JSC::FTL::LazySlowPath::stub const): * ftl/FTLLazySlowPathCall.h: (JSC::FTL::createLazyCallGenerator): * ftl/FTLLink.cpp: (JSC::FTL::link): * ftl/FTLLowerDFGToB3.cpp: (JSC::FTL::DFG::LowerDFGToB3::lower): (JSC::FTL::DFG::LowerDFGToB3::compileCallOrConstruct): (JSC::FTL::DFG::LowerDFGToB3::compileDirectCallOrConstruct): (JSC::FTL::DFG::LowerDFGToB3::compileTailCall): (JSC::FTL::DFG::LowerDFGToB3::compileCallOrConstructVarargsSpread): (JSC::FTL::DFG::LowerDFGToB3::compileCallOrConstructVarargs): (JSC::FTL::DFG::LowerDFGToB3::compileCallEval): (JSC::FTL::DFG::LowerDFGToB3::compileInvalidationPoint): (JSC::FTL::DFG::LowerDFGToB3::compileIn): (JSC::FTL::DFG::LowerDFGToB3::compileCheckSubClass): (JSC::FTL::DFG::LowerDFGToB3::compileCallDOM): (JSC::FTL::DFG::LowerDFGToB3::compileCallDOMGetter): (JSC::FTL::DFG::LowerDFGToB3::lazySlowPath): * ftl/FTLOSRExit.cpp: (JSC::FTL::OSRExit::codeLocationForRepatch const): * ftl/FTLOSRExit.h: * ftl/FTLOSRExitCompiler.cpp: (JSC::FTL::compileStub): (JSC::FTL::compileFTLOSRExit): * ftl/FTLOSRExitHandle.cpp: (JSC::FTL::OSRExitHandle::emitExitThunk): * ftl/FTLOperations.cpp: (JSC::FTL::compileFTLLazySlowPath): * ftl/FTLPatchpointExceptionHandle.cpp: (JSC::FTL::PatchpointExceptionHandle::scheduleExitCreationForUnwind): * ftl/FTLSlowPathCall.cpp: (JSC::FTL::SlowPathCallContext::keyWithTarget const): (JSC::FTL::SlowPathCallContext::makeCall): * ftl/FTLSlowPathCall.h: (JSC::FTL::callOperation): * ftl/FTLSlowPathCallKey.cpp: (JSC::FTL::SlowPathCallKey::dump const): * ftl/FTLSlowPathCallKey.h: (JSC::FTL::SlowPathCallKey::SlowPathCallKey): (JSC::FTL::SlowPathCallKey::callTarget const): (JSC::FTL::SlowPathCallKey::withCallTarget): (JSC::FTL::SlowPathCallKey::hash const): (JSC::FTL::SlowPathCallKey::callPtrTag const): Deleted. * ftl/FTLState.cpp: (JSC::FTL::State::State): * ftl/FTLThunks.cpp: (JSC::FTL::genericGenerationThunkGenerator): (JSC::FTL::osrExitGenerationThunkGenerator): (JSC::FTL::lazySlowPathGenerationThunkGenerator): (JSC::FTL::slowPathCallThunkGenerator): * ftl/FTLThunks.h: (JSC::FTL::generateIfNecessary): (JSC::FTL::keyForThunk): (JSC::FTL::Thunks::getSlowPathCallThunk): (JSC::FTL::Thunks::keyForSlowPathCallThunk): * interpreter/InterpreterInlines.h: (JSC::Interpreter::getOpcodeID): * jit/AssemblyHelpers.cpp: (JSC::AssemblyHelpers::callExceptionFuzz): (JSC::AssemblyHelpers::emitDumbVirtualCall): (JSC::AssemblyHelpers::debugCall): * jit/CCallHelpers.cpp: (JSC::CCallHelpers::ensureShadowChickenPacket): * jit/ExecutableAllocator.cpp: (JSC::FixedVMPoolExecutableAllocator::initializeSeparatedWXHeaps): (JSC::FixedVMPoolExecutableAllocator::jitWriteThunkGenerator): * jit/ExecutableAllocator.h: (JSC::performJITMemcpy): * jit/GCAwareJITStubRoutine.cpp: (JSC::GCAwareJITStubRoutine::GCAwareJITStubRoutine): (JSC::MarkingGCAwareJITStubRoutine::MarkingGCAwareJITStubRoutine): (JSC::GCAwareJITStubRoutineWithExceptionHandler::GCAwareJITStubRoutineWithExceptionHandler): (JSC::createJITStubRoutine): * jit/GCAwareJITStubRoutine.h: (JSC::createJITStubRoutine): * jit/JIT.cpp: (JSC::ctiPatchCallByReturnAddress): (JSC::JIT::compileWithoutLinking): (JSC::JIT::link): (JSC::JIT::privateCompileExceptionHandlers): * jit/JIT.h: (JSC::CallRecord::CallRecord): * jit/JITArithmetic.cpp: (JSC::JIT::emitMathICFast): (JSC::JIT::emitMathICSlow): * jit/JITCall.cpp: (JSC::JIT::compileOpCallSlowCase): * jit/JITCall32_64.cpp: (JSC::JIT::compileOpCallSlowCase): * jit/JITCode.cpp: (JSC::JITCodeWithCodeRef::JITCodeWithCodeRef): (JSC::JITCodeWithCodeRef::executableAddressAtOffset): (JSC::DirectJITCode::DirectJITCode): (JSC::DirectJITCode::initializeCodeRef): (JSC::DirectJITCode::addressForCall): (JSC::NativeJITCode::NativeJITCode): (JSC::NativeJITCode::initializeCodeRef): (JSC::NativeJITCode::addressForCall): * jit/JITCode.h: * jit/JITCodeMap.h: (JSC::JITCodeMap::Entry::Entry): (JSC::JITCodeMap::Entry::codeLocation): (JSC::JITCodeMap::append): (JSC::JITCodeMap::find const): * jit/JITDisassembler.cpp: (JSC::JITDisassembler::dumpDisassembly): * jit/JITExceptions.cpp: (JSC::genericUnwind): * jit/JITInlineCacheGenerator.cpp: (JSC::JITByIdGenerator::finalize): * jit/JITInlines.h: (JSC::JIT::emitNakedCall): (JSC::JIT::emitNakedTailCall): (JSC::JIT::appendCallWithExceptionCheck): (JSC::JIT::appendCallWithExceptionCheckAndSlowPathReturnType): (JSC::JIT::appendCallWithCallFrameRollbackOnException): (JSC::JIT::appendCallWithExceptionCheckSetJSValueResult): (JSC::JIT::appendCallWithExceptionCheckSetJSValueResultWithProfile): * jit/JITMathIC.h: (JSC::isProfileEmpty): * jit/JITOpcodes.cpp: (JSC::JIT::emit_op_catch): (JSC::JIT::emit_op_switch_imm): (JSC::JIT::emit_op_switch_char): (JSC::JIT::emit_op_switch_string): (JSC::JIT::privateCompileHasIndexedProperty): (JSC::JIT::emitSlow_op_has_indexed_property): * jit/JITOpcodes32_64.cpp: (JSC::JIT::privateCompileHasIndexedProperty): * jit/JITOperations.cpp: (JSC::getByVal): * jit/JITPropertyAccess.cpp: (JSC::JIT::stringGetByValStubGenerator): (JSC::JIT::emitGetByValWithCachedId): (JSC::JIT::emitSlow_op_get_by_val): (JSC::JIT::emitPutByValWithCachedId): (JSC::JIT::emitSlow_op_put_by_val): (JSC::JIT::emitSlow_op_try_get_by_id): (JSC::JIT::emitSlow_op_get_by_id_direct): (JSC::JIT::emitSlow_op_get_by_id): (JSC::JIT::emitSlow_op_get_by_id_with_this): (JSC::JIT::emitSlow_op_put_by_id): (JSC::JIT::privateCompileGetByVal): (JSC::JIT::privateCompileGetByValWithCachedId): (JSC::JIT::privateCompilePutByVal): (JSC::JIT::privateCompilePutByValWithCachedId): * jit/JITPropertyAccess32_64.cpp: (JSC::JIT::stringGetByValStubGenerator): (JSC::JIT::emitSlow_op_get_by_val): (JSC::JIT::emitSlow_op_put_by_val): * jit/JITStubRoutine.h: (JSC::JITStubRoutine::JITStubRoutine): (JSC::JITStubRoutine::createSelfManagedRoutine): (JSC::JITStubRoutine::code const): (JSC::JITStubRoutine::asCodePtr): * jit/JITThunks.cpp: (JSC::JITThunks::ctiNativeCall): (JSC::JITThunks::ctiNativeConstruct): (JSC::JITThunks::ctiNativeTailCall): (JSC::JITThunks::ctiNativeTailCallWithoutSavedTags): (JSC::JITThunks::ctiInternalFunctionCall): (JSC::JITThunks::ctiInternalFunctionConstruct): (JSC::JITThunks::ctiStub): (JSC::JITThunks::existingCTIStub): (JSC::JITThunks::hostFunctionStub): * jit/JITThunks.h: * jit/PCToCodeOriginMap.cpp: (JSC::PCToCodeOriginMap::PCToCodeOriginMap): * jit/PCToCodeOriginMap.h: * jit/PolymorphicCallStubRoutine.cpp: (JSC::PolymorphicCallStubRoutine::PolymorphicCallStubRoutine): * jit/PolymorphicCallStubRoutine.h: * jit/Repatch.cpp: (JSC::readPutICCallTarget): (JSC::ftlThunkAwareRepatchCall): (JSC::appropriateOptimizingGetByIdFunction): (JSC::appropriateGetByIdFunction): (JSC::tryCacheGetByID): (JSC::repatchGetByID): (JSC::tryCachePutByID): (JSC::repatchPutByID): (JSC::tryCacheIn): (JSC::repatchIn): (JSC::linkSlowFor): (JSC::linkFor): (JSC::linkDirectFor): (JSC::revertCall): (JSC::unlinkFor): (JSC::linkVirtualFor): (JSC::linkPolymorphicCall): (JSC::resetGetByID): (JSC::resetPutByID): * jit/Repatch.h: * jit/SlowPathCall.h: (JSC::JITSlowPathCall::call): * jit/SpecializedThunkJIT.h: (JSC::SpecializedThunkJIT::finalize): (JSC::SpecializedThunkJIT::callDoubleToDouble): (JSC::SpecializedThunkJIT::callDoubleToDoublePreservingReturn): * jit/ThunkGenerator.h: * jit/ThunkGenerators.cpp: (JSC::throwExceptionFromCallSlowPathGenerator): (JSC::slowPathFor): (JSC::linkCallThunkGenerator): (JSC::linkPolymorphicCallThunkGenerator): (JSC::virtualThunkFor): (JSC::nativeForGenerator): (JSC::nativeCallGenerator): (JSC::nativeTailCallGenerator): (JSC::nativeTailCallWithoutSavedTagsGenerator): (JSC::nativeConstructGenerator): (JSC::internalFunctionCallGenerator): (JSC::internalFunctionConstructGenerator): (JSC::arityFixupGenerator): (JSC::unreachableGenerator): (JSC::charCodeAtThunkGenerator): (JSC::charAtThunkGenerator): (JSC::fromCharCodeThunkGenerator): (JSC::clz32ThunkGenerator): (JSC::sqrtThunkGenerator): (JSC::floorThunkGenerator): (JSC::ceilThunkGenerator): (JSC::truncThunkGenerator): (JSC::roundThunkGenerator): (JSC::expThunkGenerator): (JSC::logThunkGenerator): (JSC::absThunkGenerator): (JSC::imulThunkGenerator): (JSC::randomThunkGenerator): (JSC::boundThisNoArgsFunctionCallGenerator): * jit/ThunkGenerators.h: * llint/LLIntData.cpp: (JSC::LLInt::initialize): * llint/LLIntData.h: (JSC::LLInt::getExecutableAddress): (JSC::LLInt::getCodePtr): (JSC::LLInt::getCodeRef): (JSC::LLInt::getCodeFunctionPtr): * llint/LLIntEntrypoint.cpp: (JSC::LLInt::setFunctionEntrypoint): (JSC::LLInt::setEvalEntrypoint): (JSC::LLInt::setProgramEntrypoint): (JSC::LLInt::setModuleProgramEntrypoint): * llint/LLIntExceptions.cpp: (JSC::LLInt::callToThrow): * llint/LLIntSlowPaths.cpp: (JSC::LLInt::LLINT_SLOW_PATH_DECL): (JSC::LLInt::setUpCall): * llint/LLIntThunks.cpp: (JSC::vmEntryToWasm): (JSC::LLInt::generateThunkWithJumpTo): (JSC::LLInt::functionForCallEntryThunkGenerator): (JSC::LLInt::functionForConstructEntryThunkGenerator): (JSC::LLInt::functionForCallArityCheckThunkGenerator): (JSC::LLInt::functionForConstructArityCheckThunkGenerator): (JSC::LLInt::evalEntryThunkGenerator): (JSC::LLInt::programEntryThunkGenerator): (JSC::LLInt::moduleProgramEntryThunkGenerator): * llint/LLIntThunks.h: * llint/LowLevelInterpreter.asm: * llint/LowLevelInterpreter32_64.asm: * llint/LowLevelInterpreter64.asm: * profiler/ProfilerCompilation.cpp: (JSC::Profiler::Compilation::addOSRExitSite): * profiler/ProfilerCompilation.h: * profiler/ProfilerOSRExitSite.cpp: (JSC::Profiler::OSRExitSite::toJS const): * profiler/ProfilerOSRExitSite.h: (JSC::Profiler::OSRExitSite::OSRExitSite): (JSC::Profiler::OSRExitSite::codeAddress const): (JSC::Profiler::OSRExitSite:: const): Deleted. * runtime/ExecutableBase.cpp: (JSC::ExecutableBase::clearCode): * runtime/ExecutableBase.h: (JSC::ExecutableBase::entrypointFor): * runtime/NativeExecutable.cpp: (JSC::NativeExecutable::finishCreation): * runtime/NativeFunction.h: (JSC::TaggedNativeFunction::TaggedNativeFunction): (JSC::TaggedNativeFunction::operator NativeFunction): * runtime/PtrTag.h: (JSC::tagCodePtr): (JSC::untagCodePtr): (JSC::retagCodePtr): (JSC::tagCFunctionPtr): (JSC::untagCFunctionPtr): (JSC::nextPtrTagID): Deleted. * runtime/PutPropertySlot.h: (JSC::PutPropertySlot::PutPropertySlot): (JSC::PutPropertySlot::setCustomValue): (JSC::PutPropertySlot::setCustomAccessor): (JSC::PutPropertySlot::customSetter const): * runtime/ScriptExecutable.cpp: (JSC::ScriptExecutable::installCode): * runtime/VM.cpp: (JSC::VM::getHostFunction): (JSC::VM::getCTIInternalFunctionTrampolineFor): * runtime/VM.h: (JSC::VM::getCTIStub): * wasm/WasmB3IRGenerator.cpp: (JSC::Wasm::B3IRGenerator::B3IRGenerator): (JSC::Wasm::B3IRGenerator::emitExceptionCheck): (JSC::Wasm::B3IRGenerator::emitTierUpCheck): (JSC::Wasm::B3IRGenerator::addCall): (JSC::Wasm::B3IRGenerator::addCallIndirect): * wasm/WasmBBQPlan.cpp: (JSC::Wasm::BBQPlan::prepare): (JSC::Wasm::BBQPlan::complete): * wasm/WasmBBQPlan.h: * wasm/WasmBinding.cpp: (JSC::Wasm::wasmToWasm): * wasm/WasmBinding.h: * wasm/WasmCallee.h: (JSC::Wasm::Callee::entrypoint const): * wasm/WasmCallingConvention.h: (JSC::Wasm::CallingConvention::setupFrameInPrologue const): * wasm/WasmCodeBlock.h: (JSC::Wasm::CodeBlock::entrypointLoadLocationFromFunctionIndexSpace): * wasm/WasmFaultSignalHandler.cpp: (JSC::Wasm::trapHandler): * wasm/WasmFormat.h: * wasm/WasmInstance.h: * wasm/WasmOMGPlan.cpp: (JSC::Wasm::OMGPlan::work): * wasm/WasmThunks.cpp: (JSC::Wasm::throwExceptionFromWasmThunkGenerator): (JSC::Wasm::throwStackOverflowFromWasmThunkGenerator): (JSC::Wasm::triggerOMGTierUpThunkGenerator): (JSC::Wasm::Thunks::stub): (JSC::Wasm::Thunks::existingStub): * wasm/WasmThunks.h: * wasm/js/JSToWasm.cpp: (JSC::Wasm::createJSToWasmWrapper): * wasm/js/JSWebAssemblyCodeBlock.h: * wasm/js/WasmToJS.cpp: (JSC::Wasm::handleBadI64Use): (JSC::Wasm::wasmToJS): * wasm/js/WasmToJS.h: * wasm/js/WebAssemblyFunction.h: * yarr/YarrJIT.cpp: (JSC::Yarr::YarrGenerator::loadFromFrameAndJump): (JSC::Yarr::YarrGenerator::BacktrackingState::linkDataLabels): (JSC::Yarr::YarrGenerator::compile): * yarr/YarrJIT.h: (JSC::Yarr::YarrCodeBlock::set8BitCode): (JSC::Yarr::YarrCodeBlock::set16BitCode): (JSC::Yarr::YarrCodeBlock::set8BitCodeMatchOnly): (JSC::Yarr::YarrCodeBlock::set16BitCodeMatchOnly): (JSC::Yarr::YarrCodeBlock::execute): (JSC::Yarr::YarrCodeBlock::clear): Source/WebCore: No new tests. This is covered by existing tests. * WebCore.xcodeproj/project.pbxproj: * css/ElementRuleCollector.cpp: (WebCore::ElementRuleCollector::ruleMatches): * cssjit/CSSPtrTag.h: Added. * cssjit/CompiledSelector.h: * cssjit/FunctionCall.h: (WebCore::FunctionCall::FunctionCall): (WebCore::FunctionCall::setFunctionAddress): (WebCore::FunctionCall::prepareAndCall): * cssjit/SelectorCompiler.cpp: (WebCore::SelectorCompiler::compileSelector): (WebCore::SelectorCompiler::SelectorFragment::appendUnoptimizedPseudoClassWithContext): (WebCore::SelectorCompiler::addPseudoClassType): (WebCore::SelectorCompiler::SelectorCodeGenerator::compile): (WebCore::SelectorCompiler::SelectorCodeGenerator::generateElementAttributeFunctionCallValueMatching): (WebCore::SelectorCompiler::SelectorCodeGenerator::generateElementFunctionCallTest): (WebCore::SelectorCompiler::SelectorCodeGenerator::generateContextFunctionCallTest): * cssjit/SelectorCompiler.h: (WebCore::SelectorCompiler::ruleCollectorSimpleSelectorCheckerFunction): (WebCore::SelectorCompiler::querySelectorSimpleSelectorCheckerFunction): (WebCore::SelectorCompiler::ruleCollectorSelectorCheckerFunctionWithCheckingContext): (WebCore::SelectorCompiler::querySelectorSelectorCheckerFunctionWithCheckingContext): * dom/SelectorQuery.cpp: (WebCore::SelectorDataList::executeCompiledSingleMultiSelectorData const): (WebCore::SelectorDataList::execute const): * dom/SelectorQuery.h: Canonical link: https://commits.webkit.org/200234@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@230748 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2018-04-18 03:31:09 +00:00
SlowPathCall SlowPathCallContext::makeCall(VM& vm, FunctionPtr<CFunctionPtrTag> callTarget)
FTL should be able to do some simple inline caches using LLVM patchpoints https://bugs.webkit.org/show_bug.cgi?id=123164 Source/JavaScriptCore: Reviewed by Mark Hahnenberg. This implements GetById inline caches in the FTL using llvm.webkit.patchpoint. The idea is that we ask LLVM for a nop slide the size of a GetById inline cache and then fill in the code after LLVM compilation is complete. For now, we just use the system calling convention for the arguments and return. We also still make some assumptions about registers that aren't correct. But, most of the scaffolding is there and this will successfully patch an inline cache. * JavaScriptCore.xcodeproj/project.pbxproj: * assembler/AbstractMacroAssembler.h: * assembler/LinkBuffer.cpp: (JSC::LinkBuffer::finalizeCodeWithoutDisassembly): (JSC::LinkBuffer::linkCode): (JSC::LinkBuffer::allocate): * assembler/LinkBuffer.h: (JSC::LinkBuffer::LinkBuffer): (JSC::LinkBuffer::link): * ftl/FTLAbbreviations.h: (JSC::FTL::constNull): (JSC::FTL::buildCall): * ftl/FTLCapabilities.cpp: (JSC::FTL::canCompile): * ftl/FTLCompile.cpp: (JSC::FTL::fixFunctionBasedOnStackMaps): * ftl/FTLInlineCacheDescriptor.h: Added. (JSC::FTL::InlineCacheDescriptor::InlineCacheDescriptor): (JSC::FTL::GetByIdDescriptor::GetByIdDescriptor): (JSC::FTL::GetByIdDescriptor::stackmapID): (JSC::FTL::GetByIdDescriptor::codeOrigin): (JSC::FTL::GetByIdDescriptor::uid): * ftl/FTLInlineCacheSize.cpp: Added. (JSC::FTL::sizeOfGetById): (JSC::FTL::sizeOfPutById): * ftl/FTLInlineCacheSize.h: Added. * ftl/FTLIntrinsicRepository.h: * ftl/FTLJITFinalizer.cpp: (JSC::FTL::JITFinalizer::finalizeFunction): * ftl/FTLJITFinalizer.h: * ftl/FTLLocation.cpp: (JSC::FTL::Location::directGPR): * ftl/FTLLocation.h: * ftl/FTLLowerDFGToLLVM.cpp: (JSC::FTL::LowerDFGToLLVM::compileGetById): * ftl/FTLOutput.h: (JSC::FTL::Output::call): * ftl/FTLSlowPathCall.cpp: Added. (JSC::FTL::callOperation): * ftl/FTLSlowPathCall.h: Added. (JSC::FTL::SlowPathCall::SlowPathCall): (JSC::FTL::SlowPathCall::call): (JSC::FTL::SlowPathCall::key): * ftl/FTLSlowPathCallKey.cpp: Added. (JSC::FTL::SlowPathCallKey::dump): * ftl/FTLSlowPathCallKey.h: Added. (JSC::FTL::SlowPathCallKey::SlowPathCallKey): (JSC::FTL::SlowPathCallKey::usedRegisters): (JSC::FTL::SlowPathCallKey::callTarget): (JSC::FTL::SlowPathCallKey::offset): (JSC::FTL::SlowPathCallKey::isEmptyValue): (JSC::FTL::SlowPathCallKey::isDeletedValue): (JSC::FTL::SlowPathCallKey::operator==): (JSC::FTL::SlowPathCallKey::hash): (JSC::FTL::SlowPathCallKeyHash::hash): (JSC::FTL::SlowPathCallKeyHash::equal): * ftl/FTLStackMaps.cpp: (JSC::FTL::StackMaps::Location::directGPR): * ftl/FTLStackMaps.h: * ftl/FTLState.h: * ftl/FTLThunks.cpp: (JSC::FTL::slowPathCallThunkGenerator): * ftl/FTLThunks.h: (JSC::FTL::Thunks::getSlowPathCallThunk): * jit/CCallHelpers.h: (JSC::CCallHelpers::setupArguments): * jit/GPRInfo.h: * jit/JITInlineCacheGenerator.cpp: (JSC::garbageStubInfo): (JSC::JITInlineCacheGenerator::JITInlineCacheGenerator): (JSC::JITByIdGenerator::finalize): * jit/JITInlineCacheGenerator.h: (JSC::JITByIdGenerator::slowPathBegin): * jit/RegisterSet.cpp: (JSC::RegisterSet::stackRegisters): (JSC::RegisterSet::specialRegisters): (JSC::RegisterSet::calleeSaveRegisters): (JSC::RegisterSet::allGPRs): (JSC::RegisterSet::allFPRs): (JSC::RegisterSet::allRegisters): (JSC::RegisterSet::dump): * jit/RegisterSet.h: (JSC::RegisterSet::exclude): (JSC::RegisterSet::numberOfSetRegisters): (JSC::RegisterSet::RegisterSet): (JSC::RegisterSet::isEmptyValue): (JSC::RegisterSet::isDeletedValue): (JSC::RegisterSet::operator==): (JSC::RegisterSet::hash): (JSC::RegisterSetHash::hash): (JSC::RegisterSetHash::equal): * runtime/Options.h: Source/WTF: Reviewed by Mark Hahnenberg. This needed some better bitvector support, like merging (|=), excluding (&=~), hashing, and bit counting. * wtf/BitVector.cpp: (WTF::BitVector::setSlow): (WTF::BitVector::excludeSlow): (WTF::BitVector::bitCountSlow): (WTF::BitVector::equalsSlowCase): (WTF::BitVector::hashSlowCase): (WTF::BitVector::dump): * wtf/BitVector.h: (WTF::BitVector::merge): (WTF::BitVector::exclude): (WTF::BitVector::bitCount): (WTF::BitVector::BitVector): (WTF::BitVector::isEmptyValue): (WTF::BitVector::isDeletedValue): (WTF::BitVector::isEmptyOrDeletedValue): (WTF::BitVector::operator==): (WTF::BitVector::hash): (WTF::BitVectorHash::hash): (WTF::BitVectorHash::equal): * wtf/HashTraits.h: (WTF::CustomHashTraits::constructDeletedValue): (WTF::CustomHashTraits::isDeletedValue): (WTF::CustomHashTraits::emptyValue): (WTF::CustomHashTraits::isEmptyValue): * wtf/StdLibExtras.h: (WTF::bitCount): Canonical link: https://commits.webkit.org/141315@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@157872 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2013-10-23 18:22:09 +00:00
{
Templatize CodePtr/Refs/FunctionPtrs with PtrTags. https://bugs.webkit.org/show_bug.cgi?id=184702 <rdar://problem/35391681> Reviewed by Filip Pizlo and Saam Barati. Source/JavaScriptCore: 1. Templatized MacroAssemblerCodePtr/Ref, FunctionPtr, and CodeLocation variants to take a PtrTag template argument. 2. Replaced some uses of raw pointers with the equivalent CodePtr / FunctionPtr. * assembler/AbstractMacroAssembler.h: (JSC::AbstractMacroAssembler::differenceBetweenCodePtr): (JSC::AbstractMacroAssembler::linkJump): (JSC::AbstractMacroAssembler::linkPointer): (JSC::AbstractMacroAssembler::getLinkerAddress): (JSC::AbstractMacroAssembler::repatchJump): (JSC::AbstractMacroAssembler::repatchJumpToNop): (JSC::AbstractMacroAssembler::repatchNearCall): (JSC::AbstractMacroAssembler::repatchCompact): (JSC::AbstractMacroAssembler::repatchInt32): (JSC::AbstractMacroAssembler::repatchPointer): (JSC::AbstractMacroAssembler::readPointer): (JSC::AbstractMacroAssembler::replaceWithLoad): (JSC::AbstractMacroAssembler::replaceWithAddressComputation): * assembler/CodeLocation.h: (JSC::CodeLocationCommon:: const): (JSC::CodeLocationCommon::CodeLocationCommon): (JSC::CodeLocationInstruction::CodeLocationInstruction): (JSC::CodeLocationLabel::CodeLocationLabel): (JSC::CodeLocationLabel::retagged): (JSC::CodeLocationLabel:: const): (JSC::CodeLocationJump::CodeLocationJump): (JSC::CodeLocationJump::retagged): (JSC::CodeLocationCall::CodeLocationCall): (JSC::CodeLocationCall::retagged): (JSC::CodeLocationNearCall::CodeLocationNearCall): (JSC::CodeLocationDataLabel32::CodeLocationDataLabel32): (JSC::CodeLocationDataLabelCompact::CodeLocationDataLabelCompact): (JSC::CodeLocationDataLabelPtr::CodeLocationDataLabelPtr): (JSC::CodeLocationConvertibleLoad::CodeLocationConvertibleLoad): (JSC::CodeLocationCommon<tag>::instructionAtOffset): (JSC::CodeLocationCommon<tag>::labelAtOffset): (JSC::CodeLocationCommon<tag>::jumpAtOffset): (JSC::CodeLocationCommon<tag>::callAtOffset): (JSC::CodeLocationCommon<tag>::nearCallAtOffset): (JSC::CodeLocationCommon<tag>::dataLabelPtrAtOffset): (JSC::CodeLocationCommon<tag>::dataLabel32AtOffset): (JSC::CodeLocationCommon<tag>::dataLabelCompactAtOffset): (JSC::CodeLocationCommon<tag>::convertibleLoadAtOffset): (JSC::CodeLocationCommon::instructionAtOffset): Deleted. (JSC::CodeLocationCommon::labelAtOffset): Deleted. (JSC::CodeLocationCommon::jumpAtOffset): Deleted. (JSC::CodeLocationCommon::callAtOffset): Deleted. (JSC::CodeLocationCommon::nearCallAtOffset): Deleted. (JSC::CodeLocationCommon::dataLabelPtrAtOffset): Deleted. (JSC::CodeLocationCommon::dataLabel32AtOffset): Deleted. (JSC::CodeLocationCommon::dataLabelCompactAtOffset): Deleted. (JSC::CodeLocationCommon::convertibleLoadAtOffset): Deleted. * assembler/LinkBuffer.cpp: (JSC::LinkBuffer::finalizeCodeWithoutDisassemblyImpl): (JSC::LinkBuffer::finalizeCodeWithDisassemblyImpl): (JSC::LinkBuffer::finalizeCodeWithoutDisassembly): Deleted. (JSC::LinkBuffer::finalizeCodeWithDisassembly): Deleted. * assembler/LinkBuffer.h: (JSC::LinkBuffer::link): (JSC::LinkBuffer::patch): (JSC::LinkBuffer::entrypoint): (JSC::LinkBuffer::locationOf): (JSC::LinkBuffer::locationOfNearCall): (JSC::LinkBuffer::finalizeCodeWithoutDisassembly): (JSC::LinkBuffer::finalizeCodeWithDisassembly): (JSC::LinkBuffer::trampolineAt): * assembler/MacroAssemblerARM.h: (JSC::MacroAssemblerARM::readCallTarget): (JSC::MacroAssemblerARM::replaceWithJump): (JSC::MacroAssemblerARM::startOfPatchableBranch32WithPatchOnAddress): (JSC::MacroAssemblerARM::startOfPatchableBranchPtrWithPatchOnAddress): (JSC::MacroAssemblerARM::startOfBranchPtrWithPatchOnRegister): (JSC::MacroAssemblerARM::revertJumpReplacementToBranchPtrWithPatch): (JSC::MacroAssemblerARM::revertJumpReplacementToPatchableBranch32WithPatch): (JSC::MacroAssemblerARM::revertJumpReplacementToPatchableBranchPtrWithPatch): (JSC::MacroAssemblerARM::repatchCall): (JSC::MacroAssemblerARM::linkCall): * assembler/MacroAssemblerARM64.h: (JSC::MacroAssemblerARM64::readCallTarget): (JSC::MacroAssemblerARM64::replaceWithVMHalt): (JSC::MacroAssemblerARM64::replaceWithJump): (JSC::MacroAssemblerARM64::startOfBranchPtrWithPatchOnRegister): (JSC::MacroAssemblerARM64::startOfPatchableBranchPtrWithPatchOnAddress): (JSC::MacroAssemblerARM64::startOfPatchableBranch32WithPatchOnAddress): (JSC::MacroAssemblerARM64::revertJumpReplacementToBranchPtrWithPatch): (JSC::MacroAssemblerARM64::revertJumpReplacementToPatchableBranchPtrWithPatch): (JSC::MacroAssemblerARM64::revertJumpReplacementToPatchableBranch32WithPatch): (JSC::MacroAssemblerARM64::repatchCall): (JSC::MacroAssemblerARM64::linkCall): * assembler/MacroAssemblerARMv7.h: (JSC::MacroAssemblerARMv7::replaceWithJump): (JSC::MacroAssemblerARMv7::readCallTarget): (JSC::MacroAssemblerARMv7::startOfBranchPtrWithPatchOnRegister): (JSC::MacroAssemblerARMv7::revertJumpReplacementToBranchPtrWithPatch): (JSC::MacroAssemblerARMv7::startOfPatchableBranchPtrWithPatchOnAddress): (JSC::MacroAssemblerARMv7::startOfPatchableBranch32WithPatchOnAddress): (JSC::MacroAssemblerARMv7::revertJumpReplacementToPatchableBranchPtrWithPatch): (JSC::MacroAssemblerARMv7::revertJumpReplacementToPatchableBranch32WithPatch): (JSC::MacroAssemblerARMv7::repatchCall): (JSC::MacroAssemblerARMv7::linkCall): * assembler/MacroAssemblerCodeRef.cpp: (JSC::MacroAssemblerCodePtrBase::dumpWithName): (JSC::MacroAssemblerCodeRefBase::tryToDisassemble): (JSC::MacroAssemblerCodeRefBase::disassembly): (JSC::MacroAssemblerCodePtr::createLLIntCodePtr): Deleted. (JSC::MacroAssemblerCodePtr::dumpWithName const): Deleted. (JSC::MacroAssemblerCodePtr::dump const): Deleted. (JSC::MacroAssemblerCodeRef::createLLIntCodeRef): Deleted. (JSC::MacroAssemblerCodeRef::tryToDisassemble const): Deleted. (JSC::MacroAssemblerCodeRef::disassembly const): Deleted. (JSC::MacroAssemblerCodeRef::dump const): Deleted. * assembler/MacroAssemblerCodeRef.h: (JSC::FunctionPtr::FunctionPtr): (JSC::FunctionPtr::retagged const): (JSC::FunctionPtr::retaggedExecutableAddress const): (JSC::FunctionPtr::operator== const): (JSC::FunctionPtr::operator!= const): (JSC::ReturnAddressPtr::ReturnAddressPtr): (JSC::MacroAssemblerCodePtr::MacroAssemblerCodePtr): (JSC::MacroAssemblerCodePtr::createFromExecutableAddress): (JSC::MacroAssemblerCodePtr::retagged const): (JSC::MacroAssemblerCodePtr:: const): (JSC::MacroAssemblerCodePtr::dumpWithName const): (JSC::MacroAssemblerCodePtr::dump const): (JSC::MacroAssemblerCodePtrHash::hash): (JSC::MacroAssemblerCodePtrHash::equal): (JSC::MacroAssemblerCodeRef::MacroAssemblerCodeRef): (JSC::MacroAssemblerCodeRef::createSelfManagedCodeRef): (JSC::MacroAssemblerCodeRef::code const): (JSC::MacroAssemblerCodeRef::retaggedCode const): (JSC::MacroAssemblerCodeRef::retagged const): (JSC::MacroAssemblerCodeRef::tryToDisassemble const): (JSC::MacroAssemblerCodeRef::disassembly const): (JSC::MacroAssemblerCodeRef::dump const): (JSC::FunctionPtr<tag>::FunctionPtr): * assembler/MacroAssemblerMIPS.h: (JSC::MacroAssemblerMIPS::readCallTarget): (JSC::MacroAssemblerMIPS::replaceWithJump): (JSC::MacroAssemblerMIPS::startOfPatchableBranch32WithPatchOnAddress): (JSC::MacroAssemblerMIPS::startOfBranchPtrWithPatchOnRegister): (JSC::MacroAssemblerMIPS::revertJumpReplacementToBranchPtrWithPatch): (JSC::MacroAssemblerMIPS::startOfPatchableBranchPtrWithPatchOnAddress): (JSC::MacroAssemblerMIPS::revertJumpReplacementToPatchableBranch32WithPatch): (JSC::MacroAssemblerMIPS::revertJumpReplacementToPatchableBranchPtrWithPatch): (JSC::MacroAssemblerMIPS::repatchCall): (JSC::MacroAssemblerMIPS::linkCall): * assembler/MacroAssemblerX86.h: (JSC::MacroAssemblerX86::readCallTarget): (JSC::MacroAssemblerX86::startOfBranchPtrWithPatchOnRegister): (JSC::MacroAssemblerX86::startOfPatchableBranchPtrWithPatchOnAddress): (JSC::MacroAssemblerX86::startOfPatchableBranch32WithPatchOnAddress): (JSC::MacroAssemblerX86::revertJumpReplacementToBranchPtrWithPatch): (JSC::MacroAssemblerX86::revertJumpReplacementToPatchableBranchPtrWithPatch): (JSC::MacroAssemblerX86::revertJumpReplacementToPatchableBranch32WithPatch): (JSC::MacroAssemblerX86::repatchCall): (JSC::MacroAssemblerX86::linkCall): * assembler/MacroAssemblerX86Common.h: (JSC::MacroAssemblerX86Common::repatchCompact): (JSC::MacroAssemblerX86Common::replaceWithVMHalt): (JSC::MacroAssemblerX86Common::replaceWithJump): * assembler/MacroAssemblerX86_64.h: (JSC::MacroAssemblerX86_64::readCallTarget): (JSC::MacroAssemblerX86_64::startOfBranchPtrWithPatchOnRegister): (JSC::MacroAssemblerX86_64::startOfBranch32WithPatchOnRegister): (JSC::MacroAssemblerX86_64::startOfPatchableBranchPtrWithPatchOnAddress): (JSC::MacroAssemblerX86_64::startOfPatchableBranch32WithPatchOnAddress): (JSC::MacroAssemblerX86_64::revertJumpReplacementToPatchableBranchPtrWithPatch): (JSC::MacroAssemblerX86_64::revertJumpReplacementToPatchableBranch32WithPatch): (JSC::MacroAssemblerX86_64::revertJumpReplacementToBranchPtrWithPatch): (JSC::MacroAssemblerX86_64::repatchCall): (JSC::MacroAssemblerX86_64::linkCall): * assembler/testmasm.cpp: (JSC::compile): (JSC::invoke): (JSC::testProbeModifiesProgramCounter): * b3/B3Compilation.cpp: (JSC::B3::Compilation::Compilation): * b3/B3Compilation.h: (JSC::B3::Compilation::code const): (JSC::B3::Compilation::codeRef const): * b3/B3Compile.cpp: (JSC::B3::compile): * b3/B3LowerMacros.cpp: * b3/air/AirDisassembler.cpp: (JSC::B3::Air::Disassembler::dump): * b3/air/testair.cpp: * b3/testb3.cpp: (JSC::B3::invoke): (JSC::B3::testInterpreter): (JSC::B3::testEntrySwitchSimple): (JSC::B3::testEntrySwitchNoEntrySwitch): (JSC::B3::testEntrySwitchWithCommonPaths): (JSC::B3::testEntrySwitchWithCommonPathsAndNonTrivialEntrypoint): (JSC::B3::testEntrySwitchLoop): * bytecode/AccessCase.cpp: (JSC::AccessCase::generateImpl): * bytecode/AccessCaseSnippetParams.cpp: (JSC::SlowPathCallGeneratorWithArguments::generateImpl): * bytecode/ByValInfo.h: (JSC::ByValInfo::ByValInfo): * bytecode/CallLinkInfo.cpp: (JSC::CallLinkInfo::callReturnLocation): (JSC::CallLinkInfo::patchableJump): (JSC::CallLinkInfo::hotPathBegin): (JSC::CallLinkInfo::slowPathStart): * bytecode/CallLinkInfo.h: (JSC::CallLinkInfo::setCallLocations): (JSC::CallLinkInfo::hotPathOther): * bytecode/CodeBlock.cpp: (JSC::CodeBlock::finishCreation): * bytecode/GetByIdStatus.cpp: (JSC::GetByIdStatus::computeForStubInfoWithoutExitSiteFeedback): * bytecode/GetByIdVariant.cpp: (JSC::GetByIdVariant::GetByIdVariant): (JSC::GetByIdVariant::dumpInContext const): * bytecode/GetByIdVariant.h: (JSC::GetByIdVariant::customAccessorGetter const): * bytecode/GetterSetterAccessCase.cpp: (JSC::GetterSetterAccessCase::create): (JSC::GetterSetterAccessCase::GetterSetterAccessCase): (JSC::GetterSetterAccessCase::dumpImpl const): * bytecode/GetterSetterAccessCase.h: (JSC::GetterSetterAccessCase::customAccessor const): (): Deleted. * bytecode/HandlerInfo.h: (JSC::HandlerInfo::initialize): * bytecode/InlineAccess.cpp: (JSC::linkCodeInline): (JSC::InlineAccess::rewireStubAsJump): * bytecode/InlineAccess.h: * bytecode/JumpTable.h: (JSC::StringJumpTable::ctiForValue): (JSC::SimpleJumpTable::ctiForValue): * bytecode/LLIntCallLinkInfo.h: (JSC::LLIntCallLinkInfo::unlink): * bytecode/PolymorphicAccess.cpp: (JSC::AccessGenerationState::emitExplicitExceptionHandler): (JSC::PolymorphicAccess::regenerate): * bytecode/PolymorphicAccess.h: (JSC::AccessGenerationResult::AccessGenerationResult): (JSC::AccessGenerationResult::code const): * bytecode/StructureStubInfo.h: (JSC::StructureStubInfo::slowPathCallLocation): (JSC::StructureStubInfo::doneLocation): (JSC::StructureStubInfo::slowPathStartLocation): (JSC::StructureStubInfo::patchableJumpForIn): * dfg/DFGCommonData.h: (JSC::DFG::CommonData::appendCatchEntrypoint): * dfg/DFGDisassembler.cpp: (JSC::DFG::Disassembler::dumpDisassembly): * dfg/DFGDriver.h: * dfg/DFGJITCompiler.cpp: (JSC::DFG::JITCompiler::linkOSRExits): (JSC::DFG::JITCompiler::compileExceptionHandlers): (JSC::DFG::JITCompiler::link): (JSC::DFG::JITCompiler::compileFunction): (JSC::DFG::JITCompiler::noticeCatchEntrypoint): * dfg/DFGJITCompiler.h: (JSC::DFG::CallLinkRecord::CallLinkRecord): (JSC::DFG::JITCompiler::appendCall): (JSC::DFG::JITCompiler::JSCallRecord::JSCallRecord): (JSC::DFG::JITCompiler::JSDirectCallRecord::JSDirectCallRecord): (JSC::DFG::JITCompiler::JSDirectTailCallRecord::JSDirectTailCallRecord): * dfg/DFGJITFinalizer.cpp: (JSC::DFG::JITFinalizer::JITFinalizer): (JSC::DFG::JITFinalizer::finalize): (JSC::DFG::JITFinalizer::finalizeFunction): * dfg/DFGJITFinalizer.h: * dfg/DFGJumpReplacement.h: (JSC::DFG::JumpReplacement::JumpReplacement): * dfg/DFGNode.h: * dfg/DFGOSREntry.cpp: (JSC::DFG::prepareOSREntry): (JSC::DFG::prepareCatchOSREntry): * dfg/DFGOSREntry.h: (JSC::DFG::prepareOSREntry): * dfg/DFGOSRExit.cpp: (JSC::DFG::OSRExit::executeOSRExit): (JSC::DFG::reifyInlinedCallFrames): (JSC::DFG::adjustAndJumpToTarget): (JSC::DFG::OSRExit::codeLocationForRepatch const): (JSC::DFG::OSRExit::emitRestoreArguments): (JSC::DFG::OSRExit::compileOSRExit): * dfg/DFGOSRExit.h: * dfg/DFGOSRExitCompilerCommon.cpp: (JSC::DFG::handleExitCounts): (JSC::DFG::reifyInlinedCallFrames): (JSC::DFG::osrWriteBarrier): (JSC::DFG::adjustAndJumpToTarget): * dfg/DFGOperations.cpp: * dfg/DFGSlowPathGenerator.h: (JSC::DFG::CallResultAndArgumentsSlowPathGenerator::CallResultAndArgumentsSlowPathGenerator): (JSC::DFG::CallResultAndArgumentsSlowPathGenerator::unpackAndGenerate): (JSC::DFG::slowPathCall): * dfg/DFGSpeculativeJIT.cpp: (JSC::DFG::SpeculativeJIT::compileMathIC): (JSC::DFG::SpeculativeJIT::compileCallDOM): (JSC::DFG::SpeculativeJIT::compileCallDOMGetter): (JSC::DFG::SpeculativeJIT::emitSwitchIntJump): (JSC::DFG::SpeculativeJIT::emitSwitchImm): (JSC::DFG::SpeculativeJIT::emitSwitchStringOnString): (JSC::DFG::SpeculativeJIT::compileHasIndexedProperty): (JSC::DFG::SpeculativeJIT::compileGetDirectPname): (JSC::DFG::SpeculativeJIT::cachedPutById): * dfg/DFGSpeculativeJIT.h: (JSC::DFG::SpeculativeJIT::callOperation): (JSC::DFG::SpeculativeJIT::appendCall): (JSC::DFG::SpeculativeJIT::appendCallWithCallFrameRollbackOnException): (JSC::DFG::SpeculativeJIT::appendCallWithCallFrameRollbackOnExceptionSetResult): (JSC::DFG::SpeculativeJIT::appendCallSetResult): * dfg/DFGSpeculativeJIT64.cpp: (JSC::DFG::SpeculativeJIT::cachedGetById): (JSC::DFG::SpeculativeJIT::cachedGetByIdWithThis): (JSC::DFG::SpeculativeJIT::compile): * dfg/DFGThunks.cpp: (JSC::DFG::osrExitThunkGenerator): (JSC::DFG::osrExitGenerationThunkGenerator): (JSC::DFG::osrEntryThunkGenerator): * dfg/DFGThunks.h: * disassembler/ARM64Disassembler.cpp: (JSC::tryToDisassemble): * disassembler/ARMv7Disassembler.cpp: (JSC::tryToDisassemble): * disassembler/Disassembler.cpp: (JSC::disassemble): (JSC::disassembleAsynchronously): * disassembler/Disassembler.h: (JSC::tryToDisassemble): * disassembler/UDis86Disassembler.cpp: (JSC::tryToDisassembleWithUDis86): * disassembler/UDis86Disassembler.h: (JSC::tryToDisassembleWithUDis86): * disassembler/X86Disassembler.cpp: (JSC::tryToDisassemble): * ftl/FTLCompile.cpp: (JSC::FTL::compile): * ftl/FTLExceptionTarget.cpp: (JSC::FTL::ExceptionTarget::label): (JSC::FTL::ExceptionTarget::jumps): * ftl/FTLExceptionTarget.h: * ftl/FTLGeneratedFunction.h: * ftl/FTLJITCode.cpp: (JSC::FTL::JITCode::initializeB3Code): (JSC::FTL::JITCode::initializeAddressForCall): (JSC::FTL::JITCode::initializeArityCheckEntrypoint): (JSC::FTL::JITCode::addressForCall): (JSC::FTL::JITCode::executableAddressAtOffset): * ftl/FTLJITCode.h: (JSC::FTL::JITCode::b3Code const): * ftl/FTLJITFinalizer.cpp: (JSC::FTL::JITFinalizer::finalizeCommon): * ftl/FTLLazySlowPath.cpp: (JSC::FTL::LazySlowPath::initialize): (JSC::FTL::LazySlowPath::generate): * ftl/FTLLazySlowPath.h: (JSC::FTL::LazySlowPath::patchableJump const): (JSC::FTL::LazySlowPath::done const): (JSC::FTL::LazySlowPath::stub const): * ftl/FTLLazySlowPathCall.h: (JSC::FTL::createLazyCallGenerator): * ftl/FTLLink.cpp: (JSC::FTL::link): * ftl/FTLLowerDFGToB3.cpp: (JSC::FTL::DFG::LowerDFGToB3::lower): (JSC::FTL::DFG::LowerDFGToB3::compileCallOrConstruct): (JSC::FTL::DFG::LowerDFGToB3::compileDirectCallOrConstruct): (JSC::FTL::DFG::LowerDFGToB3::compileTailCall): (JSC::FTL::DFG::LowerDFGToB3::compileCallOrConstructVarargsSpread): (JSC::FTL::DFG::LowerDFGToB3::compileCallOrConstructVarargs): (JSC::FTL::DFG::LowerDFGToB3::compileCallEval): (JSC::FTL::DFG::LowerDFGToB3::compileInvalidationPoint): (JSC::FTL::DFG::LowerDFGToB3::compileIn): (JSC::FTL::DFG::LowerDFGToB3::compileCheckSubClass): (JSC::FTL::DFG::LowerDFGToB3::compileCallDOM): (JSC::FTL::DFG::LowerDFGToB3::compileCallDOMGetter): (JSC::FTL::DFG::LowerDFGToB3::lazySlowPath): * ftl/FTLOSRExit.cpp: (JSC::FTL::OSRExit::codeLocationForRepatch const): * ftl/FTLOSRExit.h: * ftl/FTLOSRExitCompiler.cpp: (JSC::FTL::compileStub): (JSC::FTL::compileFTLOSRExit): * ftl/FTLOSRExitHandle.cpp: (JSC::FTL::OSRExitHandle::emitExitThunk): * ftl/FTLOperations.cpp: (JSC::FTL::compileFTLLazySlowPath): * ftl/FTLPatchpointExceptionHandle.cpp: (JSC::FTL::PatchpointExceptionHandle::scheduleExitCreationForUnwind): * ftl/FTLSlowPathCall.cpp: (JSC::FTL::SlowPathCallContext::keyWithTarget const): (JSC::FTL::SlowPathCallContext::makeCall): * ftl/FTLSlowPathCall.h: (JSC::FTL::callOperation): * ftl/FTLSlowPathCallKey.cpp: (JSC::FTL::SlowPathCallKey::dump const): * ftl/FTLSlowPathCallKey.h: (JSC::FTL::SlowPathCallKey::SlowPathCallKey): (JSC::FTL::SlowPathCallKey::callTarget const): (JSC::FTL::SlowPathCallKey::withCallTarget): (JSC::FTL::SlowPathCallKey::hash const): (JSC::FTL::SlowPathCallKey::callPtrTag const): Deleted. * ftl/FTLState.cpp: (JSC::FTL::State::State): * ftl/FTLThunks.cpp: (JSC::FTL::genericGenerationThunkGenerator): (JSC::FTL::osrExitGenerationThunkGenerator): (JSC::FTL::lazySlowPathGenerationThunkGenerator): (JSC::FTL::slowPathCallThunkGenerator): * ftl/FTLThunks.h: (JSC::FTL::generateIfNecessary): (JSC::FTL::keyForThunk): (JSC::FTL::Thunks::getSlowPathCallThunk): (JSC::FTL::Thunks::keyForSlowPathCallThunk): * interpreter/InterpreterInlines.h: (JSC::Interpreter::getOpcodeID): * jit/AssemblyHelpers.cpp: (JSC::AssemblyHelpers::callExceptionFuzz): (JSC::AssemblyHelpers::emitDumbVirtualCall): (JSC::AssemblyHelpers::debugCall): * jit/CCallHelpers.cpp: (JSC::CCallHelpers::ensureShadowChickenPacket): * jit/ExecutableAllocator.cpp: (JSC::FixedVMPoolExecutableAllocator::initializeSeparatedWXHeaps): (JSC::FixedVMPoolExecutableAllocator::jitWriteThunkGenerator): * jit/ExecutableAllocator.h: (JSC::performJITMemcpy): * jit/GCAwareJITStubRoutine.cpp: (JSC::GCAwareJITStubRoutine::GCAwareJITStubRoutine): (JSC::MarkingGCAwareJITStubRoutine::MarkingGCAwareJITStubRoutine): (JSC::GCAwareJITStubRoutineWithExceptionHandler::GCAwareJITStubRoutineWithExceptionHandler): (JSC::createJITStubRoutine): * jit/GCAwareJITStubRoutine.h: (JSC::createJITStubRoutine): * jit/JIT.cpp: (JSC::ctiPatchCallByReturnAddress): (JSC::JIT::compileWithoutLinking): (JSC::JIT::link): (JSC::JIT::privateCompileExceptionHandlers): * jit/JIT.h: (JSC::CallRecord::CallRecord): * jit/JITArithmetic.cpp: (JSC::JIT::emitMathICFast): (JSC::JIT::emitMathICSlow): * jit/JITCall.cpp: (JSC::JIT::compileOpCallSlowCase): * jit/JITCall32_64.cpp: (JSC::JIT::compileOpCallSlowCase): * jit/JITCode.cpp: (JSC::JITCodeWithCodeRef::JITCodeWithCodeRef): (JSC::JITCodeWithCodeRef::executableAddressAtOffset): (JSC::DirectJITCode::DirectJITCode): (JSC::DirectJITCode::initializeCodeRef): (JSC::DirectJITCode::addressForCall): (JSC::NativeJITCode::NativeJITCode): (JSC::NativeJITCode::initializeCodeRef): (JSC::NativeJITCode::addressForCall): * jit/JITCode.h: * jit/JITCodeMap.h: (JSC::JITCodeMap::Entry::Entry): (JSC::JITCodeMap::Entry::codeLocation): (JSC::JITCodeMap::append): (JSC::JITCodeMap::find const): * jit/JITDisassembler.cpp: (JSC::JITDisassembler::dumpDisassembly): * jit/JITExceptions.cpp: (JSC::genericUnwind): * jit/JITInlineCacheGenerator.cpp: (JSC::JITByIdGenerator::finalize): * jit/JITInlines.h: (JSC::JIT::emitNakedCall): (JSC::JIT::emitNakedTailCall): (JSC::JIT::appendCallWithExceptionCheck): (JSC::JIT::appendCallWithExceptionCheckAndSlowPathReturnType): (JSC::JIT::appendCallWithCallFrameRollbackOnException): (JSC::JIT::appendCallWithExceptionCheckSetJSValueResult): (JSC::JIT::appendCallWithExceptionCheckSetJSValueResultWithProfile): * jit/JITMathIC.h: (JSC::isProfileEmpty): * jit/JITOpcodes.cpp: (JSC::JIT::emit_op_catch): (JSC::JIT::emit_op_switch_imm): (JSC::JIT::emit_op_switch_char): (JSC::JIT::emit_op_switch_string): (JSC::JIT::privateCompileHasIndexedProperty): (JSC::JIT::emitSlow_op_has_indexed_property): * jit/JITOpcodes32_64.cpp: (JSC::JIT::privateCompileHasIndexedProperty): * jit/JITOperations.cpp: (JSC::getByVal): * jit/JITPropertyAccess.cpp: (JSC::JIT::stringGetByValStubGenerator): (JSC::JIT::emitGetByValWithCachedId): (JSC::JIT::emitSlow_op_get_by_val): (JSC::JIT::emitPutByValWithCachedId): (JSC::JIT::emitSlow_op_put_by_val): (JSC::JIT::emitSlow_op_try_get_by_id): (JSC::JIT::emitSlow_op_get_by_id_direct): (JSC::JIT::emitSlow_op_get_by_id): (JSC::JIT::emitSlow_op_get_by_id_with_this): (JSC::JIT::emitSlow_op_put_by_id): (JSC::JIT::privateCompileGetByVal): (JSC::JIT::privateCompileGetByValWithCachedId): (JSC::JIT::privateCompilePutByVal): (JSC::JIT::privateCompilePutByValWithCachedId): * jit/JITPropertyAccess32_64.cpp: (JSC::JIT::stringGetByValStubGenerator): (JSC::JIT::emitSlow_op_get_by_val): (JSC::JIT::emitSlow_op_put_by_val): * jit/JITStubRoutine.h: (JSC::JITStubRoutine::JITStubRoutine): (JSC::JITStubRoutine::createSelfManagedRoutine): (JSC::JITStubRoutine::code const): (JSC::JITStubRoutine::asCodePtr): * jit/JITThunks.cpp: (JSC::JITThunks::ctiNativeCall): (JSC::JITThunks::ctiNativeConstruct): (JSC::JITThunks::ctiNativeTailCall): (JSC::JITThunks::ctiNativeTailCallWithoutSavedTags): (JSC::JITThunks::ctiInternalFunctionCall): (JSC::JITThunks::ctiInternalFunctionConstruct): (JSC::JITThunks::ctiStub): (JSC::JITThunks::existingCTIStub): (JSC::JITThunks::hostFunctionStub): * jit/JITThunks.h: * jit/PCToCodeOriginMap.cpp: (JSC::PCToCodeOriginMap::PCToCodeOriginMap): * jit/PCToCodeOriginMap.h: * jit/PolymorphicCallStubRoutine.cpp: (JSC::PolymorphicCallStubRoutine::PolymorphicCallStubRoutine): * jit/PolymorphicCallStubRoutine.h: * jit/Repatch.cpp: (JSC::readPutICCallTarget): (JSC::ftlThunkAwareRepatchCall): (JSC::appropriateOptimizingGetByIdFunction): (JSC::appropriateGetByIdFunction): (JSC::tryCacheGetByID): (JSC::repatchGetByID): (JSC::tryCachePutByID): (JSC::repatchPutByID): (JSC::tryCacheIn): (JSC::repatchIn): (JSC::linkSlowFor): (JSC::linkFor): (JSC::linkDirectFor): (JSC::revertCall): (JSC::unlinkFor): (JSC::linkVirtualFor): (JSC::linkPolymorphicCall): (JSC::resetGetByID): (JSC::resetPutByID): * jit/Repatch.h: * jit/SlowPathCall.h: (JSC::JITSlowPathCall::call): * jit/SpecializedThunkJIT.h: (JSC::SpecializedThunkJIT::finalize): (JSC::SpecializedThunkJIT::callDoubleToDouble): (JSC::SpecializedThunkJIT::callDoubleToDoublePreservingReturn): * jit/ThunkGenerator.h: * jit/ThunkGenerators.cpp: (JSC::throwExceptionFromCallSlowPathGenerator): (JSC::slowPathFor): (JSC::linkCallThunkGenerator): (JSC::linkPolymorphicCallThunkGenerator): (JSC::virtualThunkFor): (JSC::nativeForGenerator): (JSC::nativeCallGenerator): (JSC::nativeTailCallGenerator): (JSC::nativeTailCallWithoutSavedTagsGenerator): (JSC::nativeConstructGenerator): (JSC::internalFunctionCallGenerator): (JSC::internalFunctionConstructGenerator): (JSC::arityFixupGenerator): (JSC::unreachableGenerator): (JSC::charCodeAtThunkGenerator): (JSC::charAtThunkGenerator): (JSC::fromCharCodeThunkGenerator): (JSC::clz32ThunkGenerator): (JSC::sqrtThunkGenerator): (JSC::floorThunkGenerator): (JSC::ceilThunkGenerator): (JSC::truncThunkGenerator): (JSC::roundThunkGenerator): (JSC::expThunkGenerator): (JSC::logThunkGenerator): (JSC::absThunkGenerator): (JSC::imulThunkGenerator): (JSC::randomThunkGenerator): (JSC::boundThisNoArgsFunctionCallGenerator): * jit/ThunkGenerators.h: * llint/LLIntData.cpp: (JSC::LLInt::initialize): * llint/LLIntData.h: (JSC::LLInt::getExecutableAddress): (JSC::LLInt::getCodePtr): (JSC::LLInt::getCodeRef): (JSC::LLInt::getCodeFunctionPtr): * llint/LLIntEntrypoint.cpp: (JSC::LLInt::setFunctionEntrypoint): (JSC::LLInt::setEvalEntrypoint): (JSC::LLInt::setProgramEntrypoint): (JSC::LLInt::setModuleProgramEntrypoint): * llint/LLIntExceptions.cpp: (JSC::LLInt::callToThrow): * llint/LLIntSlowPaths.cpp: (JSC::LLInt::LLINT_SLOW_PATH_DECL): (JSC::LLInt::setUpCall): * llint/LLIntThunks.cpp: (JSC::vmEntryToWasm): (JSC::LLInt::generateThunkWithJumpTo): (JSC::LLInt::functionForCallEntryThunkGenerator): (JSC::LLInt::functionForConstructEntryThunkGenerator): (JSC::LLInt::functionForCallArityCheckThunkGenerator): (JSC::LLInt::functionForConstructArityCheckThunkGenerator): (JSC::LLInt::evalEntryThunkGenerator): (JSC::LLInt::programEntryThunkGenerator): (JSC::LLInt::moduleProgramEntryThunkGenerator): * llint/LLIntThunks.h: * llint/LowLevelInterpreter.asm: * llint/LowLevelInterpreter32_64.asm: * llint/LowLevelInterpreter64.asm: * profiler/ProfilerCompilation.cpp: (JSC::Profiler::Compilation::addOSRExitSite): * profiler/ProfilerCompilation.h: * profiler/ProfilerOSRExitSite.cpp: (JSC::Profiler::OSRExitSite::toJS const): * profiler/ProfilerOSRExitSite.h: (JSC::Profiler::OSRExitSite::OSRExitSite): (JSC::Profiler::OSRExitSite::codeAddress const): (JSC::Profiler::OSRExitSite:: const): Deleted. * runtime/ExecutableBase.cpp: (JSC::ExecutableBase::clearCode): * runtime/ExecutableBase.h: (JSC::ExecutableBase::entrypointFor): * runtime/NativeExecutable.cpp: (JSC::NativeExecutable::finishCreation): * runtime/NativeFunction.h: (JSC::TaggedNativeFunction::TaggedNativeFunction): (JSC::TaggedNativeFunction::operator NativeFunction): * runtime/PtrTag.h: (JSC::tagCodePtr): (JSC::untagCodePtr): (JSC::retagCodePtr): (JSC::tagCFunctionPtr): (JSC::untagCFunctionPtr): (JSC::nextPtrTagID): Deleted. * runtime/PutPropertySlot.h: (JSC::PutPropertySlot::PutPropertySlot): (JSC::PutPropertySlot::setCustomValue): (JSC::PutPropertySlot::setCustomAccessor): (JSC::PutPropertySlot::customSetter const): * runtime/ScriptExecutable.cpp: (JSC::ScriptExecutable::installCode): * runtime/VM.cpp: (JSC::VM::getHostFunction): (JSC::VM::getCTIInternalFunctionTrampolineFor): * runtime/VM.h: (JSC::VM::getCTIStub): * wasm/WasmB3IRGenerator.cpp: (JSC::Wasm::B3IRGenerator::B3IRGenerator): (JSC::Wasm::B3IRGenerator::emitExceptionCheck): (JSC::Wasm::B3IRGenerator::emitTierUpCheck): (JSC::Wasm::B3IRGenerator::addCall): (JSC::Wasm::B3IRGenerator::addCallIndirect): * wasm/WasmBBQPlan.cpp: (JSC::Wasm::BBQPlan::prepare): (JSC::Wasm::BBQPlan::complete): * wasm/WasmBBQPlan.h: * wasm/WasmBinding.cpp: (JSC::Wasm::wasmToWasm): * wasm/WasmBinding.h: * wasm/WasmCallee.h: (JSC::Wasm::Callee::entrypoint const): * wasm/WasmCallingConvention.h: (JSC::Wasm::CallingConvention::setupFrameInPrologue const): * wasm/WasmCodeBlock.h: (JSC::Wasm::CodeBlock::entrypointLoadLocationFromFunctionIndexSpace): * wasm/WasmFaultSignalHandler.cpp: (JSC::Wasm::trapHandler): * wasm/WasmFormat.h: * wasm/WasmInstance.h: * wasm/WasmOMGPlan.cpp: (JSC::Wasm::OMGPlan::work): * wasm/WasmThunks.cpp: (JSC::Wasm::throwExceptionFromWasmThunkGenerator): (JSC::Wasm::throwStackOverflowFromWasmThunkGenerator): (JSC::Wasm::triggerOMGTierUpThunkGenerator): (JSC::Wasm::Thunks::stub): (JSC::Wasm::Thunks::existingStub): * wasm/WasmThunks.h: * wasm/js/JSToWasm.cpp: (JSC::Wasm::createJSToWasmWrapper): * wasm/js/JSWebAssemblyCodeBlock.h: * wasm/js/WasmToJS.cpp: (JSC::Wasm::handleBadI64Use): (JSC::Wasm::wasmToJS): * wasm/js/WasmToJS.h: * wasm/js/WebAssemblyFunction.h: * yarr/YarrJIT.cpp: (JSC::Yarr::YarrGenerator::loadFromFrameAndJump): (JSC::Yarr::YarrGenerator::BacktrackingState::linkDataLabels): (JSC::Yarr::YarrGenerator::compile): * yarr/YarrJIT.h: (JSC::Yarr::YarrCodeBlock::set8BitCode): (JSC::Yarr::YarrCodeBlock::set16BitCode): (JSC::Yarr::YarrCodeBlock::set8BitCodeMatchOnly): (JSC::Yarr::YarrCodeBlock::set16BitCodeMatchOnly): (JSC::Yarr::YarrCodeBlock::execute): (JSC::Yarr::YarrCodeBlock::clear): Source/WebCore: No new tests. This is covered by existing tests. * WebCore.xcodeproj/project.pbxproj: * css/ElementRuleCollector.cpp: (WebCore::ElementRuleCollector::ruleMatches): * cssjit/CSSPtrTag.h: Added. * cssjit/CompiledSelector.h: * cssjit/FunctionCall.h: (WebCore::FunctionCall::FunctionCall): (WebCore::FunctionCall::setFunctionAddress): (WebCore::FunctionCall::prepareAndCall): * cssjit/SelectorCompiler.cpp: (WebCore::SelectorCompiler::compileSelector): (WebCore::SelectorCompiler::SelectorFragment::appendUnoptimizedPseudoClassWithContext): (WebCore::SelectorCompiler::addPseudoClassType): (WebCore::SelectorCompiler::SelectorCodeGenerator::compile): (WebCore::SelectorCompiler::SelectorCodeGenerator::generateElementAttributeFunctionCallValueMatching): (WebCore::SelectorCompiler::SelectorCodeGenerator::generateElementFunctionCallTest): (WebCore::SelectorCompiler::SelectorCodeGenerator::generateContextFunctionCallTest): * cssjit/SelectorCompiler.h: (WebCore::SelectorCompiler::ruleCollectorSimpleSelectorCheckerFunction): (WebCore::SelectorCompiler::querySelectorSimpleSelectorCheckerFunction): (WebCore::SelectorCompiler::ruleCollectorSelectorCheckerFunctionWithCheckingContext): (WebCore::SelectorCompiler::querySelectorSelectorCheckerFunctionWithCheckingContext): * dom/SelectorQuery.cpp: (WebCore::SelectorDataList::executeCompiledSingleMultiSelectorData const): (WebCore::SelectorDataList::execute const): * dom/SelectorQuery.h: Canonical link: https://commits.webkit.org/200234@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@230748 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2018-04-18 03:31:09 +00:00
SlowPathCallKey key = keyWithTarget(callTarget);
Unreviewed, reland r268170 https://bugs.webkit.org/show_bug.cgi?id=217460 Source/JavaScriptCore: Fixed missing wrong OperationPtrTag use in Repatch.cpp. * assembler/AbstractMacroAssembler.h: (JSC::AbstractMacroAssembler::getLinkerAddress): * assembler/AssemblerBuffer.h: (JSC::ARM64EHash::update): (JSC::ARM64EHash::finalHash const): * assembler/JITOperationList.cpp: (JSC::addPointers): * assembler/MacroAssemblerARM64.cpp: (JSC::MacroAssembler::probe): * assembler/MacroAssemblerCodeRef.h: (JSC::MacroAssemblerCodePtr::MacroAssemblerCodePtr): (JSC::MacroAssemblerCodePtr::createFromExecutableAddress): * assembler/testmasm.cpp: (JSC::testProbeModifiesProgramCounter): * b3/air/testair.cpp: * ftl/FTLOutput.h: (JSC::FTL::Output::callWithoutSideEffects): (JSC::FTL::Output::operation): * ftl/FTLSlowPathCall.cpp: (JSC::FTL::SlowPathCallContext::makeCall): * jit/JITCode.cpp: (JSC::JITCodeWithCodeRef::executableAddressAtOffset): * jit/JITExceptions.cpp: (JSC::genericUnwind): * jit/JITOperations.cpp: * jit/Repatch.cpp: (JSC::readPutICCallTarget): (JSC::ftlThunkAwareRepatchCall): (JSC::tryCacheGetBy): (JSC::tryCachePutByID): * llint/LLIntData.cpp: (JSC::LLInt::initialize): * llint/LLIntPCRanges.h: (JSC::LLInt::isLLIntPC): * llint/LLIntSlowPaths.cpp: (JSC::LLInt::setUpCall): * llint/LLIntThunks.cpp: (JSC::LLInt::generateThunkWithJumpTo): * runtime/JSCPtrTag.h: * runtime/MachineContext.h: (JSC::MachineContext::instructionPointer): * runtime/NativeExecutable.cpp: (JSC::NativeExecutable::finishCreation): * runtime/PutPropertySlot.h: (JSC::PutPropertySlot::setCustomValue): (JSC::PutPropertySlot::setCustomAccessor): (JSC::PutPropertySlot::customSetter const): * wasm/WasmAirIRGenerator.cpp: (JSC::Wasm::AirIRGenerator::emitCCall): * wasm/WasmSlowPaths.cpp: Source/WTF: * wtf/PlatformRegisters.cpp: (WTF::threadStateLRInternal): (WTF::threadStatePCInternal): * wtf/PtrTag.h: (WTF::tagCodePtr): (WTF::untagCodePtr): (WTF::assertIsCFunctionPtr): (WTF::assertIsNullOrCFunctionPtr): (WTF::assertIsNotTagged): (WTF::assertIsTagged): (WTF::assertIsNullOrTagged): (WTF::isTaggedWith): (WTF::assertIsTaggedWith): (WTF::assertIsNullOrTaggedWith): (WTF::tagCFunctionPtr): (WTF::tagCFunction): (WTF::untagCFunctionPtr): (WTF::tagInt): Canonical link: https://commits.webkit.org/230280@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@268247 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2020-10-09 05:08:11 +00:00
SlowPathCall result = SlowPathCall(m_jit.call(JITThunkPtrTag), key);
FTL should generate code to call slow paths lazily https://bugs.webkit.org/show_bug.cgi?id=149936 Reviewed by Saam Barati. Source/JavaScriptCore: We often have complex slow paths in FTL-generated code. Those slow paths may never run. Even if they do run, they don't need stellar performance. So, it doesn't make sense to have LLVM worry about compiling such slow path code. This patch enables us to use our own MacroAssembler for compiling the slow path inside FTL code. It does this by using a crazy lambda thingy (see FTLLowerDFGToLLVM.cpp's lazySlowPath() and its documentation). The result is quite natural to use. Even for straight slow path calls via something like vmCall(), the lazySlowPath offers the benefit that the call marshalling and the exception checking are not expressed using LLVM IR and do not require LLVM to think about it. It also has the benefit that we never generate the code if it never runs. That's great, since function calls usually involve ~10 instructions total (move arguments to argument registers, make the call, check exception, etc.). This patch adds the lazy slow path abstraction and uses it for some slow paths in the FTL. The code we generate with lazy slow paths is worse than the code that LLVM would have generated. Therefore, a lazy slow path only makes sense when we have strong evidence that the slow path will execute infrequently relative to the fast path. This completely precludes the use of lazy slow paths for out-of-line Nodes that unconditionally call a C++ function. It also precludes their use for the GetByVal out-of-bounds handler, since when we generate a GetByVal with an out-of-bounds handler it means that we only know that the out-of-bounds case executed at least once. So, for all we know, it may actually be the common case. So, this patch just deployed the lazy slow path for GC slow paths and masquerades-as-undefined slow paths. It makes sense for GC slow paths because those have a statistical guarantee of slow path frequency - probably bounded at less than 1/10. It makes sense for masquerades-as- undefined because we can say quite confidently that this is an uncommon scenario on the modern Web. Something that's always been challenging about abstractions involving the MacroAssembler is that linking is a separate phase, and there is no way for someone who is just given access to the MacroAssembler& to emit code that requires linking, since linking happens once we have emitted all code and we are creating the LinkBuffer. Moreover, the FTL requires that the final parts of linking happen on the main thread. This patch ran into this issue, and solved it comprehensively, by introducing MacroAssembler::addLinkTask(). This takes a lambda and runs it at the bitter end of linking - when performFinalization() is called. This ensure that the task added by addLinkTask() runs on the main thread. This patch doesn't replace all of the previously existing idioms for dealing with this issue; we can do that later. This shows small speed-ups on a bunch of things. No big win on any benchmark aggregate. But mainly this is done for https://bugs.webkit.org/show_bug.cgi?id=149852, where we found that outlining the slow path in this way was a significant speed boost. * CMakeLists.txt: * JavaScriptCore.vcxproj/JavaScriptCore.vcxproj: * JavaScriptCore.xcodeproj/project.pbxproj: * assembler/AbstractMacroAssembler.h: (JSC::AbstractMacroAssembler::replaceWithAddressComputation): (JSC::AbstractMacroAssembler::addLinkTask): (JSC::AbstractMacroAssembler::AbstractMacroAssembler): * assembler/LinkBuffer.cpp: (JSC::LinkBuffer::linkCode): (JSC::LinkBuffer::allocate): (JSC::LinkBuffer::performFinalization): * assembler/LinkBuffer.h: (JSC::LinkBuffer::wasAlreadyDisassembled): (JSC::LinkBuffer::didAlreadyDisassemble): (JSC::LinkBuffer::vm): (JSC::LinkBuffer::executableOffsetFor): * bytecode/CodeOrigin.h: (JSC::CodeOrigin::CodeOrigin): (JSC::CodeOrigin::isSet): (JSC::CodeOrigin::operator bool): (JSC::CodeOrigin::isHashTableDeletedValue): (JSC::CodeOrigin::operator!): Deleted. * ftl/FTLCompile.cpp: (JSC::FTL::mmAllocateDataSection): * ftl/FTLInlineCacheDescriptor.h: (JSC::FTL::InlineCacheDescriptor::InlineCacheDescriptor): (JSC::FTL::CheckInDescriptor::CheckInDescriptor): (JSC::FTL::LazySlowPathDescriptor::LazySlowPathDescriptor): * ftl/FTLJITCode.h: * ftl/FTLJITFinalizer.cpp: (JSC::FTL::JITFinalizer::finalizeFunction): * ftl/FTLJITFinalizer.h: * ftl/FTLLazySlowPath.cpp: Added. (JSC::FTL::LazySlowPath::LazySlowPath): (JSC::FTL::LazySlowPath::~LazySlowPath): (JSC::FTL::LazySlowPath::generate): * ftl/FTLLazySlowPath.h: Added. (JSC::FTL::LazySlowPath::createGenerator): (JSC::FTL::LazySlowPath::patchpoint): (JSC::FTL::LazySlowPath::usedRegisters): (JSC::FTL::LazySlowPath::callSiteIndex): (JSC::FTL::LazySlowPath::stub): * ftl/FTLLazySlowPathCall.h: Added. (JSC::FTL::createLazyCallGenerator): * ftl/FTLLowerDFGToLLVM.cpp: (JSC::FTL::DFG::LowerDFGToLLVM::compileCreateActivation): (JSC::FTL::DFG::LowerDFGToLLVM::compileNewFunction): (JSC::FTL::DFG::LowerDFGToLLVM::compileCreateDirectArguments): (JSC::FTL::DFG::LowerDFGToLLVM::compileNewArrayWithSize): (JSC::FTL::DFG::LowerDFGToLLVM::compileMakeRope): (JSC::FTL::DFG::LowerDFGToLLVM::compileNotifyWrite): (JSC::FTL::DFG::LowerDFGToLLVM::compileIsObjectOrNull): (JSC::FTL::DFG::LowerDFGToLLVM::compileIsFunction): (JSC::FTL::DFG::LowerDFGToLLVM::compileIn): (JSC::FTL::DFG::LowerDFGToLLVM::compileMaterializeNewObject): (JSC::FTL::DFG::LowerDFGToLLVM::compileMaterializeCreateActivation): (JSC::FTL::DFG::LowerDFGToLLVM::compileCheckWatchdogTimer): (JSC::FTL::DFG::LowerDFGToLLVM::allocatePropertyStorageWithSizeImpl): (JSC::FTL::DFG::LowerDFGToLLVM::allocateObject): (JSC::FTL::DFG::LowerDFGToLLVM::allocateJSArray): (JSC::FTL::DFG::LowerDFGToLLVM::buildTypeOf): (JSC::FTL::DFG::LowerDFGToLLVM::sensibleDoubleToInt32): (JSC::FTL::DFG::LowerDFGToLLVM::lazySlowPath): (JSC::FTL::DFG::LowerDFGToLLVM::speculate): (JSC::FTL::DFG::LowerDFGToLLVM::emitStoreBarrier): * ftl/FTLOperations.cpp: (JSC::FTL::operationMaterializeObjectInOSR): (JSC::FTL::compileFTLLazySlowPath): * ftl/FTLOperations.h: * ftl/FTLSlowPathCall.cpp: (JSC::FTL::SlowPathCallContext::SlowPathCallContext): (JSC::FTL::SlowPathCallContext::~SlowPathCallContext): (JSC::FTL::SlowPathCallContext::keyWithTarget): (JSC::FTL::SlowPathCallContext::makeCall): (JSC::FTL::callSiteIndexForCodeOrigin): (JSC::FTL::storeCodeOrigin): Deleted. (JSC::FTL::callOperation): Deleted. * ftl/FTLSlowPathCall.h: (JSC::FTL::callOperation): * ftl/FTLState.h: * ftl/FTLThunks.cpp: (JSC::FTL::genericGenerationThunkGenerator): (JSC::FTL::osrExitGenerationThunkGenerator): (JSC::FTL::lazySlowPathGenerationThunkGenerator): (JSC::FTL::registerClobberCheck): * ftl/FTLThunks.h: * interpreter/CallFrame.h: (JSC::CallSiteIndex::CallSiteIndex): (JSC::CallSiteIndex::operator bool): (JSC::CallSiteIndex::bits): * jit/CCallHelpers.h: (JSC::CCallHelpers::setupArgument): (JSC::CCallHelpers::setupArgumentsWithExecState): * jit/JITOperations.cpp: Source/WTF: Enables SharedTask to handle any function type, not just void(). It's probably better to use SharedTask instead of std::function in performance-sensitive code. std::function uses the system malloc and has copy semantics. SharedTask uses FastMalloc and has aliasing semantics. So, you can just trust that it will have sensible performance characteristics. * wtf/ParallelHelperPool.cpp: (WTF::ParallelHelperClient::~ParallelHelperClient): (WTF::ParallelHelperClient::setTask): (WTF::ParallelHelperClient::doSomeHelping): (WTF::ParallelHelperClient::runTaskInParallel): (WTF::ParallelHelperClient::finish): (WTF::ParallelHelperClient::claimTask): (WTF::ParallelHelperClient::runTask): (WTF::ParallelHelperPool::doSomeHelping): (WTF::ParallelHelperPool::helperThreadBody): * wtf/ParallelHelperPool.h: (WTF::ParallelHelperClient::setFunction): (WTF::ParallelHelperClient::runFunctionInParallel): (WTF::ParallelHelperClient::pool): * wtf/SharedTask.h: (WTF::createSharedTask): (WTF::SharedTask::SharedTask): Deleted. (WTF::SharedTask::~SharedTask): Deleted. (WTF::SharedTaskFunctor::SharedTaskFunctor): Deleted. Canonical link: https://commits.webkit.org/168154@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@190860 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2015-10-12 17:56:26 +00:00
m_jit.addLinkTask(
LinkBuffer and ExecutableAllocator shouldn't have anything to do with VM https://bugs.webkit.org/show_bug.cgi?id=170210 Reviewed by Mark Lam. Source/JavaScriptCore: This is one more step in the direction of PIC-ified Wasm. LinkBuffer and ExecutableAllocator have no business knowing about VM. * assembler/LinkBuffer.cpp: (JSC::LinkBuffer::allocate): * assembler/LinkBuffer.h: (JSC::LinkBuffer::LinkBuffer): (JSC::LinkBuffer::vm): Deleted. * b3/B3Compile.cpp: (JSC::B3::compile): * b3/B3Compile.h: * b3/air/testair.cpp: * b3/testb3.cpp: (JSC::B3::compileProc): (JSC::B3::compileAndRun): (JSC::B3::testLoadAcq42): (JSC::B3::testAddArgZeroImmZDef): (JSC::B3::testAddLoadTwice): (JSC::B3::testMulLoadTwice): (JSC::B3::testMulAddArgsLeft): (JSC::B3::testMulAddArgsRight): (JSC::B3::testMulAddArgsLeft32): (JSC::B3::testMulAddArgsRight32): (JSC::B3::testMulSubArgsLeft): (JSC::B3::testMulSubArgsRight): (JSC::B3::testMulSubArgsLeft32): (JSC::B3::testMulSubArgsRight32): (JSC::B3::testMulNegArgs): (JSC::B3::testMulNegArgs32): (JSC::B3::testCompareFloatToDoubleThroughPhi): (JSC::B3::testDoubleToFloatThroughPhi): (JSC::B3::testReduceFloatToDoubleValidates): (JSC::B3::testDoubleProducerPhiToFloatConversion): (JSC::B3::testDoubleProducerPhiToFloatConversionWithDoubleConsumer): (JSC::B3::testDoubleProducerPhiWithNonFloatConst): (JSC::B3::testIToD64Arg): (JSC::B3::testIToF64Arg): (JSC::B3::testIToD32Arg): (JSC::B3::testIToF32Arg): (JSC::B3::testIToD64Mem): (JSC::B3::testIToF64Mem): (JSC::B3::testIToD32Mem): (JSC::B3::testIToF32Mem): (JSC::B3::testIToDReducedToIToF64Arg): (JSC::B3::testIToDReducedToIToF32Arg): (JSC::B3::testStoreRelAddLoadAcq32): (JSC::B3::testStoreRelAddLoadAcq8): (JSC::B3::testStoreRelAddFenceLoadAcq8): (JSC::B3::testStoreRelAddLoadAcq16): (JSC::B3::testStoreRelAddLoadAcq64): (JSC::B3::testBranch): (JSC::B3::testBranchPtr): (JSC::B3::testDiamond): (JSC::B3::testBranchNotEqual): (JSC::B3::testBranchNotEqualCommute): (JSC::B3::testBranchNotEqualNotEqual): (JSC::B3::testBranchEqual): (JSC::B3::testBranchEqualEqual): (JSC::B3::testBranchEqualCommute): (JSC::B3::testBranchEqualEqual1): (JSC::B3::testBranchLoadPtr): (JSC::B3::testBranchLoad32): (JSC::B3::testBranchLoad8S): (JSC::B3::testBranchLoad8Z): (JSC::B3::testBranchLoad16S): (JSC::B3::testBranchLoad16Z): (JSC::B3::testBranch8WithLoad8ZIndex): (JSC::B3::testComplex): (JSC::B3::testSimpleCheck): (JSC::B3::testCheckFalse): (JSC::B3::testCheckTrue): (JSC::B3::testCheckLessThan): (JSC::B3::testCheckMegaCombo): (JSC::B3::testCheckTrickyMegaCombo): (JSC::B3::testCheckTwoMegaCombos): (JSC::B3::testCheckTwoNonRedundantMegaCombos): (JSC::B3::testCheckAddImm): (JSC::B3::testCheckAddImmCommute): (JSC::B3::testCheckAddImmSomeRegister): (JSC::B3::testCheckAdd): (JSC::B3::testCheckAdd64): (JSC::B3::testCheckAddFold): (JSC::B3::testCheckAddFoldFail): (JSC::B3::testCheckAddSelfOverflow64): (JSC::B3::testCheckAddSelfOverflow32): (JSC::B3::testCheckSubImm): (JSC::B3::testCheckSubBadImm): (JSC::B3::testCheckSub): (JSC::B3::testCheckSub64): (JSC::B3::testCheckSubFold): (JSC::B3::testCheckSubFoldFail): (JSC::B3::testCheckNeg): (JSC::B3::testCheckNeg64): (JSC::B3::testCheckMul): (JSC::B3::testCheckMulMemory): (JSC::B3::testCheckMul2): (JSC::B3::testCheckMul64): (JSC::B3::testCheckMulFold): (JSC::B3::testCheckMulFoldFail): (JSC::B3::testCheckMul64SShr): (JSC::B3::testSwitch): (JSC::B3::testSwitchChillDiv): (JSC::B3::testSwitchTargettingSameBlock): (JSC::B3::testSwitchTargettingSameBlockFoldPathConstant): (JSC::B3::testBasicSelect): (JSC::B3::testSelectTest): (JSC::B3::testSelectCompareDouble): (JSC::B3::testSelectDouble): (JSC::B3::testSelectDoubleTest): (JSC::B3::testSelectDoubleCompareDouble): (JSC::B3::testSelectFloatCompareFloat): (JSC::B3::testSelectFold): (JSC::B3::testSelectInvert): (JSC::B3::testCheckSelect): (JSC::B3::testCheckSelectCheckSelect): (JSC::B3::testCheckSelectAndCSE): (JSC::B3::testTrivialInfiniteLoop): (JSC::B3::testFoldPathEqual): (JSC::B3::testLShiftSelf32): (JSC::B3::testRShiftSelf32): (JSC::B3::testURShiftSelf32): (JSC::B3::testLShiftSelf64): (JSC::B3::testRShiftSelf64): (JSC::B3::testURShiftSelf64): (JSC::B3::testPatchpointDoubleRegs): (JSC::B3::testSpillDefSmallerThanUse): (JSC::B3::testSpillUseLargerThanDef): (JSC::B3::testLateRegister): (JSC::B3::testInterpreter): (JSC::B3::testEntrySwitchSimple): (JSC::B3::testEntrySwitchNoEntrySwitch): (JSC::B3::testEntrySwitchWithCommonPaths): (JSC::B3::testEntrySwitchWithCommonPathsAndNonTrivialEntrypoint): (JSC::B3::testEntrySwitchLoop): (JSC::B3::testSomeEarlyRegister): (JSC::B3::testTerminalPatchpointThatNeedsToBeSpilled): (JSC::B3::testTerminalPatchpointThatNeedsToBeSpilled2): (JSC::B3::testPatchpointTerminalReturnValue): (JSC::B3::testMemoryFence): (JSC::B3::testStoreFence): (JSC::B3::testLoadFence): (JSC::B3::testPCOriginMapDoesntInsertNops): (JSC::B3::testPinRegisters): (JSC::B3::testX86LeaAddAddShlLeft): (JSC::B3::testX86LeaAddAddShlRight): (JSC::B3::testX86LeaAddAdd): (JSC::B3::testX86LeaAddShlRight): (JSC::B3::testX86LeaAddShlLeftScale1): (JSC::B3::testX86LeaAddShlLeftScale2): (JSC::B3::testX86LeaAddShlLeftScale4): (JSC::B3::testX86LeaAddShlLeftScale8): (JSC::B3::testAddShl32): (JSC::B3::testAddShl64): (JSC::B3::testAddShl65): (JSC::B3::testLoadBaseIndexShift2): (JSC::B3::testLoadBaseIndexShift32): (JSC::B3::testOptimizeMaterialization): (JSC::B3::testAtomicWeakCAS): (JSC::B3::testAtomicStrongCAS): (JSC::B3::testAtomicXchg): (JSC::B3::testDepend32): (JSC::B3::testDepend64): (JSC::B3::testWasmBoundsCheck): (JSC::B3::testWasmAddress): (JSC::B3::run): (JSC::B3::compile): Deleted. * bytecode/PolymorphicAccess.cpp: (JSC::PolymorphicAccess::regenerate): * dfg/DFGJITCompiler.cpp: (JSC::DFG::JITCompiler::compile): (JSC::DFG::JITCompiler::compileFunction): * dfg/DFGLazyJSValue.cpp: (JSC::DFG::LazyJSValue::emit): * dfg/DFGOSRExitCompiler.cpp: * dfg/DFGSpeculativeJIT32_64.cpp: (JSC::DFG::SpeculativeJIT::emitCall): * dfg/DFGSpeculativeJIT64.cpp: (JSC::DFG::SpeculativeJIT::emitCall): * dfg/DFGThunks.cpp: (JSC::DFG::osrExitGenerationThunkGenerator): (JSC::DFG::osrEntryThunkGenerator): * ftl/FTLCompile.cpp: (JSC::FTL::compile): * ftl/FTLLazySlowPath.cpp: (JSC::FTL::LazySlowPath::generate): * ftl/FTLLink.cpp: (JSC::FTL::link): * ftl/FTLLowerDFGToB3.cpp: (JSC::FTL::DFG::LowerDFGToB3::compileCallOrConstruct): (JSC::FTL::DFG::LowerDFGToB3::compileTailCall): (JSC::FTL::DFG::LowerDFGToB3::compileCallOrConstructVarargsSpread): (JSC::FTL::DFG::LowerDFGToB3::compileCallOrConstructVarargs): (JSC::FTL::DFG::LowerDFGToB3::compileCallEval): * ftl/FTLOSRExitCompiler.cpp: (JSC::FTL::compileStub): * ftl/FTLOSRExitHandle.cpp: (JSC::FTL::OSRExitHandle::emitExitThunk): * ftl/FTLSlowPathCall.cpp: (JSC::FTL::SlowPathCallContext::makeCall): * ftl/FTLSlowPathCall.h: (JSC::FTL::callOperation): * ftl/FTLState.h: * ftl/FTLThunks.cpp: (JSC::FTL::genericGenerationThunkGenerator): (JSC::FTL::slowPathCallThunkGenerator): * ftl/FTLThunks.h: (JSC::FTL::generateIfNecessary): (JSC::FTL::Thunks::getSlowPathCallThunk): * jit/AssemblyHelpers.cpp: (JSC::AssemblyHelpers::emitDumbVirtualCall): * jit/AssemblyHelpers.h: * jit/ExecutableAllocator.cpp: (JSC::ExecutableAllocator::initializeAllocator): (JSC::ExecutableAllocator::singleton): (JSC::ExecutableAllocator::ExecutableAllocator): (JSC::ExecutableAllocator::allocate): * jit/ExecutableAllocator.h: * jit/JIT.cpp: (JSC::JIT::compileWithoutLinking): * jit/JITCall.cpp: (JSC::JIT::compileCallEvalSlowCase): * jit/JITMathIC.h: (JSC::JITMathIC::generateOutOfLine): * jit/JITOpcodes.cpp: (JSC::JIT::privateCompileHasIndexedProperty): * jit/JITOpcodes32_64.cpp: (JSC::JIT::privateCompileHasIndexedProperty): * jit/JITOperations.cpp: * jit/JITOperations.h: * jit/JITPropertyAccess.cpp: (JSC::JIT::stringGetByValStubGenerator): (JSC::JIT::privateCompileGetByVal): (JSC::JIT::privateCompileGetByValWithCachedId): (JSC::JIT::privateCompilePutByVal): (JSC::JIT::privateCompilePutByValWithCachedId): * jit/JITPropertyAccess32_64.cpp: (JSC::JIT::stringGetByValStubGenerator): * jit/JITStubRoutine.h: * jit/Repatch.cpp: (JSC::ftlThunkAwareRepatchCall): (JSC::linkPolymorphicCall): * jit/SpecializedThunkJIT.h: (JSC::SpecializedThunkJIT::finalize): * jit/ThunkGenerators.cpp: (JSC::throwExceptionFromCallSlowPathGenerator): (JSC::linkCallThunkGenerator): (JSC::linkPolymorphicCallThunkGenerator): (JSC::virtualThunkFor): (JSC::nativeForGenerator): (JSC::arityFixupGenerator): (JSC::unreachableGenerator): (JSC::boundThisNoArgsFunctionCallGenerator): (JSC::throwExceptionFromWasmThunkGenerator): * llint/LLIntThunks.cpp: (JSC::LLInt::generateThunkWithJumpTo): * runtime/SamplingProfiler.cpp: (JSC::SamplingProfiler::takeSample): * runtime/VM.cpp: (JSC::VM::VM): * runtime/VM.h: * runtime/VMTraps.cpp: (JSC::VMTraps::tryInstallTrapBreakpoints): * tools/VMInspector.cpp: * wasm/WasmBinding.cpp: (JSC::Wasm::wasmToJs): (JSC::Wasm::wasmToWasm): (JSC::Wasm::exitStubGenerator): * wasm/WasmPlan.cpp: (JSC::Wasm::Plan::complete): * yarr/YarrJIT.cpp: (JSC::Yarr::YarrGenerator::compile): (JSC::Yarr::jitCompile): Source/WebCore: * cssjit/SelectorCompiler.cpp: (WebCore::SelectorCompiler::compileSelector): (WebCore::SelectorCompiler::SelectorCodeGenerator::compile): Canonical link: https://commits.webkit.org/187182@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@214571 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-03-29 22:55:53 +00:00
[result, &vm] (LinkBuffer& linkBuffer) {
Templatize CodePtr/Refs/FunctionPtrs with PtrTags. https://bugs.webkit.org/show_bug.cgi?id=184702 <rdar://problem/35391681> Reviewed by Filip Pizlo and Saam Barati. Source/JavaScriptCore: 1. Templatized MacroAssemblerCodePtr/Ref, FunctionPtr, and CodeLocation variants to take a PtrTag template argument. 2. Replaced some uses of raw pointers with the equivalent CodePtr / FunctionPtr. * assembler/AbstractMacroAssembler.h: (JSC::AbstractMacroAssembler::differenceBetweenCodePtr): (JSC::AbstractMacroAssembler::linkJump): (JSC::AbstractMacroAssembler::linkPointer): (JSC::AbstractMacroAssembler::getLinkerAddress): (JSC::AbstractMacroAssembler::repatchJump): (JSC::AbstractMacroAssembler::repatchJumpToNop): (JSC::AbstractMacroAssembler::repatchNearCall): (JSC::AbstractMacroAssembler::repatchCompact): (JSC::AbstractMacroAssembler::repatchInt32): (JSC::AbstractMacroAssembler::repatchPointer): (JSC::AbstractMacroAssembler::readPointer): (JSC::AbstractMacroAssembler::replaceWithLoad): (JSC::AbstractMacroAssembler::replaceWithAddressComputation): * assembler/CodeLocation.h: (JSC::CodeLocationCommon:: const): (JSC::CodeLocationCommon::CodeLocationCommon): (JSC::CodeLocationInstruction::CodeLocationInstruction): (JSC::CodeLocationLabel::CodeLocationLabel): (JSC::CodeLocationLabel::retagged): (JSC::CodeLocationLabel:: const): (JSC::CodeLocationJump::CodeLocationJump): (JSC::CodeLocationJump::retagged): (JSC::CodeLocationCall::CodeLocationCall): (JSC::CodeLocationCall::retagged): (JSC::CodeLocationNearCall::CodeLocationNearCall): (JSC::CodeLocationDataLabel32::CodeLocationDataLabel32): (JSC::CodeLocationDataLabelCompact::CodeLocationDataLabelCompact): (JSC::CodeLocationDataLabelPtr::CodeLocationDataLabelPtr): (JSC::CodeLocationConvertibleLoad::CodeLocationConvertibleLoad): (JSC::CodeLocationCommon<tag>::instructionAtOffset): (JSC::CodeLocationCommon<tag>::labelAtOffset): (JSC::CodeLocationCommon<tag>::jumpAtOffset): (JSC::CodeLocationCommon<tag>::callAtOffset): (JSC::CodeLocationCommon<tag>::nearCallAtOffset): (JSC::CodeLocationCommon<tag>::dataLabelPtrAtOffset): (JSC::CodeLocationCommon<tag>::dataLabel32AtOffset): (JSC::CodeLocationCommon<tag>::dataLabelCompactAtOffset): (JSC::CodeLocationCommon<tag>::convertibleLoadAtOffset): (JSC::CodeLocationCommon::instructionAtOffset): Deleted. (JSC::CodeLocationCommon::labelAtOffset): Deleted. (JSC::CodeLocationCommon::jumpAtOffset): Deleted. (JSC::CodeLocationCommon::callAtOffset): Deleted. (JSC::CodeLocationCommon::nearCallAtOffset): Deleted. (JSC::CodeLocationCommon::dataLabelPtrAtOffset): Deleted. (JSC::CodeLocationCommon::dataLabel32AtOffset): Deleted. (JSC::CodeLocationCommon::dataLabelCompactAtOffset): Deleted. (JSC::CodeLocationCommon::convertibleLoadAtOffset): Deleted. * assembler/LinkBuffer.cpp: (JSC::LinkBuffer::finalizeCodeWithoutDisassemblyImpl): (JSC::LinkBuffer::finalizeCodeWithDisassemblyImpl): (JSC::LinkBuffer::finalizeCodeWithoutDisassembly): Deleted. (JSC::LinkBuffer::finalizeCodeWithDisassembly): Deleted. * assembler/LinkBuffer.h: (JSC::LinkBuffer::link): (JSC::LinkBuffer::patch): (JSC::LinkBuffer::entrypoint): (JSC::LinkBuffer::locationOf): (JSC::LinkBuffer::locationOfNearCall): (JSC::LinkBuffer::finalizeCodeWithoutDisassembly): (JSC::LinkBuffer::finalizeCodeWithDisassembly): (JSC::LinkBuffer::trampolineAt): * assembler/MacroAssemblerARM.h: (JSC::MacroAssemblerARM::readCallTarget): (JSC::MacroAssemblerARM::replaceWithJump): (JSC::MacroAssemblerARM::startOfPatchableBranch32WithPatchOnAddress): (JSC::MacroAssemblerARM::startOfPatchableBranchPtrWithPatchOnAddress): (JSC::MacroAssemblerARM::startOfBranchPtrWithPatchOnRegister): (JSC::MacroAssemblerARM::revertJumpReplacementToBranchPtrWithPatch): (JSC::MacroAssemblerARM::revertJumpReplacementToPatchableBranch32WithPatch): (JSC::MacroAssemblerARM::revertJumpReplacementToPatchableBranchPtrWithPatch): (JSC::MacroAssemblerARM::repatchCall): (JSC::MacroAssemblerARM::linkCall): * assembler/MacroAssemblerARM64.h: (JSC::MacroAssemblerARM64::readCallTarget): (JSC::MacroAssemblerARM64::replaceWithVMHalt): (JSC::MacroAssemblerARM64::replaceWithJump): (JSC::MacroAssemblerARM64::startOfBranchPtrWithPatchOnRegister): (JSC::MacroAssemblerARM64::startOfPatchableBranchPtrWithPatchOnAddress): (JSC::MacroAssemblerARM64::startOfPatchableBranch32WithPatchOnAddress): (JSC::MacroAssemblerARM64::revertJumpReplacementToBranchPtrWithPatch): (JSC::MacroAssemblerARM64::revertJumpReplacementToPatchableBranchPtrWithPatch): (JSC::MacroAssemblerARM64::revertJumpReplacementToPatchableBranch32WithPatch): (JSC::MacroAssemblerARM64::repatchCall): (JSC::MacroAssemblerARM64::linkCall): * assembler/MacroAssemblerARMv7.h: (JSC::MacroAssemblerARMv7::replaceWithJump): (JSC::MacroAssemblerARMv7::readCallTarget): (JSC::MacroAssemblerARMv7::startOfBranchPtrWithPatchOnRegister): (JSC::MacroAssemblerARMv7::revertJumpReplacementToBranchPtrWithPatch): (JSC::MacroAssemblerARMv7::startOfPatchableBranchPtrWithPatchOnAddress): (JSC::MacroAssemblerARMv7::startOfPatchableBranch32WithPatchOnAddress): (JSC::MacroAssemblerARMv7::revertJumpReplacementToPatchableBranchPtrWithPatch): (JSC::MacroAssemblerARMv7::revertJumpReplacementToPatchableBranch32WithPatch): (JSC::MacroAssemblerARMv7::repatchCall): (JSC::MacroAssemblerARMv7::linkCall): * assembler/MacroAssemblerCodeRef.cpp: (JSC::MacroAssemblerCodePtrBase::dumpWithName): (JSC::MacroAssemblerCodeRefBase::tryToDisassemble): (JSC::MacroAssemblerCodeRefBase::disassembly): (JSC::MacroAssemblerCodePtr::createLLIntCodePtr): Deleted. (JSC::MacroAssemblerCodePtr::dumpWithName const): Deleted. (JSC::MacroAssemblerCodePtr::dump const): Deleted. (JSC::MacroAssemblerCodeRef::createLLIntCodeRef): Deleted. (JSC::MacroAssemblerCodeRef::tryToDisassemble const): Deleted. (JSC::MacroAssemblerCodeRef::disassembly const): Deleted. (JSC::MacroAssemblerCodeRef::dump const): Deleted. * assembler/MacroAssemblerCodeRef.h: (JSC::FunctionPtr::FunctionPtr): (JSC::FunctionPtr::retagged const): (JSC::FunctionPtr::retaggedExecutableAddress const): (JSC::FunctionPtr::operator== const): (JSC::FunctionPtr::operator!= const): (JSC::ReturnAddressPtr::ReturnAddressPtr): (JSC::MacroAssemblerCodePtr::MacroAssemblerCodePtr): (JSC::MacroAssemblerCodePtr::createFromExecutableAddress): (JSC::MacroAssemblerCodePtr::retagged const): (JSC::MacroAssemblerCodePtr:: const): (JSC::MacroAssemblerCodePtr::dumpWithName const): (JSC::MacroAssemblerCodePtr::dump const): (JSC::MacroAssemblerCodePtrHash::hash): (JSC::MacroAssemblerCodePtrHash::equal): (JSC::MacroAssemblerCodeRef::MacroAssemblerCodeRef): (JSC::MacroAssemblerCodeRef::createSelfManagedCodeRef): (JSC::MacroAssemblerCodeRef::code const): (JSC::MacroAssemblerCodeRef::retaggedCode const): (JSC::MacroAssemblerCodeRef::retagged const): (JSC::MacroAssemblerCodeRef::tryToDisassemble const): (JSC::MacroAssemblerCodeRef::disassembly const): (JSC::MacroAssemblerCodeRef::dump const): (JSC::FunctionPtr<tag>::FunctionPtr): * assembler/MacroAssemblerMIPS.h: (JSC::MacroAssemblerMIPS::readCallTarget): (JSC::MacroAssemblerMIPS::replaceWithJump): (JSC::MacroAssemblerMIPS::startOfPatchableBranch32WithPatchOnAddress): (JSC::MacroAssemblerMIPS::startOfBranchPtrWithPatchOnRegister): (JSC::MacroAssemblerMIPS::revertJumpReplacementToBranchPtrWithPatch): (JSC::MacroAssemblerMIPS::startOfPatchableBranchPtrWithPatchOnAddress): (JSC::MacroAssemblerMIPS::revertJumpReplacementToPatchableBranch32WithPatch): (JSC::MacroAssemblerMIPS::revertJumpReplacementToPatchableBranchPtrWithPatch): (JSC::MacroAssemblerMIPS::repatchCall): (JSC::MacroAssemblerMIPS::linkCall): * assembler/MacroAssemblerX86.h: (JSC::MacroAssemblerX86::readCallTarget): (JSC::MacroAssemblerX86::startOfBranchPtrWithPatchOnRegister): (JSC::MacroAssemblerX86::startOfPatchableBranchPtrWithPatchOnAddress): (JSC::MacroAssemblerX86::startOfPatchableBranch32WithPatchOnAddress): (JSC::MacroAssemblerX86::revertJumpReplacementToBranchPtrWithPatch): (JSC::MacroAssemblerX86::revertJumpReplacementToPatchableBranchPtrWithPatch): (JSC::MacroAssemblerX86::revertJumpReplacementToPatchableBranch32WithPatch): (JSC::MacroAssemblerX86::repatchCall): (JSC::MacroAssemblerX86::linkCall): * assembler/MacroAssemblerX86Common.h: (JSC::MacroAssemblerX86Common::repatchCompact): (JSC::MacroAssemblerX86Common::replaceWithVMHalt): (JSC::MacroAssemblerX86Common::replaceWithJump): * assembler/MacroAssemblerX86_64.h: (JSC::MacroAssemblerX86_64::readCallTarget): (JSC::MacroAssemblerX86_64::startOfBranchPtrWithPatchOnRegister): (JSC::MacroAssemblerX86_64::startOfBranch32WithPatchOnRegister): (JSC::MacroAssemblerX86_64::startOfPatchableBranchPtrWithPatchOnAddress): (JSC::MacroAssemblerX86_64::startOfPatchableBranch32WithPatchOnAddress): (JSC::MacroAssemblerX86_64::revertJumpReplacementToPatchableBranchPtrWithPatch): (JSC::MacroAssemblerX86_64::revertJumpReplacementToPatchableBranch32WithPatch): (JSC::MacroAssemblerX86_64::revertJumpReplacementToBranchPtrWithPatch): (JSC::MacroAssemblerX86_64::repatchCall): (JSC::MacroAssemblerX86_64::linkCall): * assembler/testmasm.cpp: (JSC::compile): (JSC::invoke): (JSC::testProbeModifiesProgramCounter): * b3/B3Compilation.cpp: (JSC::B3::Compilation::Compilation): * b3/B3Compilation.h: (JSC::B3::Compilation::code const): (JSC::B3::Compilation::codeRef const): * b3/B3Compile.cpp: (JSC::B3::compile): * b3/B3LowerMacros.cpp: * b3/air/AirDisassembler.cpp: (JSC::B3::Air::Disassembler::dump): * b3/air/testair.cpp: * b3/testb3.cpp: (JSC::B3::invoke): (JSC::B3::testInterpreter): (JSC::B3::testEntrySwitchSimple): (JSC::B3::testEntrySwitchNoEntrySwitch): (JSC::B3::testEntrySwitchWithCommonPaths): (JSC::B3::testEntrySwitchWithCommonPathsAndNonTrivialEntrypoint): (JSC::B3::testEntrySwitchLoop): * bytecode/AccessCase.cpp: (JSC::AccessCase::generateImpl): * bytecode/AccessCaseSnippetParams.cpp: (JSC::SlowPathCallGeneratorWithArguments::generateImpl): * bytecode/ByValInfo.h: (JSC::ByValInfo::ByValInfo): * bytecode/CallLinkInfo.cpp: (JSC::CallLinkInfo::callReturnLocation): (JSC::CallLinkInfo::patchableJump): (JSC::CallLinkInfo::hotPathBegin): (JSC::CallLinkInfo::slowPathStart): * bytecode/CallLinkInfo.h: (JSC::CallLinkInfo::setCallLocations): (JSC::CallLinkInfo::hotPathOther): * bytecode/CodeBlock.cpp: (JSC::CodeBlock::finishCreation): * bytecode/GetByIdStatus.cpp: (JSC::GetByIdStatus::computeForStubInfoWithoutExitSiteFeedback): * bytecode/GetByIdVariant.cpp: (JSC::GetByIdVariant::GetByIdVariant): (JSC::GetByIdVariant::dumpInContext const): * bytecode/GetByIdVariant.h: (JSC::GetByIdVariant::customAccessorGetter const): * bytecode/GetterSetterAccessCase.cpp: (JSC::GetterSetterAccessCase::create): (JSC::GetterSetterAccessCase::GetterSetterAccessCase): (JSC::GetterSetterAccessCase::dumpImpl const): * bytecode/GetterSetterAccessCase.h: (JSC::GetterSetterAccessCase::customAccessor const): (): Deleted. * bytecode/HandlerInfo.h: (JSC::HandlerInfo::initialize): * bytecode/InlineAccess.cpp: (JSC::linkCodeInline): (JSC::InlineAccess::rewireStubAsJump): * bytecode/InlineAccess.h: * bytecode/JumpTable.h: (JSC::StringJumpTable::ctiForValue): (JSC::SimpleJumpTable::ctiForValue): * bytecode/LLIntCallLinkInfo.h: (JSC::LLIntCallLinkInfo::unlink): * bytecode/PolymorphicAccess.cpp: (JSC::AccessGenerationState::emitExplicitExceptionHandler): (JSC::PolymorphicAccess::regenerate): * bytecode/PolymorphicAccess.h: (JSC::AccessGenerationResult::AccessGenerationResult): (JSC::AccessGenerationResult::code const): * bytecode/StructureStubInfo.h: (JSC::StructureStubInfo::slowPathCallLocation): (JSC::StructureStubInfo::doneLocation): (JSC::StructureStubInfo::slowPathStartLocation): (JSC::StructureStubInfo::patchableJumpForIn): * dfg/DFGCommonData.h: (JSC::DFG::CommonData::appendCatchEntrypoint): * dfg/DFGDisassembler.cpp: (JSC::DFG::Disassembler::dumpDisassembly): * dfg/DFGDriver.h: * dfg/DFGJITCompiler.cpp: (JSC::DFG::JITCompiler::linkOSRExits): (JSC::DFG::JITCompiler::compileExceptionHandlers): (JSC::DFG::JITCompiler::link): (JSC::DFG::JITCompiler::compileFunction): (JSC::DFG::JITCompiler::noticeCatchEntrypoint): * dfg/DFGJITCompiler.h: (JSC::DFG::CallLinkRecord::CallLinkRecord): (JSC::DFG::JITCompiler::appendCall): (JSC::DFG::JITCompiler::JSCallRecord::JSCallRecord): (JSC::DFG::JITCompiler::JSDirectCallRecord::JSDirectCallRecord): (JSC::DFG::JITCompiler::JSDirectTailCallRecord::JSDirectTailCallRecord): * dfg/DFGJITFinalizer.cpp: (JSC::DFG::JITFinalizer::JITFinalizer): (JSC::DFG::JITFinalizer::finalize): (JSC::DFG::JITFinalizer::finalizeFunction): * dfg/DFGJITFinalizer.h: * dfg/DFGJumpReplacement.h: (JSC::DFG::JumpReplacement::JumpReplacement): * dfg/DFGNode.h: * dfg/DFGOSREntry.cpp: (JSC::DFG::prepareOSREntry): (JSC::DFG::prepareCatchOSREntry): * dfg/DFGOSREntry.h: (JSC::DFG::prepareOSREntry): * dfg/DFGOSRExit.cpp: (JSC::DFG::OSRExit::executeOSRExit): (JSC::DFG::reifyInlinedCallFrames): (JSC::DFG::adjustAndJumpToTarget): (JSC::DFG::OSRExit::codeLocationForRepatch const): (JSC::DFG::OSRExit::emitRestoreArguments): (JSC::DFG::OSRExit::compileOSRExit): * dfg/DFGOSRExit.h: * dfg/DFGOSRExitCompilerCommon.cpp: (JSC::DFG::handleExitCounts): (JSC::DFG::reifyInlinedCallFrames): (JSC::DFG::osrWriteBarrier): (JSC::DFG::adjustAndJumpToTarget): * dfg/DFGOperations.cpp: * dfg/DFGSlowPathGenerator.h: (JSC::DFG::CallResultAndArgumentsSlowPathGenerator::CallResultAndArgumentsSlowPathGenerator): (JSC::DFG::CallResultAndArgumentsSlowPathGenerator::unpackAndGenerate): (JSC::DFG::slowPathCall): * dfg/DFGSpeculativeJIT.cpp: (JSC::DFG::SpeculativeJIT::compileMathIC): (JSC::DFG::SpeculativeJIT::compileCallDOM): (JSC::DFG::SpeculativeJIT::compileCallDOMGetter): (JSC::DFG::SpeculativeJIT::emitSwitchIntJump): (JSC::DFG::SpeculativeJIT::emitSwitchImm): (JSC::DFG::SpeculativeJIT::emitSwitchStringOnString): (JSC::DFG::SpeculativeJIT::compileHasIndexedProperty): (JSC::DFG::SpeculativeJIT::compileGetDirectPname): (JSC::DFG::SpeculativeJIT::cachedPutById): * dfg/DFGSpeculativeJIT.h: (JSC::DFG::SpeculativeJIT::callOperation): (JSC::DFG::SpeculativeJIT::appendCall): (JSC::DFG::SpeculativeJIT::appendCallWithCallFrameRollbackOnException): (JSC::DFG::SpeculativeJIT::appendCallWithCallFrameRollbackOnExceptionSetResult): (JSC::DFG::SpeculativeJIT::appendCallSetResult): * dfg/DFGSpeculativeJIT64.cpp: (JSC::DFG::SpeculativeJIT::cachedGetById): (JSC::DFG::SpeculativeJIT::cachedGetByIdWithThis): (JSC::DFG::SpeculativeJIT::compile): * dfg/DFGThunks.cpp: (JSC::DFG::osrExitThunkGenerator): (JSC::DFG::osrExitGenerationThunkGenerator): (JSC::DFG::osrEntryThunkGenerator): * dfg/DFGThunks.h: * disassembler/ARM64Disassembler.cpp: (JSC::tryToDisassemble): * disassembler/ARMv7Disassembler.cpp: (JSC::tryToDisassemble): * disassembler/Disassembler.cpp: (JSC::disassemble): (JSC::disassembleAsynchronously): * disassembler/Disassembler.h: (JSC::tryToDisassemble): * disassembler/UDis86Disassembler.cpp: (JSC::tryToDisassembleWithUDis86): * disassembler/UDis86Disassembler.h: (JSC::tryToDisassembleWithUDis86): * disassembler/X86Disassembler.cpp: (JSC::tryToDisassemble): * ftl/FTLCompile.cpp: (JSC::FTL::compile): * ftl/FTLExceptionTarget.cpp: (JSC::FTL::ExceptionTarget::label): (JSC::FTL::ExceptionTarget::jumps): * ftl/FTLExceptionTarget.h: * ftl/FTLGeneratedFunction.h: * ftl/FTLJITCode.cpp: (JSC::FTL::JITCode::initializeB3Code): (JSC::FTL::JITCode::initializeAddressForCall): (JSC::FTL::JITCode::initializeArityCheckEntrypoint): (JSC::FTL::JITCode::addressForCall): (JSC::FTL::JITCode::executableAddressAtOffset): * ftl/FTLJITCode.h: (JSC::FTL::JITCode::b3Code const): * ftl/FTLJITFinalizer.cpp: (JSC::FTL::JITFinalizer::finalizeCommon): * ftl/FTLLazySlowPath.cpp: (JSC::FTL::LazySlowPath::initialize): (JSC::FTL::LazySlowPath::generate): * ftl/FTLLazySlowPath.h: (JSC::FTL::LazySlowPath::patchableJump const): (JSC::FTL::LazySlowPath::done const): (JSC::FTL::LazySlowPath::stub const): * ftl/FTLLazySlowPathCall.h: (JSC::FTL::createLazyCallGenerator): * ftl/FTLLink.cpp: (JSC::FTL::link): * ftl/FTLLowerDFGToB3.cpp: (JSC::FTL::DFG::LowerDFGToB3::lower): (JSC::FTL::DFG::LowerDFGToB3::compileCallOrConstruct): (JSC::FTL::DFG::LowerDFGToB3::compileDirectCallOrConstruct): (JSC::FTL::DFG::LowerDFGToB3::compileTailCall): (JSC::FTL::DFG::LowerDFGToB3::compileCallOrConstructVarargsSpread): (JSC::FTL::DFG::LowerDFGToB3::compileCallOrConstructVarargs): (JSC::FTL::DFG::LowerDFGToB3::compileCallEval): (JSC::FTL::DFG::LowerDFGToB3::compileInvalidationPoint): (JSC::FTL::DFG::LowerDFGToB3::compileIn): (JSC::FTL::DFG::LowerDFGToB3::compileCheckSubClass): (JSC::FTL::DFG::LowerDFGToB3::compileCallDOM): (JSC::FTL::DFG::LowerDFGToB3::compileCallDOMGetter): (JSC::FTL::DFG::LowerDFGToB3::lazySlowPath): * ftl/FTLOSRExit.cpp: (JSC::FTL::OSRExit::codeLocationForRepatch const): * ftl/FTLOSRExit.h: * ftl/FTLOSRExitCompiler.cpp: (JSC::FTL::compileStub): (JSC::FTL::compileFTLOSRExit): * ftl/FTLOSRExitHandle.cpp: (JSC::FTL::OSRExitHandle::emitExitThunk): * ftl/FTLOperations.cpp: (JSC::FTL::compileFTLLazySlowPath): * ftl/FTLPatchpointExceptionHandle.cpp: (JSC::FTL::PatchpointExceptionHandle::scheduleExitCreationForUnwind): * ftl/FTLSlowPathCall.cpp: (JSC::FTL::SlowPathCallContext::keyWithTarget const): (JSC::FTL::SlowPathCallContext::makeCall): * ftl/FTLSlowPathCall.h: (JSC::FTL::callOperation): * ftl/FTLSlowPathCallKey.cpp: (JSC::FTL::SlowPathCallKey::dump const): * ftl/FTLSlowPathCallKey.h: (JSC::FTL::SlowPathCallKey::SlowPathCallKey): (JSC::FTL::SlowPathCallKey::callTarget const): (JSC::FTL::SlowPathCallKey::withCallTarget): (JSC::FTL::SlowPathCallKey::hash const): (JSC::FTL::SlowPathCallKey::callPtrTag const): Deleted. * ftl/FTLState.cpp: (JSC::FTL::State::State): * ftl/FTLThunks.cpp: (JSC::FTL::genericGenerationThunkGenerator): (JSC::FTL::osrExitGenerationThunkGenerator): (JSC::FTL::lazySlowPathGenerationThunkGenerator): (JSC::FTL::slowPathCallThunkGenerator): * ftl/FTLThunks.h: (JSC::FTL::generateIfNecessary): (JSC::FTL::keyForThunk): (JSC::FTL::Thunks::getSlowPathCallThunk): (JSC::FTL::Thunks::keyForSlowPathCallThunk): * interpreter/InterpreterInlines.h: (JSC::Interpreter::getOpcodeID): * jit/AssemblyHelpers.cpp: (JSC::AssemblyHelpers::callExceptionFuzz): (JSC::AssemblyHelpers::emitDumbVirtualCall): (JSC::AssemblyHelpers::debugCall): * jit/CCallHelpers.cpp: (JSC::CCallHelpers::ensureShadowChickenPacket): * jit/ExecutableAllocator.cpp: (JSC::FixedVMPoolExecutableAllocator::initializeSeparatedWXHeaps): (JSC::FixedVMPoolExecutableAllocator::jitWriteThunkGenerator): * jit/ExecutableAllocator.h: (JSC::performJITMemcpy): * jit/GCAwareJITStubRoutine.cpp: (JSC::GCAwareJITStubRoutine::GCAwareJITStubRoutine): (JSC::MarkingGCAwareJITStubRoutine::MarkingGCAwareJITStubRoutine): (JSC::GCAwareJITStubRoutineWithExceptionHandler::GCAwareJITStubRoutineWithExceptionHandler): (JSC::createJITStubRoutine): * jit/GCAwareJITStubRoutine.h: (JSC::createJITStubRoutine): * jit/JIT.cpp: (JSC::ctiPatchCallByReturnAddress): (JSC::JIT::compileWithoutLinking): (JSC::JIT::link): (JSC::JIT::privateCompileExceptionHandlers): * jit/JIT.h: (JSC::CallRecord::CallRecord): * jit/JITArithmetic.cpp: (JSC::JIT::emitMathICFast): (JSC::JIT::emitMathICSlow): * jit/JITCall.cpp: (JSC::JIT::compileOpCallSlowCase): * jit/JITCall32_64.cpp: (JSC::JIT::compileOpCallSlowCase): * jit/JITCode.cpp: (JSC::JITCodeWithCodeRef::JITCodeWithCodeRef): (JSC::JITCodeWithCodeRef::executableAddressAtOffset): (JSC::DirectJITCode::DirectJITCode): (JSC::DirectJITCode::initializeCodeRef): (JSC::DirectJITCode::addressForCall): (JSC::NativeJITCode::NativeJITCode): (JSC::NativeJITCode::initializeCodeRef): (JSC::NativeJITCode::addressForCall): * jit/JITCode.h: * jit/JITCodeMap.h: (JSC::JITCodeMap::Entry::Entry): (JSC::JITCodeMap::Entry::codeLocation): (JSC::JITCodeMap::append): (JSC::JITCodeMap::find const): * jit/JITDisassembler.cpp: (JSC::JITDisassembler::dumpDisassembly): * jit/JITExceptions.cpp: (JSC::genericUnwind): * jit/JITInlineCacheGenerator.cpp: (JSC::JITByIdGenerator::finalize): * jit/JITInlines.h: (JSC::JIT::emitNakedCall): (JSC::JIT::emitNakedTailCall): (JSC::JIT::appendCallWithExceptionCheck): (JSC::JIT::appendCallWithExceptionCheckAndSlowPathReturnType): (JSC::JIT::appendCallWithCallFrameRollbackOnException): (JSC::JIT::appendCallWithExceptionCheckSetJSValueResult): (JSC::JIT::appendCallWithExceptionCheckSetJSValueResultWithProfile): * jit/JITMathIC.h: (JSC::isProfileEmpty): * jit/JITOpcodes.cpp: (JSC::JIT::emit_op_catch): (JSC::JIT::emit_op_switch_imm): (JSC::JIT::emit_op_switch_char): (JSC::JIT::emit_op_switch_string): (JSC::JIT::privateCompileHasIndexedProperty): (JSC::JIT::emitSlow_op_has_indexed_property): * jit/JITOpcodes32_64.cpp: (JSC::JIT::privateCompileHasIndexedProperty): * jit/JITOperations.cpp: (JSC::getByVal): * jit/JITPropertyAccess.cpp: (JSC::JIT::stringGetByValStubGenerator): (JSC::JIT::emitGetByValWithCachedId): (JSC::JIT::emitSlow_op_get_by_val): (JSC::JIT::emitPutByValWithCachedId): (JSC::JIT::emitSlow_op_put_by_val): (JSC::JIT::emitSlow_op_try_get_by_id): (JSC::JIT::emitSlow_op_get_by_id_direct): (JSC::JIT::emitSlow_op_get_by_id): (JSC::JIT::emitSlow_op_get_by_id_with_this): (JSC::JIT::emitSlow_op_put_by_id): (JSC::JIT::privateCompileGetByVal): (JSC::JIT::privateCompileGetByValWithCachedId): (JSC::JIT::privateCompilePutByVal): (JSC::JIT::privateCompilePutByValWithCachedId): * jit/JITPropertyAccess32_64.cpp: (JSC::JIT::stringGetByValStubGenerator): (JSC::JIT::emitSlow_op_get_by_val): (JSC::JIT::emitSlow_op_put_by_val): * jit/JITStubRoutine.h: (JSC::JITStubRoutine::JITStubRoutine): (JSC::JITStubRoutine::createSelfManagedRoutine): (JSC::JITStubRoutine::code const): (JSC::JITStubRoutine::asCodePtr): * jit/JITThunks.cpp: (JSC::JITThunks::ctiNativeCall): (JSC::JITThunks::ctiNativeConstruct): (JSC::JITThunks::ctiNativeTailCall): (JSC::JITThunks::ctiNativeTailCallWithoutSavedTags): (JSC::JITThunks::ctiInternalFunctionCall): (JSC::JITThunks::ctiInternalFunctionConstruct): (JSC::JITThunks::ctiStub): (JSC::JITThunks::existingCTIStub): (JSC::JITThunks::hostFunctionStub): * jit/JITThunks.h: * jit/PCToCodeOriginMap.cpp: (JSC::PCToCodeOriginMap::PCToCodeOriginMap): * jit/PCToCodeOriginMap.h: * jit/PolymorphicCallStubRoutine.cpp: (JSC::PolymorphicCallStubRoutine::PolymorphicCallStubRoutine): * jit/PolymorphicCallStubRoutine.h: * jit/Repatch.cpp: (JSC::readPutICCallTarget): (JSC::ftlThunkAwareRepatchCall): (JSC::appropriateOptimizingGetByIdFunction): (JSC::appropriateGetByIdFunction): (JSC::tryCacheGetByID): (JSC::repatchGetByID): (JSC::tryCachePutByID): (JSC::repatchPutByID): (JSC::tryCacheIn): (JSC::repatchIn): (JSC::linkSlowFor): (JSC::linkFor): (JSC::linkDirectFor): (JSC::revertCall): (JSC::unlinkFor): (JSC::linkVirtualFor): (JSC::linkPolymorphicCall): (JSC::resetGetByID): (JSC::resetPutByID): * jit/Repatch.h: * jit/SlowPathCall.h: (JSC::JITSlowPathCall::call): * jit/SpecializedThunkJIT.h: (JSC::SpecializedThunkJIT::finalize): (JSC::SpecializedThunkJIT::callDoubleToDouble): (JSC::SpecializedThunkJIT::callDoubleToDoublePreservingReturn): * jit/ThunkGenerator.h: * jit/ThunkGenerators.cpp: (JSC::throwExceptionFromCallSlowPathGenerator): (JSC::slowPathFor): (JSC::linkCallThunkGenerator): (JSC::linkPolymorphicCallThunkGenerator): (JSC::virtualThunkFor): (JSC::nativeForGenerator): (JSC::nativeCallGenerator): (JSC::nativeTailCallGenerator): (JSC::nativeTailCallWithoutSavedTagsGenerator): (JSC::nativeConstructGenerator): (JSC::internalFunctionCallGenerator): (JSC::internalFunctionConstructGenerator): (JSC::arityFixupGenerator): (JSC::unreachableGenerator): (JSC::charCodeAtThunkGenerator): (JSC::charAtThunkGenerator): (JSC::fromCharCodeThunkGenerator): (JSC::clz32ThunkGenerator): (JSC::sqrtThunkGenerator): (JSC::floorThunkGenerator): (JSC::ceilThunkGenerator): (JSC::truncThunkGenerator): (JSC::roundThunkGenerator): (JSC::expThunkGenerator): (JSC::logThunkGenerator): (JSC::absThunkGenerator): (JSC::imulThunkGenerator): (JSC::randomThunkGenerator): (JSC::boundThisNoArgsFunctionCallGenerator): * jit/ThunkGenerators.h: * llint/LLIntData.cpp: (JSC::LLInt::initialize): * llint/LLIntData.h: (JSC::LLInt::getExecutableAddress): (JSC::LLInt::getCodePtr): (JSC::LLInt::getCodeRef): (JSC::LLInt::getCodeFunctionPtr): * llint/LLIntEntrypoint.cpp: (JSC::LLInt::setFunctionEntrypoint): (JSC::LLInt::setEvalEntrypoint): (JSC::LLInt::setProgramEntrypoint): (JSC::LLInt::setModuleProgramEntrypoint): * llint/LLIntExceptions.cpp: (JSC::LLInt::callToThrow): * llint/LLIntSlowPaths.cpp: (JSC::LLInt::LLINT_SLOW_PATH_DECL): (JSC::LLInt::setUpCall): * llint/LLIntThunks.cpp: (JSC::vmEntryToWasm): (JSC::LLInt::generateThunkWithJumpTo): (JSC::LLInt::functionForCallEntryThunkGenerator): (JSC::LLInt::functionForConstructEntryThunkGenerator): (JSC::LLInt::functionForCallArityCheckThunkGenerator): (JSC::LLInt::functionForConstructArityCheckThunkGenerator): (JSC::LLInt::evalEntryThunkGenerator): (JSC::LLInt::programEntryThunkGenerator): (JSC::LLInt::moduleProgramEntryThunkGenerator): * llint/LLIntThunks.h: * llint/LowLevelInterpreter.asm: * llint/LowLevelInterpreter32_64.asm: * llint/LowLevelInterpreter64.asm: * profiler/ProfilerCompilation.cpp: (JSC::Profiler::Compilation::addOSRExitSite): * profiler/ProfilerCompilation.h: * profiler/ProfilerOSRExitSite.cpp: (JSC::Profiler::OSRExitSite::toJS const): * profiler/ProfilerOSRExitSite.h: (JSC::Profiler::OSRExitSite::OSRExitSite): (JSC::Profiler::OSRExitSite::codeAddress const): (JSC::Profiler::OSRExitSite:: const): Deleted. * runtime/ExecutableBase.cpp: (JSC::ExecutableBase::clearCode): * runtime/ExecutableBase.h: (JSC::ExecutableBase::entrypointFor): * runtime/NativeExecutable.cpp: (JSC::NativeExecutable::finishCreation): * runtime/NativeFunction.h: (JSC::TaggedNativeFunction::TaggedNativeFunction): (JSC::TaggedNativeFunction::operator NativeFunction): * runtime/PtrTag.h: (JSC::tagCodePtr): (JSC::untagCodePtr): (JSC::retagCodePtr): (JSC::tagCFunctionPtr): (JSC::untagCFunctionPtr): (JSC::nextPtrTagID): Deleted. * runtime/PutPropertySlot.h: (JSC::PutPropertySlot::PutPropertySlot): (JSC::PutPropertySlot::setCustomValue): (JSC::PutPropertySlot::setCustomAccessor): (JSC::PutPropertySlot::customSetter const): * runtime/ScriptExecutable.cpp: (JSC::ScriptExecutable::installCode): * runtime/VM.cpp: (JSC::VM::getHostFunction): (JSC::VM::getCTIInternalFunctionTrampolineFor): * runtime/VM.h: (JSC::VM::getCTIStub): * wasm/WasmB3IRGenerator.cpp: (JSC::Wasm::B3IRGenerator::B3IRGenerator): (JSC::Wasm::B3IRGenerator::emitExceptionCheck): (JSC::Wasm::B3IRGenerator::emitTierUpCheck): (JSC::Wasm::B3IRGenerator::addCall): (JSC::Wasm::B3IRGenerator::addCallIndirect): * wasm/WasmBBQPlan.cpp: (JSC::Wasm::BBQPlan::prepare): (JSC::Wasm::BBQPlan::complete): * wasm/WasmBBQPlan.h: * wasm/WasmBinding.cpp: (JSC::Wasm::wasmToWasm): * wasm/WasmBinding.h: * wasm/WasmCallee.h: (JSC::Wasm::Callee::entrypoint const): * wasm/WasmCallingConvention.h: (JSC::Wasm::CallingConvention::setupFrameInPrologue const): * wasm/WasmCodeBlock.h: (JSC::Wasm::CodeBlock::entrypointLoadLocationFromFunctionIndexSpace): * wasm/WasmFaultSignalHandler.cpp: (JSC::Wasm::trapHandler): * wasm/WasmFormat.h: * wasm/WasmInstance.h: * wasm/WasmOMGPlan.cpp: (JSC::Wasm::OMGPlan::work): * wasm/WasmThunks.cpp: (JSC::Wasm::throwExceptionFromWasmThunkGenerator): (JSC::Wasm::throwStackOverflowFromWasmThunkGenerator): (JSC::Wasm::triggerOMGTierUpThunkGenerator): (JSC::Wasm::Thunks::stub): (JSC::Wasm::Thunks::existingStub): * wasm/WasmThunks.h: * wasm/js/JSToWasm.cpp: (JSC::Wasm::createJSToWasmWrapper): * wasm/js/JSWebAssemblyCodeBlock.h: * wasm/js/WasmToJS.cpp: (JSC::Wasm::handleBadI64Use): (JSC::Wasm::wasmToJS): * wasm/js/WasmToJS.h: * wasm/js/WebAssemblyFunction.h: * yarr/YarrJIT.cpp: (JSC::Yarr::YarrGenerator::loadFromFrameAndJump): (JSC::Yarr::YarrGenerator::BacktrackingState::linkDataLabels): (JSC::Yarr::YarrGenerator::compile): * yarr/YarrJIT.h: (JSC::Yarr::YarrCodeBlock::set8BitCode): (JSC::Yarr::YarrCodeBlock::set16BitCode): (JSC::Yarr::YarrCodeBlock::set8BitCodeMatchOnly): (JSC::Yarr::YarrCodeBlock::set16BitCodeMatchOnly): (JSC::Yarr::YarrCodeBlock::execute): (JSC::Yarr::YarrCodeBlock::clear): Source/WebCore: No new tests. This is covered by existing tests. * WebCore.xcodeproj/project.pbxproj: * css/ElementRuleCollector.cpp: (WebCore::ElementRuleCollector::ruleMatches): * cssjit/CSSPtrTag.h: Added. * cssjit/CompiledSelector.h: * cssjit/FunctionCall.h: (WebCore::FunctionCall::FunctionCall): (WebCore::FunctionCall::setFunctionAddress): (WebCore::FunctionCall::prepareAndCall): * cssjit/SelectorCompiler.cpp: (WebCore::SelectorCompiler::compileSelector): (WebCore::SelectorCompiler::SelectorFragment::appendUnoptimizedPseudoClassWithContext): (WebCore::SelectorCompiler::addPseudoClassType): (WebCore::SelectorCompiler::SelectorCodeGenerator::compile): (WebCore::SelectorCompiler::SelectorCodeGenerator::generateElementAttributeFunctionCallValueMatching): (WebCore::SelectorCompiler::SelectorCodeGenerator::generateElementFunctionCallTest): (WebCore::SelectorCompiler::SelectorCodeGenerator::generateContextFunctionCallTest): * cssjit/SelectorCompiler.h: (WebCore::SelectorCompiler::ruleCollectorSimpleSelectorCheckerFunction): (WebCore::SelectorCompiler::querySelectorSimpleSelectorCheckerFunction): (WebCore::SelectorCompiler::ruleCollectorSelectorCheckerFunctionWithCheckingContext): (WebCore::SelectorCompiler::querySelectorSelectorCheckerFunctionWithCheckingContext): * dom/SelectorQuery.cpp: (WebCore::SelectorDataList::executeCompiledSingleMultiSelectorData const): (WebCore::SelectorDataList::execute const): * dom/SelectorQuery.h: Canonical link: https://commits.webkit.org/200234@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@230748 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2018-04-18 03:31:09 +00:00
MacroAssemblerCodeRef<JITThunkPtrTag> thunk =
[JSC] Figure out missing prepareCallOperation https://bugs.webkit.org/show_bug.cgi?id=203285 Reviewed by Mark Lam. Source/JavaScriptCore: We start using __builtin_frame_address to get CallFrame* in JIT operations. For the platform which is not supporting this API (MSVC), we put frame-pointer to vm.topCallFrame in the caller side. The problem is that all Apple platform is now using __builtin_frame_address, and we are not testing vm.topCallFrame version at all. To find missing prepareCallOperation call, we introduce JITOperationPrologueCallFrameTracer. When USE(BUILTIN_FRAME_ADDRESS) is enabled and if it is debug build, we anyway put frame-pointer to vm.topCallFrame. And after that, we ensure that vm.topCallFrame is the same to the CallFrame* gained by __builtin_frame_address. By doing this, we can find places missing this call in debug build of Apple ports. We also found that FTL's custom getter calling is putting wrong value to vm.topCallFrame. This patch fixes it too. * dfg/DFGOSRExit.cpp: (JSC::DFG::OSRExit::emitRestoreArguments): (JSC::DFG::operationCompileOSRExit): (JSC::DFG::OSRExit::compileExit): (JSC::DFG::operationDebugPrintSpeculationFailure): (JSC::DFG::OSRExit::compileOSRExit): Deleted. (JSC::DFG::OSRExit::debugOperationPrintSpeculationFailure): Deleted. * dfg/DFGOSRExit.h: * dfg/DFGOSRExitCompilerCommon.cpp: (JSC::DFG::handleExitCounts): (JSC::DFG::osrWriteBarrier): * dfg/DFGOSRExitCompilerCommon.h: * dfg/DFGOperations.cpp: * dfg/DFGOperations.h: * dfg/DFGSpeculativeJIT64.cpp: (JSC::DFG::SpeculativeJIT::compile): * dfg/DFGThunks.cpp: (JSC::DFG::osrExitThunkGenerator): (JSC::DFG::osrExitGenerationThunkGenerator): * ftl/FTLLowerDFGToB3.cpp: (JSC::FTL::DFG::LowerDFGToB3::compileCallOrConstructVarargsSpread): (JSC::FTL::DFG::LowerDFGToB3::compileCallOrConstructVarargs): (JSC::FTL::DFG::LowerDFGToB3::compileCallEval): (JSC::FTL::DFG::LowerDFGToB3::compileCallDOMGetter): (JSC::FTL::DFG::LowerDFGToB3::callPreflight): (JSC::FTL::DFG::LowerDFGToB3::callCheck): * ftl/FTLOSRExitCompiler.cpp: (JSC::FTL::compileStub): (JSC::FTL::operationCompileFTLOSRExit): (JSC::FTL::compileFTLOSRExit): Deleted. * ftl/FTLOSRExitCompiler.h: * ftl/FTLOperations.cpp: (JSC::FTL::operationPopulateObjectInOSR): (JSC::FTL::operationMaterializeObjectInOSR): (JSC::FTL::operationCompileFTLLazySlowPath): (JSC::FTL::compileFTLLazySlowPath): Deleted. * ftl/FTLOperations.h: * ftl/FTLSlowPathCall.cpp: (JSC::FTL::SlowPathCallContext::makeCall): * ftl/FTLThunks.cpp: (JSC::FTL::genericGenerationThunkGenerator): (JSC::FTL::osrExitGenerationThunkGenerator): (JSC::FTL::lazySlowPathGenerationThunkGenerator): (JSC::FTL::slowPathCallThunkGenerator): * ftl/FTLThunks.h: (JSC::FTL::generateIfNecessary): (JSC::FTL::Thunks::getSlowPathCallThunk): * interpreter/FrameTracers.h: (JSC::SlowPathFrameTracer::SlowPathFrameTracer): (JSC::JITOperationPrologueCallFrameTracer::JITOperationPrologueCallFrameTracer): (JSC::JITOperationPrologueCallFrameTracer::~JITOperationPrologueCallFrameTracer): * jit/AssemblyHelpers.cpp: (JSC::AssemblyHelpers::callExceptionFuzz): (JSC::AssemblyHelpers::debugCall): * jit/AssemblyHelpers.h: (JSC::AssemblyHelpers::prepareCallOperation): * jit/CCallHelpers.cpp: (JSC::CCallHelpers::ensureShadowChickenPacket): * jit/CCallHelpers.h: (JSC::CCallHelpers::prepareCallOperation): Deleted. * jit/JITOperations.cpp: * jit/JITOperations.h: * jit/Repatch.cpp: (JSC::ftlThunkAwareRepatchCall): * jit/ThunkGenerators.cpp: (JSC::boundThisNoArgsFunctionCallGenerator): * llint/LLIntSlowPaths.cpp: (JSC::LLInt::LLINT_SLOW_PATH_DECL): (JSC::LLInt::handleHostCall): * runtime/AtomicsObject.cpp: (JSC::operationAtomicsAdd): (JSC::operationAtomicsAnd): (JSC::operationAtomicsCompareExchange): (JSC::operationAtomicsExchange): (JSC::operationAtomicsIsLockFree): (JSC::operationAtomicsLoad): (JSC::operationAtomicsOr): (JSC::operationAtomicsStore): (JSC::operationAtomicsSub): (JSC::operationAtomicsXor): * runtime/CommonSlowPaths.cpp: (JSC::SLOW_PATH_DECL): * runtime/StringPrototype.cpp: (JSC::operationStringProtoFuncReplaceRegExpEmptyStr): (JSC::operationStringProtoFuncReplaceRegExpString): (JSC::operationStringProtoFuncReplaceGeneric): * tools/JSDollarVM.cpp: (IGNORE_WARNINGS_BEGIN): * wasm/WasmAirIRGenerator.cpp: (JSC::Wasm::AirIRGenerator::emitLoopTierUpCheck): * wasm/WasmB3IRGenerator.cpp: (JSC::Wasm::B3IRGenerator::emitLoopTierUpCheck): * wasm/WasmOperations.cpp: (JSC::Wasm::operationWasmThrowBadI64): (JSC::Wasm::operationWasmTriggerOSREntryNow): (JSC::Wasm::operationWasmTriggerTierUpNow): (JSC::Wasm::operationThrowBadI64): Deleted. (JSC::Wasm::triggerOSREntryNow): Deleted. (JSC::Wasm::triggerTierUpNow): Deleted. * wasm/WasmOperations.h: * wasm/WasmThunks.cpp: (JSC::Wasm::triggerOMGEntryTierUpThunkGenerator): * wasm/js/JSWebAssembly.cpp: (JSC::instantiate): * wasm/js/WasmToJS.cpp: (JSC::Wasm::handleBadI64Use): (JSC::Wasm::operationWasmToJSException): (JSC::Wasm::emitThrowWasmToJSException): (JSC::Wasm::wasmToJSException): Deleted. * wasm/js/WasmToJS.h: * wasm/js/WebAssemblyInstanceConstructor.cpp: (JSC::constructJSWebAssemblyInstance): Source/WebCore: Use JITOperationPrologueCallFrameTracer instead of NativeCallFrameTracer. * bindings/scripts/CodeGeneratorJS.pm: (GenerateOperationDefinition): * bindings/scripts/test/JS/JSTestDOMJIT.cpp: (WebCore::jsTestDOMJITPrototypeFunctionGetAttributeWithoutTypeCheck): (WebCore::jsTestDOMJITPrototypeFunctionItemWithoutTypeCheck): (WebCore::jsTestDOMJITPrototypeFunctionHasAttributeWithoutTypeCheck): (WebCore::jsTestDOMJITPrototypeFunctionGetElementByIdWithoutTypeCheck): (WebCore::jsTestDOMJITPrototypeFunctionGetElementsByNameWithoutTypeCheck): * domjit/DOMJITHelpers.h: (WebCore::DOMJIT::toWrapperSlow): Source/WTF: Enable USE(BUILTIN_FRAME_ADDRESS) regardless of platform is the compilers and architectures match. * wtf/Platform.h: Canonical link: https://commits.webkit.org/216753@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@251518 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2019-10-24 01:00:07 +00:00
vm.ftlThunks->getSlowPathCallThunk(vm, result.key());
FTL should generate code to call slow paths lazily https://bugs.webkit.org/show_bug.cgi?id=149936 Reviewed by Saam Barati. Source/JavaScriptCore: We often have complex slow paths in FTL-generated code. Those slow paths may never run. Even if they do run, they don't need stellar performance. So, it doesn't make sense to have LLVM worry about compiling such slow path code. This patch enables us to use our own MacroAssembler for compiling the slow path inside FTL code. It does this by using a crazy lambda thingy (see FTLLowerDFGToLLVM.cpp's lazySlowPath() and its documentation). The result is quite natural to use. Even for straight slow path calls via something like vmCall(), the lazySlowPath offers the benefit that the call marshalling and the exception checking are not expressed using LLVM IR and do not require LLVM to think about it. It also has the benefit that we never generate the code if it never runs. That's great, since function calls usually involve ~10 instructions total (move arguments to argument registers, make the call, check exception, etc.). This patch adds the lazy slow path abstraction and uses it for some slow paths in the FTL. The code we generate with lazy slow paths is worse than the code that LLVM would have generated. Therefore, a lazy slow path only makes sense when we have strong evidence that the slow path will execute infrequently relative to the fast path. This completely precludes the use of lazy slow paths for out-of-line Nodes that unconditionally call a C++ function. It also precludes their use for the GetByVal out-of-bounds handler, since when we generate a GetByVal with an out-of-bounds handler it means that we only know that the out-of-bounds case executed at least once. So, for all we know, it may actually be the common case. So, this patch just deployed the lazy slow path for GC slow paths and masquerades-as-undefined slow paths. It makes sense for GC slow paths because those have a statistical guarantee of slow path frequency - probably bounded at less than 1/10. It makes sense for masquerades-as- undefined because we can say quite confidently that this is an uncommon scenario on the modern Web. Something that's always been challenging about abstractions involving the MacroAssembler is that linking is a separate phase, and there is no way for someone who is just given access to the MacroAssembler& to emit code that requires linking, since linking happens once we have emitted all code and we are creating the LinkBuffer. Moreover, the FTL requires that the final parts of linking happen on the main thread. This patch ran into this issue, and solved it comprehensively, by introducing MacroAssembler::addLinkTask(). This takes a lambda and runs it at the bitter end of linking - when performFinalization() is called. This ensure that the task added by addLinkTask() runs on the main thread. This patch doesn't replace all of the previously existing idioms for dealing with this issue; we can do that later. This shows small speed-ups on a bunch of things. No big win on any benchmark aggregate. But mainly this is done for https://bugs.webkit.org/show_bug.cgi?id=149852, where we found that outlining the slow path in this way was a significant speed boost. * CMakeLists.txt: * JavaScriptCore.vcxproj/JavaScriptCore.vcxproj: * JavaScriptCore.xcodeproj/project.pbxproj: * assembler/AbstractMacroAssembler.h: (JSC::AbstractMacroAssembler::replaceWithAddressComputation): (JSC::AbstractMacroAssembler::addLinkTask): (JSC::AbstractMacroAssembler::AbstractMacroAssembler): * assembler/LinkBuffer.cpp: (JSC::LinkBuffer::linkCode): (JSC::LinkBuffer::allocate): (JSC::LinkBuffer::performFinalization): * assembler/LinkBuffer.h: (JSC::LinkBuffer::wasAlreadyDisassembled): (JSC::LinkBuffer::didAlreadyDisassemble): (JSC::LinkBuffer::vm): (JSC::LinkBuffer::executableOffsetFor): * bytecode/CodeOrigin.h: (JSC::CodeOrigin::CodeOrigin): (JSC::CodeOrigin::isSet): (JSC::CodeOrigin::operator bool): (JSC::CodeOrigin::isHashTableDeletedValue): (JSC::CodeOrigin::operator!): Deleted. * ftl/FTLCompile.cpp: (JSC::FTL::mmAllocateDataSection): * ftl/FTLInlineCacheDescriptor.h: (JSC::FTL::InlineCacheDescriptor::InlineCacheDescriptor): (JSC::FTL::CheckInDescriptor::CheckInDescriptor): (JSC::FTL::LazySlowPathDescriptor::LazySlowPathDescriptor): * ftl/FTLJITCode.h: * ftl/FTLJITFinalizer.cpp: (JSC::FTL::JITFinalizer::finalizeFunction): * ftl/FTLJITFinalizer.h: * ftl/FTLLazySlowPath.cpp: Added. (JSC::FTL::LazySlowPath::LazySlowPath): (JSC::FTL::LazySlowPath::~LazySlowPath): (JSC::FTL::LazySlowPath::generate): * ftl/FTLLazySlowPath.h: Added. (JSC::FTL::LazySlowPath::createGenerator): (JSC::FTL::LazySlowPath::patchpoint): (JSC::FTL::LazySlowPath::usedRegisters): (JSC::FTL::LazySlowPath::callSiteIndex): (JSC::FTL::LazySlowPath::stub): * ftl/FTLLazySlowPathCall.h: Added. (JSC::FTL::createLazyCallGenerator): * ftl/FTLLowerDFGToLLVM.cpp: (JSC::FTL::DFG::LowerDFGToLLVM::compileCreateActivation): (JSC::FTL::DFG::LowerDFGToLLVM::compileNewFunction): (JSC::FTL::DFG::LowerDFGToLLVM::compileCreateDirectArguments): (JSC::FTL::DFG::LowerDFGToLLVM::compileNewArrayWithSize): (JSC::FTL::DFG::LowerDFGToLLVM::compileMakeRope): (JSC::FTL::DFG::LowerDFGToLLVM::compileNotifyWrite): (JSC::FTL::DFG::LowerDFGToLLVM::compileIsObjectOrNull): (JSC::FTL::DFG::LowerDFGToLLVM::compileIsFunction): (JSC::FTL::DFG::LowerDFGToLLVM::compileIn): (JSC::FTL::DFG::LowerDFGToLLVM::compileMaterializeNewObject): (JSC::FTL::DFG::LowerDFGToLLVM::compileMaterializeCreateActivation): (JSC::FTL::DFG::LowerDFGToLLVM::compileCheckWatchdogTimer): (JSC::FTL::DFG::LowerDFGToLLVM::allocatePropertyStorageWithSizeImpl): (JSC::FTL::DFG::LowerDFGToLLVM::allocateObject): (JSC::FTL::DFG::LowerDFGToLLVM::allocateJSArray): (JSC::FTL::DFG::LowerDFGToLLVM::buildTypeOf): (JSC::FTL::DFG::LowerDFGToLLVM::sensibleDoubleToInt32): (JSC::FTL::DFG::LowerDFGToLLVM::lazySlowPath): (JSC::FTL::DFG::LowerDFGToLLVM::speculate): (JSC::FTL::DFG::LowerDFGToLLVM::emitStoreBarrier): * ftl/FTLOperations.cpp: (JSC::FTL::operationMaterializeObjectInOSR): (JSC::FTL::compileFTLLazySlowPath): * ftl/FTLOperations.h: * ftl/FTLSlowPathCall.cpp: (JSC::FTL::SlowPathCallContext::SlowPathCallContext): (JSC::FTL::SlowPathCallContext::~SlowPathCallContext): (JSC::FTL::SlowPathCallContext::keyWithTarget): (JSC::FTL::SlowPathCallContext::makeCall): (JSC::FTL::callSiteIndexForCodeOrigin): (JSC::FTL::storeCodeOrigin): Deleted. (JSC::FTL::callOperation): Deleted. * ftl/FTLSlowPathCall.h: (JSC::FTL::callOperation): * ftl/FTLState.h: * ftl/FTLThunks.cpp: (JSC::FTL::genericGenerationThunkGenerator): (JSC::FTL::osrExitGenerationThunkGenerator): (JSC::FTL::lazySlowPathGenerationThunkGenerator): (JSC::FTL::registerClobberCheck): * ftl/FTLThunks.h: * interpreter/CallFrame.h: (JSC::CallSiteIndex::CallSiteIndex): (JSC::CallSiteIndex::operator bool): (JSC::CallSiteIndex::bits): * jit/CCallHelpers.h: (JSC::CCallHelpers::setupArgument): (JSC::CCallHelpers::setupArgumentsWithExecState): * jit/JITOperations.cpp: Source/WTF: Enables SharedTask to handle any function type, not just void(). It's probably better to use SharedTask instead of std::function in performance-sensitive code. std::function uses the system malloc and has copy semantics. SharedTask uses FastMalloc and has aliasing semantics. So, you can just trust that it will have sensible performance characteristics. * wtf/ParallelHelperPool.cpp: (WTF::ParallelHelperClient::~ParallelHelperClient): (WTF::ParallelHelperClient::setTask): (WTF::ParallelHelperClient::doSomeHelping): (WTF::ParallelHelperClient::runTaskInParallel): (WTF::ParallelHelperClient::finish): (WTF::ParallelHelperClient::claimTask): (WTF::ParallelHelperClient::runTask): (WTF::ParallelHelperPool::doSomeHelping): (WTF::ParallelHelperPool::helperThreadBody): * wtf/ParallelHelperPool.h: (WTF::ParallelHelperClient::setFunction): (WTF::ParallelHelperClient::runFunctionInParallel): (WTF::ParallelHelperClient::pool): * wtf/SharedTask.h: (WTF::createSharedTask): (WTF::SharedTask::SharedTask): Deleted. (WTF::SharedTask::~SharedTask): Deleted. (WTF::SharedTaskFunctor::SharedTaskFunctor): Deleted. Canonical link: https://commits.webkit.org/168154@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@190860 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2015-10-12 17:56:26 +00:00
Unreviewed, reland r268170 https://bugs.webkit.org/show_bug.cgi?id=217460 Source/JavaScriptCore: Fixed missing wrong OperationPtrTag use in Repatch.cpp. * assembler/AbstractMacroAssembler.h: (JSC::AbstractMacroAssembler::getLinkerAddress): * assembler/AssemblerBuffer.h: (JSC::ARM64EHash::update): (JSC::ARM64EHash::finalHash const): * assembler/JITOperationList.cpp: (JSC::addPointers): * assembler/MacroAssemblerARM64.cpp: (JSC::MacroAssembler::probe): * assembler/MacroAssemblerCodeRef.h: (JSC::MacroAssemblerCodePtr::MacroAssemblerCodePtr): (JSC::MacroAssemblerCodePtr::createFromExecutableAddress): * assembler/testmasm.cpp: (JSC::testProbeModifiesProgramCounter): * b3/air/testair.cpp: * ftl/FTLOutput.h: (JSC::FTL::Output::callWithoutSideEffects): (JSC::FTL::Output::operation): * ftl/FTLSlowPathCall.cpp: (JSC::FTL::SlowPathCallContext::makeCall): * jit/JITCode.cpp: (JSC::JITCodeWithCodeRef::executableAddressAtOffset): * jit/JITExceptions.cpp: (JSC::genericUnwind): * jit/JITOperations.cpp: * jit/Repatch.cpp: (JSC::readPutICCallTarget): (JSC::ftlThunkAwareRepatchCall): (JSC::tryCacheGetBy): (JSC::tryCachePutByID): * llint/LLIntData.cpp: (JSC::LLInt::initialize): * llint/LLIntPCRanges.h: (JSC::LLInt::isLLIntPC): * llint/LLIntSlowPaths.cpp: (JSC::LLInt::setUpCall): * llint/LLIntThunks.cpp: (JSC::LLInt::generateThunkWithJumpTo): * runtime/JSCPtrTag.h: * runtime/MachineContext.h: (JSC::MachineContext::instructionPointer): * runtime/NativeExecutable.cpp: (JSC::NativeExecutable::finishCreation): * runtime/PutPropertySlot.h: (JSC::PutPropertySlot::setCustomValue): (JSC::PutPropertySlot::setCustomAccessor): (JSC::PutPropertySlot::customSetter const): * wasm/WasmAirIRGenerator.cpp: (JSC::Wasm::AirIRGenerator::emitCCall): * wasm/WasmSlowPaths.cpp: Source/WTF: * wtf/PlatformRegisters.cpp: (WTF::threadStateLRInternal): (WTF::threadStatePCInternal): * wtf/PtrTag.h: (WTF::tagCodePtr): (WTF::untagCodePtr): (WTF::assertIsCFunctionPtr): (WTF::assertIsNullOrCFunctionPtr): (WTF::assertIsNotTagged): (WTF::assertIsTagged): (WTF::assertIsNullOrTagged): (WTF::isTaggedWith): (WTF::assertIsTaggedWith): (WTF::assertIsNullOrTaggedWith): (WTF::tagCFunctionPtr): (WTF::tagCFunction): (WTF::untagCFunctionPtr): (WTF::tagInt): Canonical link: https://commits.webkit.org/230280@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@268247 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2020-10-09 05:08:11 +00:00
linkBuffer.link(result.call(), CodeLocationLabel<JITThunkPtrTag>(thunk.code()));
FTL should generate code to call slow paths lazily https://bugs.webkit.org/show_bug.cgi?id=149936 Reviewed by Saam Barati. Source/JavaScriptCore: We often have complex slow paths in FTL-generated code. Those slow paths may never run. Even if they do run, they don't need stellar performance. So, it doesn't make sense to have LLVM worry about compiling such slow path code. This patch enables us to use our own MacroAssembler for compiling the slow path inside FTL code. It does this by using a crazy lambda thingy (see FTLLowerDFGToLLVM.cpp's lazySlowPath() and its documentation). The result is quite natural to use. Even for straight slow path calls via something like vmCall(), the lazySlowPath offers the benefit that the call marshalling and the exception checking are not expressed using LLVM IR and do not require LLVM to think about it. It also has the benefit that we never generate the code if it never runs. That's great, since function calls usually involve ~10 instructions total (move arguments to argument registers, make the call, check exception, etc.). This patch adds the lazy slow path abstraction and uses it for some slow paths in the FTL. The code we generate with lazy slow paths is worse than the code that LLVM would have generated. Therefore, a lazy slow path only makes sense when we have strong evidence that the slow path will execute infrequently relative to the fast path. This completely precludes the use of lazy slow paths for out-of-line Nodes that unconditionally call a C++ function. It also precludes their use for the GetByVal out-of-bounds handler, since when we generate a GetByVal with an out-of-bounds handler it means that we only know that the out-of-bounds case executed at least once. So, for all we know, it may actually be the common case. So, this patch just deployed the lazy slow path for GC slow paths and masquerades-as-undefined slow paths. It makes sense for GC slow paths because those have a statistical guarantee of slow path frequency - probably bounded at less than 1/10. It makes sense for masquerades-as- undefined because we can say quite confidently that this is an uncommon scenario on the modern Web. Something that's always been challenging about abstractions involving the MacroAssembler is that linking is a separate phase, and there is no way for someone who is just given access to the MacroAssembler& to emit code that requires linking, since linking happens once we have emitted all code and we are creating the LinkBuffer. Moreover, the FTL requires that the final parts of linking happen on the main thread. This patch ran into this issue, and solved it comprehensively, by introducing MacroAssembler::addLinkTask(). This takes a lambda and runs it at the bitter end of linking - when performFinalization() is called. This ensure that the task added by addLinkTask() runs on the main thread. This patch doesn't replace all of the previously existing idioms for dealing with this issue; we can do that later. This shows small speed-ups on a bunch of things. No big win on any benchmark aggregate. But mainly this is done for https://bugs.webkit.org/show_bug.cgi?id=149852, where we found that outlining the slow path in this way was a significant speed boost. * CMakeLists.txt: * JavaScriptCore.vcxproj/JavaScriptCore.vcxproj: * JavaScriptCore.xcodeproj/project.pbxproj: * assembler/AbstractMacroAssembler.h: (JSC::AbstractMacroAssembler::replaceWithAddressComputation): (JSC::AbstractMacroAssembler::addLinkTask): (JSC::AbstractMacroAssembler::AbstractMacroAssembler): * assembler/LinkBuffer.cpp: (JSC::LinkBuffer::linkCode): (JSC::LinkBuffer::allocate): (JSC::LinkBuffer::performFinalization): * assembler/LinkBuffer.h: (JSC::LinkBuffer::wasAlreadyDisassembled): (JSC::LinkBuffer::didAlreadyDisassemble): (JSC::LinkBuffer::vm): (JSC::LinkBuffer::executableOffsetFor): * bytecode/CodeOrigin.h: (JSC::CodeOrigin::CodeOrigin): (JSC::CodeOrigin::isSet): (JSC::CodeOrigin::operator bool): (JSC::CodeOrigin::isHashTableDeletedValue): (JSC::CodeOrigin::operator!): Deleted. * ftl/FTLCompile.cpp: (JSC::FTL::mmAllocateDataSection): * ftl/FTLInlineCacheDescriptor.h: (JSC::FTL::InlineCacheDescriptor::InlineCacheDescriptor): (JSC::FTL::CheckInDescriptor::CheckInDescriptor): (JSC::FTL::LazySlowPathDescriptor::LazySlowPathDescriptor): * ftl/FTLJITCode.h: * ftl/FTLJITFinalizer.cpp: (JSC::FTL::JITFinalizer::finalizeFunction): * ftl/FTLJITFinalizer.h: * ftl/FTLLazySlowPath.cpp: Added. (JSC::FTL::LazySlowPath::LazySlowPath): (JSC::FTL::LazySlowPath::~LazySlowPath): (JSC::FTL::LazySlowPath::generate): * ftl/FTLLazySlowPath.h: Added. (JSC::FTL::LazySlowPath::createGenerator): (JSC::FTL::LazySlowPath::patchpoint): (JSC::FTL::LazySlowPath::usedRegisters): (JSC::FTL::LazySlowPath::callSiteIndex): (JSC::FTL::LazySlowPath::stub): * ftl/FTLLazySlowPathCall.h: Added. (JSC::FTL::createLazyCallGenerator): * ftl/FTLLowerDFGToLLVM.cpp: (JSC::FTL::DFG::LowerDFGToLLVM::compileCreateActivation): (JSC::FTL::DFG::LowerDFGToLLVM::compileNewFunction): (JSC::FTL::DFG::LowerDFGToLLVM::compileCreateDirectArguments): (JSC::FTL::DFG::LowerDFGToLLVM::compileNewArrayWithSize): (JSC::FTL::DFG::LowerDFGToLLVM::compileMakeRope): (JSC::FTL::DFG::LowerDFGToLLVM::compileNotifyWrite): (JSC::FTL::DFG::LowerDFGToLLVM::compileIsObjectOrNull): (JSC::FTL::DFG::LowerDFGToLLVM::compileIsFunction): (JSC::FTL::DFG::LowerDFGToLLVM::compileIn): (JSC::FTL::DFG::LowerDFGToLLVM::compileMaterializeNewObject): (JSC::FTL::DFG::LowerDFGToLLVM::compileMaterializeCreateActivation): (JSC::FTL::DFG::LowerDFGToLLVM::compileCheckWatchdogTimer): (JSC::FTL::DFG::LowerDFGToLLVM::allocatePropertyStorageWithSizeImpl): (JSC::FTL::DFG::LowerDFGToLLVM::allocateObject): (JSC::FTL::DFG::LowerDFGToLLVM::allocateJSArray): (JSC::FTL::DFG::LowerDFGToLLVM::buildTypeOf): (JSC::FTL::DFG::LowerDFGToLLVM::sensibleDoubleToInt32): (JSC::FTL::DFG::LowerDFGToLLVM::lazySlowPath): (JSC::FTL::DFG::LowerDFGToLLVM::speculate): (JSC::FTL::DFG::LowerDFGToLLVM::emitStoreBarrier): * ftl/FTLOperations.cpp: (JSC::FTL::operationMaterializeObjectInOSR): (JSC::FTL::compileFTLLazySlowPath): * ftl/FTLOperations.h: * ftl/FTLSlowPathCall.cpp: (JSC::FTL::SlowPathCallContext::SlowPathCallContext): (JSC::FTL::SlowPathCallContext::~SlowPathCallContext): (JSC::FTL::SlowPathCallContext::keyWithTarget): (JSC::FTL::SlowPathCallContext::makeCall): (JSC::FTL::callSiteIndexForCodeOrigin): (JSC::FTL::storeCodeOrigin): Deleted. (JSC::FTL::callOperation): Deleted. * ftl/FTLSlowPathCall.h: (JSC::FTL::callOperation): * ftl/FTLState.h: * ftl/FTLThunks.cpp: (JSC::FTL::genericGenerationThunkGenerator): (JSC::FTL::osrExitGenerationThunkGenerator): (JSC::FTL::lazySlowPathGenerationThunkGenerator): (JSC::FTL::registerClobberCheck): * ftl/FTLThunks.h: * interpreter/CallFrame.h: (JSC::CallSiteIndex::CallSiteIndex): (JSC::CallSiteIndex::operator bool): (JSC::CallSiteIndex::bits): * jit/CCallHelpers.h: (JSC::CCallHelpers::setupArgument): (JSC::CCallHelpers::setupArgumentsWithExecState): * jit/JITOperations.cpp: Source/WTF: Enables SharedTask to handle any function type, not just void(). It's probably better to use SharedTask instead of std::function in performance-sensitive code. std::function uses the system malloc and has copy semantics. SharedTask uses FastMalloc and has aliasing semantics. So, you can just trust that it will have sensible performance characteristics. * wtf/ParallelHelperPool.cpp: (WTF::ParallelHelperClient::~ParallelHelperClient): (WTF::ParallelHelperClient::setTask): (WTF::ParallelHelperClient::doSomeHelping): (WTF::ParallelHelperClient::runTaskInParallel): (WTF::ParallelHelperClient::finish): (WTF::ParallelHelperClient::claimTask): (WTF::ParallelHelperClient::runTask): (WTF::ParallelHelperPool::doSomeHelping): (WTF::ParallelHelperPool::helperThreadBody): * wtf/ParallelHelperPool.h: (WTF::ParallelHelperClient::setFunction): (WTF::ParallelHelperClient::runFunctionInParallel): (WTF::ParallelHelperClient::pool): * wtf/SharedTask.h: (WTF::createSharedTask): (WTF::SharedTask::SharedTask): Deleted. (WTF::SharedTask::~SharedTask): Deleted. (WTF::SharedTaskFunctor::SharedTaskFunctor): Deleted. Canonical link: https://commits.webkit.org/168154@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@190860 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2015-10-12 17:56:26 +00:00
});
return result;
FTL should be able to do some simple inline caches using LLVM patchpoints https://bugs.webkit.org/show_bug.cgi?id=123164 Source/JavaScriptCore: Reviewed by Mark Hahnenberg. This implements GetById inline caches in the FTL using llvm.webkit.patchpoint. The idea is that we ask LLVM for a nop slide the size of a GetById inline cache and then fill in the code after LLVM compilation is complete. For now, we just use the system calling convention for the arguments and return. We also still make some assumptions about registers that aren't correct. But, most of the scaffolding is there and this will successfully patch an inline cache. * JavaScriptCore.xcodeproj/project.pbxproj: * assembler/AbstractMacroAssembler.h: * assembler/LinkBuffer.cpp: (JSC::LinkBuffer::finalizeCodeWithoutDisassembly): (JSC::LinkBuffer::linkCode): (JSC::LinkBuffer::allocate): * assembler/LinkBuffer.h: (JSC::LinkBuffer::LinkBuffer): (JSC::LinkBuffer::link): * ftl/FTLAbbreviations.h: (JSC::FTL::constNull): (JSC::FTL::buildCall): * ftl/FTLCapabilities.cpp: (JSC::FTL::canCompile): * ftl/FTLCompile.cpp: (JSC::FTL::fixFunctionBasedOnStackMaps): * ftl/FTLInlineCacheDescriptor.h: Added. (JSC::FTL::InlineCacheDescriptor::InlineCacheDescriptor): (JSC::FTL::GetByIdDescriptor::GetByIdDescriptor): (JSC::FTL::GetByIdDescriptor::stackmapID): (JSC::FTL::GetByIdDescriptor::codeOrigin): (JSC::FTL::GetByIdDescriptor::uid): * ftl/FTLInlineCacheSize.cpp: Added. (JSC::FTL::sizeOfGetById): (JSC::FTL::sizeOfPutById): * ftl/FTLInlineCacheSize.h: Added. * ftl/FTLIntrinsicRepository.h: * ftl/FTLJITFinalizer.cpp: (JSC::FTL::JITFinalizer::finalizeFunction): * ftl/FTLJITFinalizer.h: * ftl/FTLLocation.cpp: (JSC::FTL::Location::directGPR): * ftl/FTLLocation.h: * ftl/FTLLowerDFGToLLVM.cpp: (JSC::FTL::LowerDFGToLLVM::compileGetById): * ftl/FTLOutput.h: (JSC::FTL::Output::call): * ftl/FTLSlowPathCall.cpp: Added. (JSC::FTL::callOperation): * ftl/FTLSlowPathCall.h: Added. (JSC::FTL::SlowPathCall::SlowPathCall): (JSC::FTL::SlowPathCall::call): (JSC::FTL::SlowPathCall::key): * ftl/FTLSlowPathCallKey.cpp: Added. (JSC::FTL::SlowPathCallKey::dump): * ftl/FTLSlowPathCallKey.h: Added. (JSC::FTL::SlowPathCallKey::SlowPathCallKey): (JSC::FTL::SlowPathCallKey::usedRegisters): (JSC::FTL::SlowPathCallKey::callTarget): (JSC::FTL::SlowPathCallKey::offset): (JSC::FTL::SlowPathCallKey::isEmptyValue): (JSC::FTL::SlowPathCallKey::isDeletedValue): (JSC::FTL::SlowPathCallKey::operator==): (JSC::FTL::SlowPathCallKey::hash): (JSC::FTL::SlowPathCallKeyHash::hash): (JSC::FTL::SlowPathCallKeyHash::equal): * ftl/FTLStackMaps.cpp: (JSC::FTL::StackMaps::Location::directGPR): * ftl/FTLStackMaps.h: * ftl/FTLState.h: * ftl/FTLThunks.cpp: (JSC::FTL::slowPathCallThunkGenerator): * ftl/FTLThunks.h: (JSC::FTL::Thunks::getSlowPathCallThunk): * jit/CCallHelpers.h: (JSC::CCallHelpers::setupArguments): * jit/GPRInfo.h: * jit/JITInlineCacheGenerator.cpp: (JSC::garbageStubInfo): (JSC::JITInlineCacheGenerator::JITInlineCacheGenerator): (JSC::JITByIdGenerator::finalize): * jit/JITInlineCacheGenerator.h: (JSC::JITByIdGenerator::slowPathBegin): * jit/RegisterSet.cpp: (JSC::RegisterSet::stackRegisters): (JSC::RegisterSet::specialRegisters): (JSC::RegisterSet::calleeSaveRegisters): (JSC::RegisterSet::allGPRs): (JSC::RegisterSet::allFPRs): (JSC::RegisterSet::allRegisters): (JSC::RegisterSet::dump): * jit/RegisterSet.h: (JSC::RegisterSet::exclude): (JSC::RegisterSet::numberOfSetRegisters): (JSC::RegisterSet::RegisterSet): (JSC::RegisterSet::isEmptyValue): (JSC::RegisterSet::isDeletedValue): (JSC::RegisterSet::operator==): (JSC::RegisterSet::hash): (JSC::RegisterSetHash::hash): (JSC::RegisterSetHash::equal): * runtime/Options.h: Source/WTF: Reviewed by Mark Hahnenberg. This needed some better bitvector support, like merging (|=), excluding (&=~), hashing, and bit counting. * wtf/BitVector.cpp: (WTF::BitVector::setSlow): (WTF::BitVector::excludeSlow): (WTF::BitVector::bitCountSlow): (WTF::BitVector::equalsSlowCase): (WTF::BitVector::hashSlowCase): (WTF::BitVector::dump): * wtf/BitVector.h: (WTF::BitVector::merge): (WTF::BitVector::exclude): (WTF::BitVector::bitCount): (WTF::BitVector::BitVector): (WTF::BitVector::isEmptyValue): (WTF::BitVector::isDeletedValue): (WTF::BitVector::isEmptyOrDeletedValue): (WTF::BitVector::operator==): (WTF::BitVector::hash): (WTF::BitVectorHash::hash): (WTF::BitVectorHash::equal): * wtf/HashTraits.h: (WTF::CustomHashTraits::constructDeletedValue): (WTF::CustomHashTraits::isDeletedValue): (WTF::CustomHashTraits::emptyValue): (WTF::CustomHashTraits::isEmptyValue): * wtf/StdLibExtras.h: (WTF::bitCount): Canonical link: https://commits.webkit.org/141315@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@157872 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2013-10-23 18:22:09 +00:00
}
[JSC] Use DataIC for AccessCase https://bugs.webkit.org/show_bug.cgi?id=226072 Reviewed by Saam Barati and Filip Pizlo. Source/JavaScriptCore: This patch adds non-repatching IC for Baseline JIT in ARM64. This does not work in non-ARM64 architectures (including X64) due to the use of link-register. 1. We add non-repatching IC, which is enabled only in Baseline due to performance reason. We are using the existing IC in DFG and FTL. Non-repatching includes fast-path, and slow-path's operation function. 2. We still keep InlineAccess in all tiers. Removing that causes 0.3 ~ 1.0% regression in Speedometer2. This means that we still need some repatching when we first introduce stubs. 3. We add a mechanism to share generated code stubs in non-repatching IC. Currently, getter / setter / custom accessors are excluded since their code relies on JSGlobalObject, CodeBlock etc. which are not included in AccessCase's data structure. 4. This patch still relies on that CodeBlock will be destroyed synchronously since we need to ensure that sharing-hash-table does not include already-dead JIT code stubs. We can fix it (e.g. annotating epoch to these stubs, bump them in finalizeUnconditionally), but we avoid doing that to prevent from further enlarging this patch. This patch is already significant in its size. 5. Added callOperation(Address) support in CCallHelpers, which can save the target in nonArgGPR0, and call it so that we can use Address including GPR which is also used for arguments. Performance is neutral in JetStream2 and Speedometer2. But it offers the way to remove some code generation in Baseline. * assembler/MacroAssemblerARM64E.h: (JSC::MacroAssemblerARM64E::call): * bytecode/AccessCase.cpp: (JSC::AccessCase::create): (JSC::AccessCase::createTransition): (JSC::AccessCase::createDelete): (JSC::AccessCase::createCheckPrivateBrand): (JSC::AccessCase::createSetPrivateBrand): (JSC::AccessCase::fromStructureStubInfo): (JSC::AccessCase::clone const): (JSC::AccessCase::generateWithGuard): (JSC::AccessCase::generateImpl): (JSC::AccessCase::canBeShared): * bytecode/AccessCase.h: (JSC::AccessCase::hash const): (JSC::AccessCase::AccessCase): (JSC::SharedJITStubSet::Hash::Key::Key): (JSC::SharedJITStubSet::Hash::Key::isHashTableDeletedValue const): (JSC::SharedJITStubSet::Hash::Key::operator==): (JSC::SharedJITStubSet::Hash::hash): (JSC::SharedJITStubSet::Hash::equal): (JSC::SharedJITStubSet::Searcher::Translator::hash): (JSC::SharedJITStubSet::Searcher::Translator::equal): (JSC::SharedJITStubSet::PointerTranslator::hash): (JSC::SharedJITStubSet::PointerTranslator::equal): (JSC::SharedJITStubSet::add): (JSC::SharedJITStubSet::remove): (JSC::SharedJITStubSet::find): * bytecode/ByValInfo.h: (JSC::ByValInfo::setUp): (JSC::ByValInfo::offsetOfSlowOperation): (JSC::ByValInfo::offsetOfNotIndexJumpTarget): (JSC::ByValInfo::offsetOfBadTypeJumpTarget): * bytecode/CallLinkInfo.cpp: (JSC::CallLinkInfo::initializeDirectCall): (JSC::CallLinkInfo::setDirectCallTarget): * bytecode/CodeBlock.h: (JSC::CodeBlock::useDataIC const): * bytecode/GetterSetterAccessCase.cpp: (JSC::GetterSetterAccessCase::create): (JSC::GetterSetterAccessCase::clone const): (JSC::GetterSetterAccessCase::emitDOMJITGetter): * bytecode/GetterSetterAccessCase.h: * bytecode/InlineAccess.cpp: (JSC::getScratchRegister): (JSC::InlineAccess::rewireStubAsJumpInAccessNotUsingInlineAccess): (JSC::InlineAccess::rewireStubAsJumpInAccess): (JSC::InlineAccess::resetStubAsJumpInAccess): (JSC::InlineAccess::resetStubAsJumpInAccessNotUsingInlineAccess): (JSC::InlineAccess::rewireStubAsJump): Deleted. * bytecode/InlineAccess.h: * bytecode/InstanceOfAccessCase.cpp: (JSC::InstanceOfAccessCase::create): (JSC::InstanceOfAccessCase::clone const): * bytecode/InstanceOfAccessCase.h: * bytecode/IntrinsicGetterAccessCase.cpp: (JSC::IntrinsicGetterAccessCase::create): (JSC::IntrinsicGetterAccessCase::clone const): * bytecode/IntrinsicGetterAccessCase.h: * bytecode/ModuleNamespaceAccessCase.cpp: (JSC::ModuleNamespaceAccessCase::create): (JSC::ModuleNamespaceAccessCase::clone const): * bytecode/ModuleNamespaceAccessCase.h: * bytecode/ObjectPropertyConditionSet.h: (JSC::ObjectPropertyConditionSet::hash const): (JSC::ObjectPropertyConditionSet::operator==): (JSC::ObjectPropertyConditionSet::operator!=): * bytecode/PolymorphicAccess.cpp: (JSC::AccessGenerationState::installWatchpoint): (JSC::AccessGenerationState::succeed): (JSC::AccessGenerationState::preserveLiveRegistersToStackForCallWithoutExceptions): (JSC::PolymorphicAccess::addCases): (JSC::PolymorphicAccess::addCase): (JSC::PolymorphicAccess::visitWeak const): (JSC::PolymorphicAccess::regenerate): * bytecode/PolymorphicAccess.h: (JSC::AccessGenerationState::AccessGenerationState): (JSC::AccessGenerationState::preserveLiveRegistersToStackForCallWithoutExceptions): * bytecode/ProxyableAccessCase.cpp: (JSC::ProxyableAccessCase::create): (JSC::ProxyableAccessCase::clone const): * bytecode/ProxyableAccessCase.h: * bytecode/StructureStubInfo.cpp: (JSC::StructureStubInfo::StructureStubInfo): (JSC::StructureStubInfo::addAccessCase): (JSC::StructureStubInfo::reset): * bytecode/StructureStubInfo.h: (JSC::StructureStubInfo::offsetOfCodePtr): (JSC::StructureStubInfo::offsetOfSlowPathStartLocation): (JSC::StructureStubInfo::offsetOfSlowOperation): (JSC::StructureStubInfo::patchableJump): Deleted. * dfg/DFGJITCompiler.h: (JSC::DFG::JITCompiler::appendCall): * dfg/DFGSlowPathGenerator.h: (JSC::DFG::slowPathICCall): * dfg/DFGSpeculativeJIT.cpp: (JSC::DFG::SpeculativeJIT::compileGetById): (JSC::DFG::SpeculativeJIT::compileGetByIdFlush): (JSC::DFG::SpeculativeJIT::compileDeleteById): (JSC::DFG::SpeculativeJIT::compileDeleteByVal): (JSC::DFG::SpeculativeJIT::compileInById): (JSC::DFG::SpeculativeJIT::compileInByVal): (JSC::DFG::SpeculativeJIT::compileGetPrivateNameByVal): (JSC::DFG::SpeculativeJIT::compileGetPrivateNameById): (JSC::DFG::SpeculativeJIT::compilePutPrivateNameById): (JSC::DFG::SpeculativeJIT::compileCheckPrivateBrand): (JSC::DFG::SpeculativeJIT::compileSetPrivateBrand): (JSC::DFG::SpeculativeJIT::compileInstanceOfForCells): (JSC::DFG::SpeculativeJIT::compileInstanceOf): (JSC::DFG::SpeculativeJIT::compilePutByIdFlush): (JSC::DFG::SpeculativeJIT::compilePutById): (JSC::DFG::SpeculativeJIT::compilePutByIdDirect): (JSC::DFG::SpeculativeJIT::cachedPutById): * dfg/DFGSpeculativeJIT.h: (JSC::DFG::SpeculativeJIT::callOperation): (JSC::DFG::SpeculativeJIT::appendCall): (JSC::DFG::SpeculativeJIT::appendCallSetResult): * dfg/DFGSpeculativeJIT32_64.cpp: (JSC::DFG::SpeculativeJIT::cachedGetById): (JSC::DFG::SpeculativeJIT::cachedGetByIdWithThis): (JSC::DFG::SpeculativeJIT::compile): * dfg/DFGSpeculativeJIT64.cpp: (JSC::DFG::SpeculativeJIT::cachedGetById): (JSC::DFG::SpeculativeJIT::cachedGetByIdWithThis): (JSC::DFG::SpeculativeJIT::compile): * ftl/FTLLowerDFGToB3.cpp: (JSC::FTL::DFG::LowerDFGToB3::getPrivateName): (JSC::FTL::DFG::LowerDFGToB3::compilePrivateBrandAccess): (JSC::FTL::DFG::LowerDFGToB3::cachedPutById): (JSC::FTL::DFG::LowerDFGToB3::compileGetByVal): (JSC::FTL::DFG::LowerDFGToB3::compileDelBy): (JSC::FTL::DFG::LowerDFGToB3::compileCompareStrictEq): * ftl/FTLSlowPathCall.cpp: (JSC::FTL::SlowPathCallContext::SlowPathCallContext): (JSC::FTL::SlowPathCallContext::keyWithTarget const): (JSC::FTL::SlowPathCallContext::makeCall): * ftl/FTLSlowPathCall.h: (JSC::FTL::callOperation): * ftl/FTLSlowPathCallKey.cpp: (JSC::FTL::SlowPathCallKey::dump const): * ftl/FTLSlowPathCallKey.h: (JSC::FTL::SlowPathCallKey::SlowPathCallKey): (JSC::FTL::SlowPathCallKey::indirectOffset const): (JSC::FTL::SlowPathCallKey::withCallTarget): (JSC::FTL::SlowPathCallKey::operator== const): (JSC::FTL::SlowPathCallKey::hash const): * ftl/FTLThunks.cpp: (JSC::FTL::genericGenerationThunkGenerator): (JSC::FTL::slowPathCallThunkGenerator): * jit/AssemblyHelpers.cpp: (JSC::AssemblyHelpers::emitVirtualCall): * jit/CCallHelpers.cpp: (JSC::CCallHelpers::emitJITCodeOver): * jit/CCallHelpers.h: (JSC::CCallHelpers::ArgCollection::ArgCollection): (JSC::CCallHelpers::ArgCollection::pushRegArg): (JSC::CCallHelpers::ArgCollection::pushExtraRegArg): (JSC::CCallHelpers::ArgCollection::pushNonArg): (JSC::CCallHelpers::ArgCollection::addGPRArg): (JSC::CCallHelpers::ArgCollection::addGPRExtraArg): (JSC::CCallHelpers::ArgCollection::addStackArg): (JSC::CCallHelpers::ArgCollection::addPoke): (JSC::CCallHelpers::calculatePokeOffset): (JSC::CCallHelpers::pokeForArgument): (JSC::CCallHelpers::stackAligned): (JSC::CCallHelpers::marshallArgumentRegister): (JSC::CCallHelpers::setupArgumentsImpl): (JSC::CCallHelpers::pokeArgumentsAligned): (JSC::CCallHelpers::std::is_integral<CURRENT_ARGUMENT_TYPE>::value): (JSC::CCallHelpers::std::is_pointer<CURRENT_ARGUMENT_TYPE>::value): (JSC::CCallHelpers::setupArgumentsEntryImpl): (JSC::CCallHelpers::setupArguments): (JSC::CCallHelpers::setupArgumentsForIndirectCall): * jit/GCAwareJITStubRoutine.cpp: (JSC::GCAwareJITStubRoutine::GCAwareJITStubRoutine): (JSC::GCAwareJITStubRoutine::makeGCAware): (JSC::GCAwareJITStubRoutine::observeZeroRefCount): (JSC::PolymorphicAccessJITStubRoutine::PolymorphicAccessJITStubRoutine): (JSC::PolymorphicAccessJITStubRoutine::observeZeroRefCount): (JSC::PolymorphicAccessJITStubRoutine::computeHash): (JSC::MarkingGCAwareJITStubRoutine::MarkingGCAwareJITStubRoutine): (JSC::GCAwareJITStubRoutineWithExceptionHandler::GCAwareJITStubRoutineWithExceptionHandler): (JSC::createICJITStubRoutine): (JSC::createJITStubRoutine): Deleted. * jit/GCAwareJITStubRoutine.h: (JSC::GCAwareJITStubRoutine::create): (JSC::PolymorphicAccessJITStubRoutine::cases const): (JSC::PolymorphicAccessJITStubRoutine::weakStructures const): (JSC::PolymorphicAccessJITStubRoutine::hash const): * jit/GPRInfo.h: * jit/JIT.cpp: (JSC::JIT::link): * jit/JIT.h: * jit/JITCall.cpp: (JSC::JIT::emit_op_iterator_open): (JSC::JIT::emitSlow_op_iterator_open): (JSC::JIT::emit_op_iterator_next): (JSC::JIT::emitSlow_op_iterator_next): * jit/JITCall32_64.cpp: (JSC::JIT::emit_op_iterator_open): (JSC::JIT::emit_op_iterator_next): * jit/JITCode.h: (JSC::JITCode::useDataIC): * jit/JITInlineCacheGenerator.cpp: (JSC::JITInlineCacheGenerator::JITInlineCacheGenerator): (JSC::JITInlineCacheGenerator::finalize): (JSC::JITByIdGenerator::JITByIdGenerator): (JSC::JITByIdGenerator::finalize): (JSC::JITByIdGenerator::generateFastCommon): (JSC::JITGetByIdGenerator::JITGetByIdGenerator): (JSC::JITGetByIdWithThisGenerator::JITGetByIdWithThisGenerator): (JSC::JITPutByIdGenerator::JITPutByIdGenerator): (JSC::JITDelByValGenerator::JITDelByValGenerator): (JSC::JITDelByValGenerator::generateFastPath): (JSC::JITDelByValGenerator::finalize): (JSC::JITDelByIdGenerator::JITDelByIdGenerator): (JSC::JITDelByIdGenerator::generateFastPath): (JSC::JITDelByIdGenerator::finalize): (JSC::JITInByValGenerator::JITInByValGenerator): (JSC::JITInByValGenerator::generateFastPath): (JSC::JITInByValGenerator::finalize): (JSC::JITInByIdGenerator::JITInByIdGenerator): (JSC::JITInstanceOfGenerator::JITInstanceOfGenerator): (JSC::JITInstanceOfGenerator::generateFastPath): (JSC::JITInstanceOfGenerator::finalize): (JSC::JITGetByValGenerator::JITGetByValGenerator): (JSC::JITGetByValGenerator::generateFastPath): (JSC::JITGetByValGenerator::finalize): (JSC::JITPrivateBrandAccessGenerator::JITPrivateBrandAccessGenerator): (JSC::JITPrivateBrandAccessGenerator::generateFastPath): (JSC::JITPrivateBrandAccessGenerator::finalize): * jit/JITInlineCacheGenerator.h: (JSC::JITGetByIdGenerator::JITGetByIdGenerator): Deleted. (JSC::JITGetByIdWithThisGenerator::JITGetByIdWithThisGenerator): Deleted. (JSC::JITPutByIdGenerator::JITPutByIdGenerator): Deleted. (JSC::JITDelByValGenerator::JITDelByValGenerator): Deleted. (JSC::JITDelByValGenerator::slowPathJump const): Deleted. (JSC::JITDelByIdGenerator::JITDelByIdGenerator): Deleted. (JSC::JITDelByIdGenerator::slowPathJump const): Deleted. (JSC::JITInByIdGenerator::JITInByIdGenerator): Deleted. (JSC::JITInstanceOfGenerator::JITInstanceOfGenerator): Deleted. (JSC::JITGetByValGenerator::JITGetByValGenerator): Deleted. (JSC::JITGetByValGenerator::slowPathJump const): Deleted. (JSC::JITPrivateBrandAccessGenerator::JITPrivateBrandAccessGenerator): Deleted. (JSC::JITPrivateBrandAccessGenerator::slowPathJump const): Deleted. * jit/JITInlines.h: (JSC::JIT::emitLoadForArrayMode): (JSC::JIT::appendCallWithExceptionCheck): (JSC::JIT::appendCallWithExceptionCheckSetJSValueResult): (JSC::JIT::appendCallWithExceptionCheckSetJSValueResultWithProfile): * jit/JITOpcodes.cpp: (JSC::JIT::emit_op_instanceof): (JSC::JIT::emitSlow_op_instanceof): (JSC::JIT::privateCompileHasIndexedProperty): (JSC::JIT::emit_op_has_enumerable_indexed_property): (JSC::JIT::emitSlow_op_has_enumerable_indexed_property): * jit/JITOpcodes32_64.cpp: (JSC::JIT::emit_op_instanceof): (JSC::JIT::privateCompileHasIndexedProperty): (JSC::JIT::emit_op_has_enumerable_indexed_property): * jit/JITOperations.cpp: (JSC::JSC_DEFINE_JIT_OPERATION): * jit/JITPropertyAccess.cpp: (JSC::JIT::emit_op_get_by_val): (JSC::JIT::emitSlow_op_get_by_val): (JSC::JIT::slow_op_get_by_val_prepareCallGenerator): (JSC::JIT::emit_op_get_private_name): (JSC::JIT::emitSlow_op_get_private_name): (JSC::JIT::slow_op_get_private_name_prepareCallGenerator): (JSC::JIT::emit_op_set_private_brand): (JSC::JIT::emitSlow_op_set_private_brand): (JSC::JIT::emit_op_check_private_brand): (JSC::JIT::emitSlow_op_check_private_brand): (JSC::JIT::emit_op_put_by_val): (JSC::JIT::emitGenericContiguousPutByVal): (JSC::JIT::emitArrayStoragePutByVal): (JSC::JIT::emitPutByValWithCachedId): (JSC::JIT::emitSlow_op_put_by_val): (JSC::JIT::slow_op_put_by_val_prepareCallGenerator): (JSC::JIT::emit_op_put_private_name): (JSC::JIT::emitSlow_op_put_private_name): (JSC::JIT::slow_op_put_private_name_prepareCallGenerator): (JSC::JIT::emit_op_del_by_id): (JSC::JIT::emitSlow_op_del_by_id): (JSC::JIT::slow_op_del_by_id_prepareCallGenerator): (JSC::JIT::emit_op_del_by_val): (JSC::JIT::emitSlow_op_del_by_val): (JSC::JIT::slow_op_del_by_val_prepareCallGenerator): (JSC::JIT::emit_op_try_get_by_id): (JSC::JIT::emitSlow_op_try_get_by_id): (JSC::JIT::emit_op_get_by_id_direct): (JSC::JIT::emitSlow_op_get_by_id_direct): (JSC::JIT::emit_op_get_by_id): (JSC::JIT::emit_op_get_by_id_with_this): (JSC::JIT::emitSlow_op_get_by_id): (JSC::JIT::slow_op_get_by_id_prepareCallGenerator): (JSC::JIT::emitSlow_op_get_by_id_with_this): (JSC::JIT::slow_op_get_by_id_with_this_prepareCallGenerator): (JSC::JIT::emit_op_put_by_id): (JSC::JIT::emitSlow_op_put_by_id): (JSC::JIT::slow_op_put_by_id_prepareCallGenerator): (JSC::JIT::emit_op_in_by_id): (JSC::JIT::emitSlow_op_in_by_id): (JSC::JIT::emit_op_in_by_val): (JSC::JIT::emitSlow_op_in_by_val): (JSC::JIT::privateCompilePutByVal): (JSC::JIT::privateCompilePutPrivateNameWithCachedId): (JSC::JIT::privateCompilePutByValWithCachedId): (JSC::JIT::emitDoubleLoad): (JSC::JIT::emitContiguousLoad): (JSC::JIT::emitArrayStorageLoad): (JSC::JIT::emitIntTypedArrayPutByVal): (JSC::JIT::emitFloatTypedArrayPutByVal): * jit/JITPropertyAccess32_64.cpp: (JSC::JIT::emit_op_del_by_id): (JSC::JIT::emit_op_del_by_val): (JSC::JIT::emit_op_get_by_val): (JSC::JIT::emit_op_get_private_name): (JSC::JIT::emit_op_set_private_brand): (JSC::JIT::emit_op_check_private_brand): (JSC::JIT::emit_op_put_by_val): (JSC::JIT::emitGenericContiguousPutByVal): (JSC::JIT::emitArrayStoragePutByVal): (JSC::JIT::emit_op_try_get_by_id): (JSC::JIT::emit_op_get_by_id_direct): (JSC::JIT::emit_op_get_by_id): (JSC::JIT::emit_op_get_by_id_with_this): (JSC::JIT::emit_op_put_by_id): (JSC::JIT::emit_op_in_by_id): (JSC::JIT::emit_op_in_by_val): * jit/JITStubRoutine.h: * jit/PolymorphicCallStubRoutine.cpp: (JSC::PolymorphicCallStubRoutine::PolymorphicCallStubRoutine): * jit/Repatch.cpp: (JSC::readPutICCallTarget): (JSC::repatchSlowPathCall): (JSC::tryCacheGetBy): (JSC::repatchGetBy): (JSC::tryCacheArrayGetByVal): (JSC::repatchArrayGetByVal): (JSC::tryCachePutByID): (JSC::repatchPutByID): (JSC::tryCacheDeleteBy): (JSC::repatchDeleteBy): (JSC::tryCacheInBy): (JSC::repatchInBy): (JSC::tryCacheCheckPrivateBrand): (JSC::repatchCheckPrivateBrand): (JSC::tryCacheSetPrivateBrand): (JSC::repatchSetPrivateBrand): (JSC::tryCacheInstanceOf): (JSC::repatchInstanceOf): (JSC::linkSlowFor): (JSC::linkVirtualFor): (JSC::resetGetBy): (JSC::resetPutByID): (JSC::resetDelBy): (JSC::resetInBy): (JSC::resetInstanceOf): (JSC::resetCheckPrivateBrand): (JSC::resetSetPrivateBrand): (JSC::resetPatchableJump): Deleted. * jit/Repatch.h: * runtime/Options.cpp: (JSC::Options::recomputeDependentOptions): * runtime/OptionsList.h: * runtime/StructureIDTable.h: * runtime/VM.cpp: (JSC::VM::VM): (JSC::VM::~VM): * runtime/VM.h: Tools: * Scripts/run-jsc-stress-tests: Canonical link: https://commits.webkit.org/238638@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@278656 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-06-09 11:17:36 +00:00
SlowPathCall SlowPathCallContext::makeCall(VM& vm, CCallHelpers::Address callTarget)
{
SlowPathCallKey key = keyWithTarget(callTarget);
SlowPathCall result = SlowPathCall(m_jit.call(JITThunkPtrTag), key);
m_jit.addLinkTask(
[result, &vm] (LinkBuffer& linkBuffer) {
MacroAssemblerCodeRef<JITThunkPtrTag> thunk =
vm.ftlThunks->getSlowPathCallThunk(vm, result.key());
linkBuffer.link(result.call(), CodeLocationLabel<JITThunkPtrTag>(thunk.code()));
});
return result;
}
FTL should generate code to call slow paths lazily https://bugs.webkit.org/show_bug.cgi?id=149936 Reviewed by Saam Barati. Source/JavaScriptCore: We often have complex slow paths in FTL-generated code. Those slow paths may never run. Even if they do run, they don't need stellar performance. So, it doesn't make sense to have LLVM worry about compiling such slow path code. This patch enables us to use our own MacroAssembler for compiling the slow path inside FTL code. It does this by using a crazy lambda thingy (see FTLLowerDFGToLLVM.cpp's lazySlowPath() and its documentation). The result is quite natural to use. Even for straight slow path calls via something like vmCall(), the lazySlowPath offers the benefit that the call marshalling and the exception checking are not expressed using LLVM IR and do not require LLVM to think about it. It also has the benefit that we never generate the code if it never runs. That's great, since function calls usually involve ~10 instructions total (move arguments to argument registers, make the call, check exception, etc.). This patch adds the lazy slow path abstraction and uses it for some slow paths in the FTL. The code we generate with lazy slow paths is worse than the code that LLVM would have generated. Therefore, a lazy slow path only makes sense when we have strong evidence that the slow path will execute infrequently relative to the fast path. This completely precludes the use of lazy slow paths for out-of-line Nodes that unconditionally call a C++ function. It also precludes their use for the GetByVal out-of-bounds handler, since when we generate a GetByVal with an out-of-bounds handler it means that we only know that the out-of-bounds case executed at least once. So, for all we know, it may actually be the common case. So, this patch just deployed the lazy slow path for GC slow paths and masquerades-as-undefined slow paths. It makes sense for GC slow paths because those have a statistical guarantee of slow path frequency - probably bounded at less than 1/10. It makes sense for masquerades-as- undefined because we can say quite confidently that this is an uncommon scenario on the modern Web. Something that's always been challenging about abstractions involving the MacroAssembler is that linking is a separate phase, and there is no way for someone who is just given access to the MacroAssembler& to emit code that requires linking, since linking happens once we have emitted all code and we are creating the LinkBuffer. Moreover, the FTL requires that the final parts of linking happen on the main thread. This patch ran into this issue, and solved it comprehensively, by introducing MacroAssembler::addLinkTask(). This takes a lambda and runs it at the bitter end of linking - when performFinalization() is called. This ensure that the task added by addLinkTask() runs on the main thread. This patch doesn't replace all of the previously existing idioms for dealing with this issue; we can do that later. This shows small speed-ups on a bunch of things. No big win on any benchmark aggregate. But mainly this is done for https://bugs.webkit.org/show_bug.cgi?id=149852, where we found that outlining the slow path in this way was a significant speed boost. * CMakeLists.txt: * JavaScriptCore.vcxproj/JavaScriptCore.vcxproj: * JavaScriptCore.xcodeproj/project.pbxproj: * assembler/AbstractMacroAssembler.h: (JSC::AbstractMacroAssembler::replaceWithAddressComputation): (JSC::AbstractMacroAssembler::addLinkTask): (JSC::AbstractMacroAssembler::AbstractMacroAssembler): * assembler/LinkBuffer.cpp: (JSC::LinkBuffer::linkCode): (JSC::LinkBuffer::allocate): (JSC::LinkBuffer::performFinalization): * assembler/LinkBuffer.h: (JSC::LinkBuffer::wasAlreadyDisassembled): (JSC::LinkBuffer::didAlreadyDisassemble): (JSC::LinkBuffer::vm): (JSC::LinkBuffer::executableOffsetFor): * bytecode/CodeOrigin.h: (JSC::CodeOrigin::CodeOrigin): (JSC::CodeOrigin::isSet): (JSC::CodeOrigin::operator bool): (JSC::CodeOrigin::isHashTableDeletedValue): (JSC::CodeOrigin::operator!): Deleted. * ftl/FTLCompile.cpp: (JSC::FTL::mmAllocateDataSection): * ftl/FTLInlineCacheDescriptor.h: (JSC::FTL::InlineCacheDescriptor::InlineCacheDescriptor): (JSC::FTL::CheckInDescriptor::CheckInDescriptor): (JSC::FTL::LazySlowPathDescriptor::LazySlowPathDescriptor): * ftl/FTLJITCode.h: * ftl/FTLJITFinalizer.cpp: (JSC::FTL::JITFinalizer::finalizeFunction): * ftl/FTLJITFinalizer.h: * ftl/FTLLazySlowPath.cpp: Added. (JSC::FTL::LazySlowPath::LazySlowPath): (JSC::FTL::LazySlowPath::~LazySlowPath): (JSC::FTL::LazySlowPath::generate): * ftl/FTLLazySlowPath.h: Added. (JSC::FTL::LazySlowPath::createGenerator): (JSC::FTL::LazySlowPath::patchpoint): (JSC::FTL::LazySlowPath::usedRegisters): (JSC::FTL::LazySlowPath::callSiteIndex): (JSC::FTL::LazySlowPath::stub): * ftl/FTLLazySlowPathCall.h: Added. (JSC::FTL::createLazyCallGenerator): * ftl/FTLLowerDFGToLLVM.cpp: (JSC::FTL::DFG::LowerDFGToLLVM::compileCreateActivation): (JSC::FTL::DFG::LowerDFGToLLVM::compileNewFunction): (JSC::FTL::DFG::LowerDFGToLLVM::compileCreateDirectArguments): (JSC::FTL::DFG::LowerDFGToLLVM::compileNewArrayWithSize): (JSC::FTL::DFG::LowerDFGToLLVM::compileMakeRope): (JSC::FTL::DFG::LowerDFGToLLVM::compileNotifyWrite): (JSC::FTL::DFG::LowerDFGToLLVM::compileIsObjectOrNull): (JSC::FTL::DFG::LowerDFGToLLVM::compileIsFunction): (JSC::FTL::DFG::LowerDFGToLLVM::compileIn): (JSC::FTL::DFG::LowerDFGToLLVM::compileMaterializeNewObject): (JSC::FTL::DFG::LowerDFGToLLVM::compileMaterializeCreateActivation): (JSC::FTL::DFG::LowerDFGToLLVM::compileCheckWatchdogTimer): (JSC::FTL::DFG::LowerDFGToLLVM::allocatePropertyStorageWithSizeImpl): (JSC::FTL::DFG::LowerDFGToLLVM::allocateObject): (JSC::FTL::DFG::LowerDFGToLLVM::allocateJSArray): (JSC::FTL::DFG::LowerDFGToLLVM::buildTypeOf): (JSC::FTL::DFG::LowerDFGToLLVM::sensibleDoubleToInt32): (JSC::FTL::DFG::LowerDFGToLLVM::lazySlowPath): (JSC::FTL::DFG::LowerDFGToLLVM::speculate): (JSC::FTL::DFG::LowerDFGToLLVM::emitStoreBarrier): * ftl/FTLOperations.cpp: (JSC::FTL::operationMaterializeObjectInOSR): (JSC::FTL::compileFTLLazySlowPath): * ftl/FTLOperations.h: * ftl/FTLSlowPathCall.cpp: (JSC::FTL::SlowPathCallContext::SlowPathCallContext): (JSC::FTL::SlowPathCallContext::~SlowPathCallContext): (JSC::FTL::SlowPathCallContext::keyWithTarget): (JSC::FTL::SlowPathCallContext::makeCall): (JSC::FTL::callSiteIndexForCodeOrigin): (JSC::FTL::storeCodeOrigin): Deleted. (JSC::FTL::callOperation): Deleted. * ftl/FTLSlowPathCall.h: (JSC::FTL::callOperation): * ftl/FTLState.h: * ftl/FTLThunks.cpp: (JSC::FTL::genericGenerationThunkGenerator): (JSC::FTL::osrExitGenerationThunkGenerator): (JSC::FTL::lazySlowPathGenerationThunkGenerator): (JSC::FTL::registerClobberCheck): * ftl/FTLThunks.h: * interpreter/CallFrame.h: (JSC::CallSiteIndex::CallSiteIndex): (JSC::CallSiteIndex::operator bool): (JSC::CallSiteIndex::bits): * jit/CCallHelpers.h: (JSC::CCallHelpers::setupArgument): (JSC::CCallHelpers::setupArgumentsWithExecState): * jit/JITOperations.cpp: Source/WTF: Enables SharedTask to handle any function type, not just void(). It's probably better to use SharedTask instead of std::function in performance-sensitive code. std::function uses the system malloc and has copy semantics. SharedTask uses FastMalloc and has aliasing semantics. So, you can just trust that it will have sensible performance characteristics. * wtf/ParallelHelperPool.cpp: (WTF::ParallelHelperClient::~ParallelHelperClient): (WTF::ParallelHelperClient::setTask): (WTF::ParallelHelperClient::doSomeHelping): (WTF::ParallelHelperClient::runTaskInParallel): (WTF::ParallelHelperClient::finish): (WTF::ParallelHelperClient::claimTask): (WTF::ParallelHelperClient::runTask): (WTF::ParallelHelperPool::doSomeHelping): (WTF::ParallelHelperPool::helperThreadBody): * wtf/ParallelHelperPool.h: (WTF::ParallelHelperClient::setFunction): (WTF::ParallelHelperClient::runFunctionInParallel): (WTF::ParallelHelperClient::pool): * wtf/SharedTask.h: (WTF::createSharedTask): (WTF::SharedTask::SharedTask): Deleted. (WTF::SharedTask::~SharedTask): Deleted. (WTF::SharedTaskFunctor::SharedTaskFunctor): Deleted. Canonical link: https://commits.webkit.org/168154@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@190860 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2015-10-12 17:56:26 +00:00
CallSiteIndex callSiteIndexForCodeOrigin(State& state, CodeOrigin codeOrigin)
{
FTL should generate code to call slow paths lazily https://bugs.webkit.org/show_bug.cgi?id=149936 Reviewed by Saam Barati. Source/JavaScriptCore: We often have complex slow paths in FTL-generated code. Those slow paths may never run. Even if they do run, they don't need stellar performance. So, it doesn't make sense to have LLVM worry about compiling such slow path code. This patch enables us to use our own MacroAssembler for compiling the slow path inside FTL code. It does this by using a crazy lambda thingy (see FTLLowerDFGToLLVM.cpp's lazySlowPath() and its documentation). The result is quite natural to use. Even for straight slow path calls via something like vmCall(), the lazySlowPath offers the benefit that the call marshalling and the exception checking are not expressed using LLVM IR and do not require LLVM to think about it. It also has the benefit that we never generate the code if it never runs. That's great, since function calls usually involve ~10 instructions total (move arguments to argument registers, make the call, check exception, etc.). This patch adds the lazy slow path abstraction and uses it for some slow paths in the FTL. The code we generate with lazy slow paths is worse than the code that LLVM would have generated. Therefore, a lazy slow path only makes sense when we have strong evidence that the slow path will execute infrequently relative to the fast path. This completely precludes the use of lazy slow paths for out-of-line Nodes that unconditionally call a C++ function. It also precludes their use for the GetByVal out-of-bounds handler, since when we generate a GetByVal with an out-of-bounds handler it means that we only know that the out-of-bounds case executed at least once. So, for all we know, it may actually be the common case. So, this patch just deployed the lazy slow path for GC slow paths and masquerades-as-undefined slow paths. It makes sense for GC slow paths because those have a statistical guarantee of slow path frequency - probably bounded at less than 1/10. It makes sense for masquerades-as- undefined because we can say quite confidently that this is an uncommon scenario on the modern Web. Something that's always been challenging about abstractions involving the MacroAssembler is that linking is a separate phase, and there is no way for someone who is just given access to the MacroAssembler& to emit code that requires linking, since linking happens once we have emitted all code and we are creating the LinkBuffer. Moreover, the FTL requires that the final parts of linking happen on the main thread. This patch ran into this issue, and solved it comprehensively, by introducing MacroAssembler::addLinkTask(). This takes a lambda and runs it at the bitter end of linking - when performFinalization() is called. This ensure that the task added by addLinkTask() runs on the main thread. This patch doesn't replace all of the previously existing idioms for dealing with this issue; we can do that later. This shows small speed-ups on a bunch of things. No big win on any benchmark aggregate. But mainly this is done for https://bugs.webkit.org/show_bug.cgi?id=149852, where we found that outlining the slow path in this way was a significant speed boost. * CMakeLists.txt: * JavaScriptCore.vcxproj/JavaScriptCore.vcxproj: * JavaScriptCore.xcodeproj/project.pbxproj: * assembler/AbstractMacroAssembler.h: (JSC::AbstractMacroAssembler::replaceWithAddressComputation): (JSC::AbstractMacroAssembler::addLinkTask): (JSC::AbstractMacroAssembler::AbstractMacroAssembler): * assembler/LinkBuffer.cpp: (JSC::LinkBuffer::linkCode): (JSC::LinkBuffer::allocate): (JSC::LinkBuffer::performFinalization): * assembler/LinkBuffer.h: (JSC::LinkBuffer::wasAlreadyDisassembled): (JSC::LinkBuffer::didAlreadyDisassemble): (JSC::LinkBuffer::vm): (JSC::LinkBuffer::executableOffsetFor): * bytecode/CodeOrigin.h: (JSC::CodeOrigin::CodeOrigin): (JSC::CodeOrigin::isSet): (JSC::CodeOrigin::operator bool): (JSC::CodeOrigin::isHashTableDeletedValue): (JSC::CodeOrigin::operator!): Deleted. * ftl/FTLCompile.cpp: (JSC::FTL::mmAllocateDataSection): * ftl/FTLInlineCacheDescriptor.h: (JSC::FTL::InlineCacheDescriptor::InlineCacheDescriptor): (JSC::FTL::CheckInDescriptor::CheckInDescriptor): (JSC::FTL::LazySlowPathDescriptor::LazySlowPathDescriptor): * ftl/FTLJITCode.h: * ftl/FTLJITFinalizer.cpp: (JSC::FTL::JITFinalizer::finalizeFunction): * ftl/FTLJITFinalizer.h: * ftl/FTLLazySlowPath.cpp: Added. (JSC::FTL::LazySlowPath::LazySlowPath): (JSC::FTL::LazySlowPath::~LazySlowPath): (JSC::FTL::LazySlowPath::generate): * ftl/FTLLazySlowPath.h: Added. (JSC::FTL::LazySlowPath::createGenerator): (JSC::FTL::LazySlowPath::patchpoint): (JSC::FTL::LazySlowPath::usedRegisters): (JSC::FTL::LazySlowPath::callSiteIndex): (JSC::FTL::LazySlowPath::stub): * ftl/FTLLazySlowPathCall.h: Added. (JSC::FTL::createLazyCallGenerator): * ftl/FTLLowerDFGToLLVM.cpp: (JSC::FTL::DFG::LowerDFGToLLVM::compileCreateActivation): (JSC::FTL::DFG::LowerDFGToLLVM::compileNewFunction): (JSC::FTL::DFG::LowerDFGToLLVM::compileCreateDirectArguments): (JSC::FTL::DFG::LowerDFGToLLVM::compileNewArrayWithSize): (JSC::FTL::DFG::LowerDFGToLLVM::compileMakeRope): (JSC::FTL::DFG::LowerDFGToLLVM::compileNotifyWrite): (JSC::FTL::DFG::LowerDFGToLLVM::compileIsObjectOrNull): (JSC::FTL::DFG::LowerDFGToLLVM::compileIsFunction): (JSC::FTL::DFG::LowerDFGToLLVM::compileIn): (JSC::FTL::DFG::LowerDFGToLLVM::compileMaterializeNewObject): (JSC::FTL::DFG::LowerDFGToLLVM::compileMaterializeCreateActivation): (JSC::FTL::DFG::LowerDFGToLLVM::compileCheckWatchdogTimer): (JSC::FTL::DFG::LowerDFGToLLVM::allocatePropertyStorageWithSizeImpl): (JSC::FTL::DFG::LowerDFGToLLVM::allocateObject): (JSC::FTL::DFG::LowerDFGToLLVM::allocateJSArray): (JSC::FTL::DFG::LowerDFGToLLVM::buildTypeOf): (JSC::FTL::DFG::LowerDFGToLLVM::sensibleDoubleToInt32): (JSC::FTL::DFG::LowerDFGToLLVM::lazySlowPath): (JSC::FTL::DFG::LowerDFGToLLVM::speculate): (JSC::FTL::DFG::LowerDFGToLLVM::emitStoreBarrier): * ftl/FTLOperations.cpp: (JSC::FTL::operationMaterializeObjectInOSR): (JSC::FTL::compileFTLLazySlowPath): * ftl/FTLOperations.h: * ftl/FTLSlowPathCall.cpp: (JSC::FTL::SlowPathCallContext::SlowPathCallContext): (JSC::FTL::SlowPathCallContext::~SlowPathCallContext): (JSC::FTL::SlowPathCallContext::keyWithTarget): (JSC::FTL::SlowPathCallContext::makeCall): (JSC::FTL::callSiteIndexForCodeOrigin): (JSC::FTL::storeCodeOrigin): Deleted. (JSC::FTL::callOperation): Deleted. * ftl/FTLSlowPathCall.h: (JSC::FTL::callOperation): * ftl/FTLState.h: * ftl/FTLThunks.cpp: (JSC::FTL::genericGenerationThunkGenerator): (JSC::FTL::osrExitGenerationThunkGenerator): (JSC::FTL::lazySlowPathGenerationThunkGenerator): (JSC::FTL::registerClobberCheck): * ftl/FTLThunks.h: * interpreter/CallFrame.h: (JSC::CallSiteIndex::CallSiteIndex): (JSC::CallSiteIndex::operator bool): (JSC::CallSiteIndex::bits): * jit/CCallHelpers.h: (JSC::CCallHelpers::setupArgument): (JSC::CCallHelpers::setupArgumentsWithExecState): * jit/JITOperations.cpp: Source/WTF: Enables SharedTask to handle any function type, not just void(). It's probably better to use SharedTask instead of std::function in performance-sensitive code. std::function uses the system malloc and has copy semantics. SharedTask uses FastMalloc and has aliasing semantics. So, you can just trust that it will have sensible performance characteristics. * wtf/ParallelHelperPool.cpp: (WTF::ParallelHelperClient::~ParallelHelperClient): (WTF::ParallelHelperClient::setTask): (WTF::ParallelHelperClient::doSomeHelping): (WTF::ParallelHelperClient::runTaskInParallel): (WTF::ParallelHelperClient::finish): (WTF::ParallelHelperClient::claimTask): (WTF::ParallelHelperClient::runTask): (WTF::ParallelHelperPool::doSomeHelping): (WTF::ParallelHelperPool::helperThreadBody): * wtf/ParallelHelperPool.h: (WTF::ParallelHelperClient::setFunction): (WTF::ParallelHelperClient::runFunctionInParallel): (WTF::ParallelHelperClient::pool): * wtf/SharedTask.h: (WTF::createSharedTask): (WTF::SharedTask::SharedTask): Deleted. (WTF::SharedTask::~SharedTask): Deleted. (WTF::SharedTaskFunctor::SharedTaskFunctor): Deleted. Canonical link: https://commits.webkit.org/168154@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@190860 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2015-10-12 17:56:26 +00:00
if (codeOrigin)
[JSC] Return DisposableCallSiteIndex when destroying GCAwareJITStubRoutineWithExceptionHandler https://bugs.webkit.org/show_bug.cgi?id=213069 <rdar://problem/64205186> Reviewed by Saam Barati. JSTests: * stress/dont-dispose-callsiteindex-while-executing.js: Added. (emptyFunction): (foo.bar.set catch): (foo.bar): (foo): Source/JavaScriptCore: Inside GCAwareJITStubRoutineWithExceptionHandler::observeZeroRefCount, we are returning DisposableCallSiteIndex to freelist. However, GCAwareJITStubRoutineWithExceptionHandler::observeZeroRefCount can be called even if the code of GCAwareJITStubRoutineWithExceptionHandler is on the stack. Let's consider the following scenario. 1. Execute GCAwareJITStubRoutineWithExceptionHandler's code. Set CallSiteIndex to the stack. 2. Execute more code. (1)'s GCAwareJITStubRoutineWithExceptionHandler's code is on the stack. 3. (1)'s GCAwareJITStubRoutineWithExceptionHandler's refcount becomes zero. 4. CallSiteIndex of GCAwareJITStubRoutineWithExceptionHandler is returned. 5. Execute StackVisitor to construct frames. But we cannot find CodeOrigin corresponding to CallSiteIndex stored in (1) since it is already returned. DisposableCallSiteIndex should be returned after ensuring that GCAwareJITStubRoutineWithExceptionHandler's code is not on the stack. Detecting this is the functionality what GCAwareJITStubRoutineWithExceptionHandler can offer. It is destroyed after ensuring that GCAwareJITStubRoutineWithExceptionHandler's code is not on the stack. This patch delays DisposableCallSiteIndex returning until we destroy owner GCAwareJITStubRoutineWithExceptionHandler. But it is possible that CodeBlock* corresponding to GCAwareJITStubRoutineWithExceptionHandler is already destroyed. To avoid this condition, we extract CodeOrigins vector as Ref<DFG::CodeOriginPool> and keep it alive from GCAwareJITStubRoutineWithExceptionHandler too. And since CodeOrigin addition / removal happens only from the main thread after finishing the compilation, and GCAwareJITStubRoutineWithExceptionHandler's destructor is called from the Heap's finalizer, which must be executed from the main thread, we can just modify it without a lock. * CMakeLists.txt: * JavaScriptCore.xcodeproj/project.pbxproj: * Sources.txt: * bytecode/CodeBlock.cpp: (JSC::CodeBlock::newExceptionHandlingCallSiteIndex): (JSC::CodeBlock::codeOrigins): * bytecode/CodeBlock.h: (JSC::CodeBlock::codeOrigin): * dfg/DFGCodeOriginPool.cpp: Added. (JSC::DFG::CodeOriginPool::addCodeOrigin): (JSC::DFG::CodeOriginPool::addUniqueCallSiteIndex): (JSC::DFG::CodeOriginPool::lastCallSite const): (JSC::DFG::CodeOriginPool::addDisposableCallSiteIndex): (JSC::DFG::CodeOriginPool::removeDisposableCallSiteIndex): (JSC::DFG::CodeOriginPool::shrinkToFit): * dfg/DFGCodeOriginPool.h: Added. (JSC::DFG::CodeOriginPool::create): (JSC::DFG::CodeOriginPool::get): (JSC::DFG::CodeOriginPool::size const): * dfg/DFGCommonData.cpp: (JSC::DFG::CommonData::shrinkToFit): (JSC::DFG::CommonData::addCodeOrigin): Deleted. (JSC::DFG::CommonData::addUniqueCallSiteIndex): Deleted. (JSC::DFG::CommonData::lastCallSite const): Deleted. (JSC::DFG::CommonData::addDisposableCallSiteIndex): Deleted. (JSC::DFG::CommonData::removeDisposableCallSiteIndex): Deleted. * dfg/DFGCommonData.h: (JSC::DFG::CommonData::CommonData): * dfg/DFGJITCompiler.cpp: (JSC::DFG::JITCompiler::exceptionCheck): * dfg/DFGJITCompiler.h: (JSC::DFG::JITCompiler::addCallSite): * ftl/FTLLowerDFGToB3.cpp: (JSC::FTL::DFG::LowerDFGToB3::compilePutById): (JSC::FTL::DFG::LowerDFGToB3::compileGetByVal): (JSC::FTL::DFG::LowerDFGToB3::compileDelBy): (JSC::FTL::DFG::LowerDFGToB3::compileCallOrConstruct): (JSC::FTL::DFG::LowerDFGToB3::compileDirectCallOrConstruct): (JSC::FTL::DFG::LowerDFGToB3::compileTailCall): (JSC::FTL::DFG::LowerDFGToB3::compileCallOrConstructVarargsSpread): (JSC::FTL::DFG::LowerDFGToB3::compileCallOrConstructVarargs): (JSC::FTL::DFG::LowerDFGToB3::compileCallEval): (JSC::FTL::DFG::LowerDFGToB3::compileInById): (JSC::FTL::DFG::LowerDFGToB3::compileInstanceOf): (JSC::FTL::DFG::LowerDFGToB3::compileLogShadowChickenTail): (JSC::FTL::DFG::LowerDFGToB3::getById): (JSC::FTL::DFG::LowerDFGToB3::getByIdWithThis): (JSC::FTL::DFG::LowerDFGToB3::lazySlowPath): (JSC::FTL::DFG::LowerDFGToB3::callPreflight): * ftl/FTLSlowPathCall.cpp: (JSC::FTL::callSiteIndexForCodeOrigin): * jit/GCAwareJITStubRoutine.cpp: (JSC::GCAwareJITStubRoutineWithExceptionHandler::GCAwareJITStubRoutineWithExceptionHandler): (JSC::GCAwareJITStubRoutineWithExceptionHandler::~GCAwareJITStubRoutineWithExceptionHandler): (JSC::GCAwareJITStubRoutineWithExceptionHandler::aboutToDie): (JSC::GCAwareJITStubRoutineWithExceptionHandler::observeZeroRefCount): * jit/GCAwareJITStubRoutine.h: Canonical link: https://commits.webkit.org/225876@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@262920 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2020-06-11 20:11:34 +00:00
return state.jitCode->common.codeOrigins->addCodeOrigin(codeOrigin);
FTL should generate code to call slow paths lazily https://bugs.webkit.org/show_bug.cgi?id=149936 Reviewed by Saam Barati. Source/JavaScriptCore: We often have complex slow paths in FTL-generated code. Those slow paths may never run. Even if they do run, they don't need stellar performance. So, it doesn't make sense to have LLVM worry about compiling such slow path code. This patch enables us to use our own MacroAssembler for compiling the slow path inside FTL code. It does this by using a crazy lambda thingy (see FTLLowerDFGToLLVM.cpp's lazySlowPath() and its documentation). The result is quite natural to use. Even for straight slow path calls via something like vmCall(), the lazySlowPath offers the benefit that the call marshalling and the exception checking are not expressed using LLVM IR and do not require LLVM to think about it. It also has the benefit that we never generate the code if it never runs. That's great, since function calls usually involve ~10 instructions total (move arguments to argument registers, make the call, check exception, etc.). This patch adds the lazy slow path abstraction and uses it for some slow paths in the FTL. The code we generate with lazy slow paths is worse than the code that LLVM would have generated. Therefore, a lazy slow path only makes sense when we have strong evidence that the slow path will execute infrequently relative to the fast path. This completely precludes the use of lazy slow paths for out-of-line Nodes that unconditionally call a C++ function. It also precludes their use for the GetByVal out-of-bounds handler, since when we generate a GetByVal with an out-of-bounds handler it means that we only know that the out-of-bounds case executed at least once. So, for all we know, it may actually be the common case. So, this patch just deployed the lazy slow path for GC slow paths and masquerades-as-undefined slow paths. It makes sense for GC slow paths because those have a statistical guarantee of slow path frequency - probably bounded at less than 1/10. It makes sense for masquerades-as- undefined because we can say quite confidently that this is an uncommon scenario on the modern Web. Something that's always been challenging about abstractions involving the MacroAssembler is that linking is a separate phase, and there is no way for someone who is just given access to the MacroAssembler& to emit code that requires linking, since linking happens once we have emitted all code and we are creating the LinkBuffer. Moreover, the FTL requires that the final parts of linking happen on the main thread. This patch ran into this issue, and solved it comprehensively, by introducing MacroAssembler::addLinkTask(). This takes a lambda and runs it at the bitter end of linking - when performFinalization() is called. This ensure that the task added by addLinkTask() runs on the main thread. This patch doesn't replace all of the previously existing idioms for dealing with this issue; we can do that later. This shows small speed-ups on a bunch of things. No big win on any benchmark aggregate. But mainly this is done for https://bugs.webkit.org/show_bug.cgi?id=149852, where we found that outlining the slow path in this way was a significant speed boost. * CMakeLists.txt: * JavaScriptCore.vcxproj/JavaScriptCore.vcxproj: * JavaScriptCore.xcodeproj/project.pbxproj: * assembler/AbstractMacroAssembler.h: (JSC::AbstractMacroAssembler::replaceWithAddressComputation): (JSC::AbstractMacroAssembler::addLinkTask): (JSC::AbstractMacroAssembler::AbstractMacroAssembler): * assembler/LinkBuffer.cpp: (JSC::LinkBuffer::linkCode): (JSC::LinkBuffer::allocate): (JSC::LinkBuffer::performFinalization): * assembler/LinkBuffer.h: (JSC::LinkBuffer::wasAlreadyDisassembled): (JSC::LinkBuffer::didAlreadyDisassemble): (JSC::LinkBuffer::vm): (JSC::LinkBuffer::executableOffsetFor): * bytecode/CodeOrigin.h: (JSC::CodeOrigin::CodeOrigin): (JSC::CodeOrigin::isSet): (JSC::CodeOrigin::operator bool): (JSC::CodeOrigin::isHashTableDeletedValue): (JSC::CodeOrigin::operator!): Deleted. * ftl/FTLCompile.cpp: (JSC::FTL::mmAllocateDataSection): * ftl/FTLInlineCacheDescriptor.h: (JSC::FTL::InlineCacheDescriptor::InlineCacheDescriptor): (JSC::FTL::CheckInDescriptor::CheckInDescriptor): (JSC::FTL::LazySlowPathDescriptor::LazySlowPathDescriptor): * ftl/FTLJITCode.h: * ftl/FTLJITFinalizer.cpp: (JSC::FTL::JITFinalizer::finalizeFunction): * ftl/FTLJITFinalizer.h: * ftl/FTLLazySlowPath.cpp: Added. (JSC::FTL::LazySlowPath::LazySlowPath): (JSC::FTL::LazySlowPath::~LazySlowPath): (JSC::FTL::LazySlowPath::generate): * ftl/FTLLazySlowPath.h: Added. (JSC::FTL::LazySlowPath::createGenerator): (JSC::FTL::LazySlowPath::patchpoint): (JSC::FTL::LazySlowPath::usedRegisters): (JSC::FTL::LazySlowPath::callSiteIndex): (JSC::FTL::LazySlowPath::stub): * ftl/FTLLazySlowPathCall.h: Added. (JSC::FTL::createLazyCallGenerator): * ftl/FTLLowerDFGToLLVM.cpp: (JSC::FTL::DFG::LowerDFGToLLVM::compileCreateActivation): (JSC::FTL::DFG::LowerDFGToLLVM::compileNewFunction): (JSC::FTL::DFG::LowerDFGToLLVM::compileCreateDirectArguments): (JSC::FTL::DFG::LowerDFGToLLVM::compileNewArrayWithSize): (JSC::FTL::DFG::LowerDFGToLLVM::compileMakeRope): (JSC::FTL::DFG::LowerDFGToLLVM::compileNotifyWrite): (JSC::FTL::DFG::LowerDFGToLLVM::compileIsObjectOrNull): (JSC::FTL::DFG::LowerDFGToLLVM::compileIsFunction): (JSC::FTL::DFG::LowerDFGToLLVM::compileIn): (JSC::FTL::DFG::LowerDFGToLLVM::compileMaterializeNewObject): (JSC::FTL::DFG::LowerDFGToLLVM::compileMaterializeCreateActivation): (JSC::FTL::DFG::LowerDFGToLLVM::compileCheckWatchdogTimer): (JSC::FTL::DFG::LowerDFGToLLVM::allocatePropertyStorageWithSizeImpl): (JSC::FTL::DFG::LowerDFGToLLVM::allocateObject): (JSC::FTL::DFG::LowerDFGToLLVM::allocateJSArray): (JSC::FTL::DFG::LowerDFGToLLVM::buildTypeOf): (JSC::FTL::DFG::LowerDFGToLLVM::sensibleDoubleToInt32): (JSC::FTL::DFG::LowerDFGToLLVM::lazySlowPath): (JSC::FTL::DFG::LowerDFGToLLVM::speculate): (JSC::FTL::DFG::LowerDFGToLLVM::emitStoreBarrier): * ftl/FTLOperations.cpp: (JSC::FTL::operationMaterializeObjectInOSR): (JSC::FTL::compileFTLLazySlowPath): * ftl/FTLOperations.h: * ftl/FTLSlowPathCall.cpp: (JSC::FTL::SlowPathCallContext::SlowPathCallContext): (JSC::FTL::SlowPathCallContext::~SlowPathCallContext): (JSC::FTL::SlowPathCallContext::keyWithTarget): (JSC::FTL::SlowPathCallContext::makeCall): (JSC::FTL::callSiteIndexForCodeOrigin): (JSC::FTL::storeCodeOrigin): Deleted. (JSC::FTL::callOperation): Deleted. * ftl/FTLSlowPathCall.h: (JSC::FTL::callOperation): * ftl/FTLState.h: * ftl/FTLThunks.cpp: (JSC::FTL::genericGenerationThunkGenerator): (JSC::FTL::osrExitGenerationThunkGenerator): (JSC::FTL::lazySlowPathGenerationThunkGenerator): (JSC::FTL::registerClobberCheck): * ftl/FTLThunks.h: * interpreter/CallFrame.h: (JSC::CallSiteIndex::CallSiteIndex): (JSC::CallSiteIndex::operator bool): (JSC::CallSiteIndex::bits): * jit/CCallHelpers.h: (JSC::CCallHelpers::setupArgument): (JSC::CCallHelpers::setupArgumentsWithExecState): * jit/JITOperations.cpp: Source/WTF: Enables SharedTask to handle any function type, not just void(). It's probably better to use SharedTask instead of std::function in performance-sensitive code. std::function uses the system malloc and has copy semantics. SharedTask uses FastMalloc and has aliasing semantics. So, you can just trust that it will have sensible performance characteristics. * wtf/ParallelHelperPool.cpp: (WTF::ParallelHelperClient::~ParallelHelperClient): (WTF::ParallelHelperClient::setTask): (WTF::ParallelHelperClient::doSomeHelping): (WTF::ParallelHelperClient::runTaskInParallel): (WTF::ParallelHelperClient::finish): (WTF::ParallelHelperClient::claimTask): (WTF::ParallelHelperClient::runTask): (WTF::ParallelHelperPool::doSomeHelping): (WTF::ParallelHelperPool::helperThreadBody): * wtf/ParallelHelperPool.h: (WTF::ParallelHelperClient::setFunction): (WTF::ParallelHelperClient::runFunctionInParallel): (WTF::ParallelHelperClient::pool): * wtf/SharedTask.h: (WTF::createSharedTask): (WTF::SharedTask::SharedTask): Deleted. (WTF::SharedTask::~SharedTask): Deleted. (WTF::SharedTaskFunctor::SharedTaskFunctor): Deleted. Canonical link: https://commits.webkit.org/168154@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@190860 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2015-10-12 17:56:26 +00:00
return CallSiteIndex();
}
FTL should be able to do some simple inline caches using LLVM patchpoints https://bugs.webkit.org/show_bug.cgi?id=123164 Source/JavaScriptCore: Reviewed by Mark Hahnenberg. This implements GetById inline caches in the FTL using llvm.webkit.patchpoint. The idea is that we ask LLVM for a nop slide the size of a GetById inline cache and then fill in the code after LLVM compilation is complete. For now, we just use the system calling convention for the arguments and return. We also still make some assumptions about registers that aren't correct. But, most of the scaffolding is there and this will successfully patch an inline cache. * JavaScriptCore.xcodeproj/project.pbxproj: * assembler/AbstractMacroAssembler.h: * assembler/LinkBuffer.cpp: (JSC::LinkBuffer::finalizeCodeWithoutDisassembly): (JSC::LinkBuffer::linkCode): (JSC::LinkBuffer::allocate): * assembler/LinkBuffer.h: (JSC::LinkBuffer::LinkBuffer): (JSC::LinkBuffer::link): * ftl/FTLAbbreviations.h: (JSC::FTL::constNull): (JSC::FTL::buildCall): * ftl/FTLCapabilities.cpp: (JSC::FTL::canCompile): * ftl/FTLCompile.cpp: (JSC::FTL::fixFunctionBasedOnStackMaps): * ftl/FTLInlineCacheDescriptor.h: Added. (JSC::FTL::InlineCacheDescriptor::InlineCacheDescriptor): (JSC::FTL::GetByIdDescriptor::GetByIdDescriptor): (JSC::FTL::GetByIdDescriptor::stackmapID): (JSC::FTL::GetByIdDescriptor::codeOrigin): (JSC::FTL::GetByIdDescriptor::uid): * ftl/FTLInlineCacheSize.cpp: Added. (JSC::FTL::sizeOfGetById): (JSC::FTL::sizeOfPutById): * ftl/FTLInlineCacheSize.h: Added. * ftl/FTLIntrinsicRepository.h: * ftl/FTLJITFinalizer.cpp: (JSC::FTL::JITFinalizer::finalizeFunction): * ftl/FTLJITFinalizer.h: * ftl/FTLLocation.cpp: (JSC::FTL::Location::directGPR): * ftl/FTLLocation.h: * ftl/FTLLowerDFGToLLVM.cpp: (JSC::FTL::LowerDFGToLLVM::compileGetById): * ftl/FTLOutput.h: (JSC::FTL::Output::call): * ftl/FTLSlowPathCall.cpp: Added. (JSC::FTL::callOperation): * ftl/FTLSlowPathCall.h: Added. (JSC::FTL::SlowPathCall::SlowPathCall): (JSC::FTL::SlowPathCall::call): (JSC::FTL::SlowPathCall::key): * ftl/FTLSlowPathCallKey.cpp: Added. (JSC::FTL::SlowPathCallKey::dump): * ftl/FTLSlowPathCallKey.h: Added. (JSC::FTL::SlowPathCallKey::SlowPathCallKey): (JSC::FTL::SlowPathCallKey::usedRegisters): (JSC::FTL::SlowPathCallKey::callTarget): (JSC::FTL::SlowPathCallKey::offset): (JSC::FTL::SlowPathCallKey::isEmptyValue): (JSC::FTL::SlowPathCallKey::isDeletedValue): (JSC::FTL::SlowPathCallKey::operator==): (JSC::FTL::SlowPathCallKey::hash): (JSC::FTL::SlowPathCallKeyHash::hash): (JSC::FTL::SlowPathCallKeyHash::equal): * ftl/FTLStackMaps.cpp: (JSC::FTL::StackMaps::Location::directGPR): * ftl/FTLStackMaps.h: * ftl/FTLState.h: * ftl/FTLThunks.cpp: (JSC::FTL::slowPathCallThunkGenerator): * ftl/FTLThunks.h: (JSC::FTL::Thunks::getSlowPathCallThunk): * jit/CCallHelpers.h: (JSC::CCallHelpers::setupArguments): * jit/GPRInfo.h: * jit/JITInlineCacheGenerator.cpp: (JSC::garbageStubInfo): (JSC::JITInlineCacheGenerator::JITInlineCacheGenerator): (JSC::JITByIdGenerator::finalize): * jit/JITInlineCacheGenerator.h: (JSC::JITByIdGenerator::slowPathBegin): * jit/RegisterSet.cpp: (JSC::RegisterSet::stackRegisters): (JSC::RegisterSet::specialRegisters): (JSC::RegisterSet::calleeSaveRegisters): (JSC::RegisterSet::allGPRs): (JSC::RegisterSet::allFPRs): (JSC::RegisterSet::allRegisters): (JSC::RegisterSet::dump): * jit/RegisterSet.h: (JSC::RegisterSet::exclude): (JSC::RegisterSet::numberOfSetRegisters): (JSC::RegisterSet::RegisterSet): (JSC::RegisterSet::isEmptyValue): (JSC::RegisterSet::isDeletedValue): (JSC::RegisterSet::operator==): (JSC::RegisterSet::hash): (JSC::RegisterSetHash::hash): (JSC::RegisterSetHash::equal): * runtime/Options.h: Source/WTF: Reviewed by Mark Hahnenberg. This needed some better bitvector support, like merging (|=), excluding (&=~), hashing, and bit counting. * wtf/BitVector.cpp: (WTF::BitVector::setSlow): (WTF::BitVector::excludeSlow): (WTF::BitVector::bitCountSlow): (WTF::BitVector::equalsSlowCase): (WTF::BitVector::hashSlowCase): (WTF::BitVector::dump): * wtf/BitVector.h: (WTF::BitVector::merge): (WTF::BitVector::exclude): (WTF::BitVector::bitCount): (WTF::BitVector::BitVector): (WTF::BitVector::isEmptyValue): (WTF::BitVector::isDeletedValue): (WTF::BitVector::isEmptyOrDeletedValue): (WTF::BitVector::operator==): (WTF::BitVector::hash): (WTF::BitVectorHash::hash): (WTF::BitVectorHash::equal): * wtf/HashTraits.h: (WTF::CustomHashTraits::constructDeletedValue): (WTF::CustomHashTraits::isDeletedValue): (WTF::CustomHashTraits::emptyValue): (WTF::CustomHashTraits::isEmptyValue): * wtf/StdLibExtras.h: (WTF::bitCount): Canonical link: https://commits.webkit.org/141315@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@157872 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2013-10-23 18:22:09 +00:00
} } // namespace JSC::FTL
#endif // ENABLE(FTL_JIT)