haikuwebkit/Source/WTF/wtf/FastTLS.h

64 lines
2.4 KiB
C
Raw Permalink Normal View History

The JITs should be able to emit fast TLS loads https://bugs.webkit.org/show_bug.cgi?id=169483 Reviewed by Keith Miller. Source/JavaScriptCore: Added loadFromTLS32/64/Ptr to the MacroAssembler and added a B3 test for this. * assembler/ARM64Assembler.h: (JSC::ARM64Assembler::mrs_TPIDRRO_EL0): * assembler/MacroAssembler.h: (JSC::MacroAssembler::loadFromTLSPtr): * assembler/MacroAssemblerARM64.h: (JSC::MacroAssemblerARM64::loadFromTLS32): (JSC::MacroAssemblerARM64::loadFromTLS64): * assembler/MacroAssemblerX86Common.h: (JSC::MacroAssemblerX86Common::loadFromTLS32): * assembler/MacroAssemblerX86_64.h: (JSC::MacroAssemblerX86_64::loadFromTLS64): * assembler/X86Assembler.h: (JSC::X86Assembler::adcl_im): (JSC::X86Assembler::addl_mr): (JSC::X86Assembler::addl_im): (JSC::X86Assembler::andl_im): (JSC::X86Assembler::orl_im): (JSC::X86Assembler::orl_rm): (JSC::X86Assembler::subl_im): (JSC::X86Assembler::cmpb_im): (JSC::X86Assembler::cmpl_rm): (JSC::X86Assembler::cmpl_im): (JSC::X86Assembler::testb_im): (JSC::X86Assembler::movb_i8m): (JSC::X86Assembler::movb_rm): (JSC::X86Assembler::movl_mr): (JSC::X86Assembler::movq_mr): (JSC::X86Assembler::movsxd_rr): (JSC::X86Assembler::gs): (JSC::X86Assembler::X86InstructionFormatter::SingleInstructionBufferWriter::memoryModRM): * b3/testb3.cpp: (JSC::B3::testFastTLS): (JSC::B3::run): Source/WTF: Consolidated what we know about fast TLS in FastTLS.h. * WTF.xcodeproj/project.pbxproj: * wtf/CMakeLists.txt: * wtf/FastTLS.h: Added. (WTF::loadFastTLS): (WTF::fastTLSOffsetForKey): * wtf/Platform.h: * wtf/WTFThreadData.cpp: (WTF::WTFThreadData::createAndRegisterForGetspecificDirect): * wtf/WTFThreadData.h: (WTF::wtfThreadData): Canonical link: https://commits.webkit.org/186504@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@213753 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-03-11 07:17:48 +00:00
/*
* Copyright (C) 2017-2018 Apple Inc. All rights reserved.
The JITs should be able to emit fast TLS loads https://bugs.webkit.org/show_bug.cgi?id=169483 Reviewed by Keith Miller. Source/JavaScriptCore: Added loadFromTLS32/64/Ptr to the MacroAssembler and added a B3 test for this. * assembler/ARM64Assembler.h: (JSC::ARM64Assembler::mrs_TPIDRRO_EL0): * assembler/MacroAssembler.h: (JSC::MacroAssembler::loadFromTLSPtr): * assembler/MacroAssemblerARM64.h: (JSC::MacroAssemblerARM64::loadFromTLS32): (JSC::MacroAssemblerARM64::loadFromTLS64): * assembler/MacroAssemblerX86Common.h: (JSC::MacroAssemblerX86Common::loadFromTLS32): * assembler/MacroAssemblerX86_64.h: (JSC::MacroAssemblerX86_64::loadFromTLS64): * assembler/X86Assembler.h: (JSC::X86Assembler::adcl_im): (JSC::X86Assembler::addl_mr): (JSC::X86Assembler::addl_im): (JSC::X86Assembler::andl_im): (JSC::X86Assembler::orl_im): (JSC::X86Assembler::orl_rm): (JSC::X86Assembler::subl_im): (JSC::X86Assembler::cmpb_im): (JSC::X86Assembler::cmpl_rm): (JSC::X86Assembler::cmpl_im): (JSC::X86Assembler::testb_im): (JSC::X86Assembler::movb_i8m): (JSC::X86Assembler::movb_rm): (JSC::X86Assembler::movl_mr): (JSC::X86Assembler::movq_mr): (JSC::X86Assembler::movsxd_rr): (JSC::X86Assembler::gs): (JSC::X86Assembler::X86InstructionFormatter::SingleInstructionBufferWriter::memoryModRM): * b3/testb3.cpp: (JSC::B3::testFastTLS): (JSC::B3::run): Source/WTF: Consolidated what we know about fast TLS in FastTLS.h. * WTF.xcodeproj/project.pbxproj: * wtf/CMakeLists.txt: * wtf/FastTLS.h: Added. (WTF::loadFastTLS): (WTF::fastTLSOffsetForKey): * wtf/Platform.h: * wtf/WTFThreadData.cpp: (WTF::WTFThreadData::createAndRegisterForGetspecificDirect): * wtf/WTFThreadData.h: (WTF::wtfThreadData): Canonical link: https://commits.webkit.org/186504@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@213753 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-03-11 07:17:48 +00:00
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
* are met:
* 1. Redistributions of source code must retain the above copyright
* notice, this list of conditions and the following disclaimer.
* 2. Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
*
* THIS SOFTWARE IS PROVIDED BY APPLE INC. ``AS IS'' AND ANY
* EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
* PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR
* CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
* EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
* PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
* PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
* OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
* (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
* OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
#pragma once
#if HAVE(FAST_TLS)
#include <pthread.h>
#include <System/pthread_machdep.h>
WebAssembly: store state in TLS instead of on VM https://bugs.webkit.org/show_bug.cgi?id=169611 Reviewed by Filip Pizlo. Source/JavaScriptCore: Using thread-local storage instead of VM makes code more position independent. We used to store the WebAssembly top Instance (the latest one in the call stack) on VM, now we instead store it in TLS. This top Instance is used to access a bunch of state such as Memory location, size, table (for call_indirect), etc. Instead of calling it "top", which is confusing, we now just call it WasmContext. Making the code PIC means future patches will be able to postMessage and structured clone into IDB without having to recompile the code. This wasn't possible before because we hard-coded the address of VM at compilation time. That doesn't work between workers, and doesn't work across reloads (which IDB is intended to do). It'll also potentially make code faster once we start tuning what's in TLS, what's in which of the 4 free slots, and what's in pinned registers. I'm leaving this tuning for later because there's lower lying fruit for us to pick. * CMakeLists.txt: * JavaScriptCore.xcodeproj/project.pbxproj: * assembler/AbstractMacroAssembler.h: * assembler/AllowMacroScratchRegisterUsageIf.h: Copied from assembler/AllowMacroScratchRegisterUsage.h. (JSC::AllowMacroScratchRegisterUsageIf::AllowMacroScratchRegisterUsageIf): (JSC::AllowMacroScratchRegisterUsageIf::~AllowMacroScratchRegisterUsageIf): * assembler/MacroAssembler.h: (JSC::MacroAssembler::storeToTLSPtr): we previously didn't have the code required to store to TLS, only to load * assembler/MacroAssemblerARM64.h: (JSC::MacroAssemblerARM64::loadFromTLSPtrNeedsMacroScratchRegister): (JSC::MacroAssemblerARM64::storeToTLS32): (JSC::MacroAssemblerARM64::storeToTLS64): (JSC::MacroAssemblerARM64::storeToTLSPtrNeedsMacroScratchRegister): * assembler/MacroAssemblerX86Common.h: (JSC::MacroAssemblerX86Common::loadFromTLSPtrNeedsMacroScratchRegister): (JSC::MacroAssemblerX86Common::storeToTLS32): (JSC::MacroAssemblerX86Common::storeToTLSPtrNeedsMacroScratchRegister): * assembler/MacroAssemblerX86_64.h: (JSC::MacroAssemblerX86_64::loadFromTLS64): was loading 32-bit instead of 64-bit (JSC::MacroAssemblerX86_64::storeToTLS64): * assembler/X86Assembler.h: (JSC::X86Assembler::movl_rm): (JSC::X86Assembler::movq_rm): * b3/testb3.cpp: (JSC::B3::testFastTLSLoad): (JSC::B3::testFastTLSStore): (JSC::B3::run): * jit/AssemblyHelpers.h: (JSC::AssemblyHelpers::loadWasmContext): (JSC::AssemblyHelpers::storeWasmContext): (JSC::AssemblyHelpers::loadWasmContextNeedsMacroScratchRegister): (JSC::AssemblyHelpers::storeWasmContextNeedsMacroScratchRegister): * jit/Repatch.cpp: (JSC::webAssemblyOwner): * jit/ThunkGenerators.cpp: (JSC::throwExceptionFromWasmThunkGenerator): * runtime/Options.h: * runtime/VM.cpp: (JSC::VM::VM): * runtime/VM.h: * wasm/WasmB3IRGenerator.cpp: (JSC::Wasm::loadWasmContext): (JSC::Wasm::storeWasmContext): (JSC::Wasm::B3IRGenerator::B3IRGenerator): (JSC::Wasm::getMemoryBaseAndSize): (JSC::Wasm::restoreWebAssemblyGlobalState): (JSC::Wasm::createJSToWasmWrapper): (JSC::Wasm::parseAndCompile): * wasm/WasmBinding.cpp: (JSC::Wasm::materializeImportJSCell): (JSC::Wasm::wasmToJs): (JSC::Wasm::wasmToWasm): * wasm/WasmContext.cpp: Added. (JSC::loadWasmContext): (JSC::storeWasmContext): * wasm/WasmContext.h: Added. Replaces "top" JSWebAssemblyInstance. * wasm/js/WebAssemblyFunction.cpp: (JSC::callWebAssemblyFunction): * wasm/js/WebAssemblyInstanceConstructor.h: Source/WTF: * wtf/FastTLS.h: reserve one key for WebAssembly, delete a bunch of dead code which clang couldn't compile (it's valid GCC assembly which LLVM dislikes). Canonical link: https://commits.webkit.org/187009@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@214384 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-03-24 23:25:16 +00:00
#include <wtf/Platform.h>
The JITs should be able to emit fast TLS loads https://bugs.webkit.org/show_bug.cgi?id=169483 Reviewed by Keith Miller. Source/JavaScriptCore: Added loadFromTLS32/64/Ptr to the MacroAssembler and added a B3 test for this. * assembler/ARM64Assembler.h: (JSC::ARM64Assembler::mrs_TPIDRRO_EL0): * assembler/MacroAssembler.h: (JSC::MacroAssembler::loadFromTLSPtr): * assembler/MacroAssemblerARM64.h: (JSC::MacroAssemblerARM64::loadFromTLS32): (JSC::MacroAssemblerARM64::loadFromTLS64): * assembler/MacroAssemblerX86Common.h: (JSC::MacroAssemblerX86Common::loadFromTLS32): * assembler/MacroAssemblerX86_64.h: (JSC::MacroAssemblerX86_64::loadFromTLS64): * assembler/X86Assembler.h: (JSC::X86Assembler::adcl_im): (JSC::X86Assembler::addl_mr): (JSC::X86Assembler::addl_im): (JSC::X86Assembler::andl_im): (JSC::X86Assembler::orl_im): (JSC::X86Assembler::orl_rm): (JSC::X86Assembler::subl_im): (JSC::X86Assembler::cmpb_im): (JSC::X86Assembler::cmpl_rm): (JSC::X86Assembler::cmpl_im): (JSC::X86Assembler::testb_im): (JSC::X86Assembler::movb_i8m): (JSC::X86Assembler::movb_rm): (JSC::X86Assembler::movl_mr): (JSC::X86Assembler::movq_mr): (JSC::X86Assembler::movsxd_rr): (JSC::X86Assembler::gs): (JSC::X86Assembler::X86InstructionFormatter::SingleInstructionBufferWriter::memoryModRM): * b3/testb3.cpp: (JSC::B3::testFastTLS): (JSC::B3::run): Source/WTF: Consolidated what we know about fast TLS in FastTLS.h. * WTF.xcodeproj/project.pbxproj: * wtf/CMakeLists.txt: * wtf/FastTLS.h: Added. (WTF::loadFastTLS): (WTF::fastTLSOffsetForKey): * wtf/Platform.h: * wtf/WTFThreadData.cpp: (WTF::WTFThreadData::createAndRegisterForGetspecificDirect): * wtf/WTFThreadData.h: (WTF::wtfThreadData): Canonical link: https://commits.webkit.org/186504@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@213753 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-03-11 07:17:48 +00:00
namespace WTF {
bmalloc should support strictly type-segregated isolated heaps https://bugs.webkit.org/show_bug.cgi?id=178108 Reviewed by Saam Barati, Simon Fraser, and Ryosuke Niwa. Source/bmalloc: This introduces a new allocation API in bmalloc called IsoHeap. An IsoHeap is templatized by type and created in static storage. When unused, it takes only a few words. When you do use it, each IsoHeap gets a bag of virtual pages unique to it. This prevents use-after-free bugs in one IsoHeap from affecting any other memory. At worst, two pointers of the same type will point to the same object even though they should not have. IsoHeaps allocate using a first-fit discipline that combines ideas from bmalloc and Riptide (the JSC GC): Like Riptide, it uses a bump'n'pop allocator. What Riptide calls blocks, IsoHeaps calls pages. Pages are collected into directories. Directories track pages using bitvectors, so that it's easy to quickly find a completely free page or one that has at least one free object. I think that the bump'n'pop allocator is as fast as the bmalloc Immix-style (page and line) allocator, but is better at allocating in holes. It's guaranteed to follow a first-fit discipline. However, the real reason why I wrote it that was is that this is what I'm more familiar with. This is a part of the design I want to revisit (bug 179278). Like bmalloc, it uses a deallocation log. This means that the internal IsoHeap data structures can be locked with a coarse-grained lock, since the deallocator only grabs it when flushing the log. Similarly, the allocator only grabs it when refilling the bump'n'pop FreeList. This adds a unit test for IsoHeaps. In this change, IsoHeaps are adopted only by WebCore's RenderObject. Note that despite the use of GC concepts, it's not a goal to make this code directly sharable with GC. The GC will probably have to do isolated heaps its own way (likely a special Subspace or something like that). * bmalloc.xcodeproj/project.pbxproj: * bmalloc/Algorithm.h: (bmalloc::findBitInWord): * bmalloc/AllIsoHeaps.cpp: Added. (bmalloc::AllIsoHeaps::AllIsoHeaps): (bmalloc::AllIsoHeaps::add): (bmalloc::AllIsoHeaps::head): * bmalloc/AllIsoHeaps.h: Added. * bmalloc/AllIsoHeapsInlines.h: Added. (bmalloc::AllIsoHeaps::forEach): * bmalloc/BMalloced.h: Added. * bmalloc/Bits.h: Added. (bmalloc::bitsArrayLength): (bmalloc::BitsWordView::BitsWordView): (bmalloc::BitsWordView::numBits const): (bmalloc::BitsWordView::word const): (bmalloc::BitsWordOwner::BitsWordOwner): (bmalloc::BitsWordOwner::view const): (bmalloc::BitsWordOwner::operator=): (bmalloc::BitsWordOwner::setAll): (bmalloc::BitsWordOwner::clearAll): (bmalloc::BitsWordOwner::set): (bmalloc::BitsWordOwner::numBits const): (bmalloc::BitsWordOwner::arrayLength const): (bmalloc::BitsWordOwner::word const): (bmalloc::BitsWordOwner::word): (bmalloc::BitsWordOwner::words const): (bmalloc::BitsWordOwner::words): (bmalloc::BitsAndWords::BitsAndWords): (bmalloc::BitsAndWords::view const): (bmalloc::BitsAndWords::numBits const): (bmalloc::BitsAndWords::word const): (bmalloc::BitsOrWords::BitsOrWords): (bmalloc::BitsOrWords::view const): (bmalloc::BitsOrWords::numBits const): (bmalloc::BitsOrWords::word const): (bmalloc::BitsNotWords::BitsNotWords): (bmalloc::BitsNotWords::view const): (bmalloc::BitsNotWords::numBits const): (bmalloc::BitsNotWords::word const): (bmalloc::BitsImpl::BitsImpl): (bmalloc::BitsImpl::numBits const): (bmalloc::BitsImpl::size const): (bmalloc::BitsImpl::arrayLength const): (bmalloc::BitsImpl::operator== const): (bmalloc::BitsImpl::operator!= const): (bmalloc::BitsImpl::at const): (bmalloc::BitsImpl::operator[] const): (bmalloc::BitsImpl::isEmpty const): (bmalloc::BitsImpl::operator& const): (bmalloc::BitsImpl::operator| const): (bmalloc::BitsImpl::operator~ const): (bmalloc::BitsImpl::forEachSetBit const): (bmalloc::BitsImpl::forEachClearBit const): (bmalloc::BitsImpl::forEachBit const): (bmalloc::BitsImpl::findBit const): (bmalloc::BitsImpl::findSetBit const): (bmalloc::BitsImpl::findClearBit const): (bmalloc::BitsImpl::wordView const): (bmalloc::BitsImpl::atImpl const): (bmalloc::Bits::Bits): (bmalloc::Bits::operator=): (bmalloc::Bits::resize): (bmalloc::Bits::setAll): (bmalloc::Bits::clearAll): (bmalloc::Bits::setAndCheck): (bmalloc::Bits::operator|=): (bmalloc::Bits::operator&=): (bmalloc::Bits::at const): (bmalloc::Bits::operator[] const): (bmalloc::Bits::BitReference::BitReference): (bmalloc::Bits::BitReference::operator bool const): (bmalloc::Bits::BitReference::operator=): (bmalloc::Bits::at): (bmalloc::Bits::operator[]): * bmalloc/CryptoRandom.cpp: Replaced with Source/bmalloc/bmalloc/CryptoRandom.cpp. (bmalloc::cryptoRandom): * bmalloc/CryptoRandom.h: Replaced with Source/bmalloc/bmalloc/CryptoRandom.h. * bmalloc/DeferredDecommit.h: Added. * bmalloc/DeferredDecommitInlines.h: Added. (bmalloc::DeferredDecommit::DeferredDecommit): * bmalloc/DeferredTrigger.h: Added. (bmalloc::DeferredTrigger::DeferredTrigger): * bmalloc/DeferredTriggerInlines.h: Added. (bmalloc::DeferredTrigger<trigger>::didBecome): (bmalloc::DeferredTrigger<trigger>::handleDeferral): * bmalloc/EligibilityResult.h: Added. (bmalloc::EligibilityResult::EligibilityResult): * bmalloc/EligibilityResultInlines.h: Added. (bmalloc::EligibilityResult<Config>::EligibilityResult): * bmalloc/FixedVector.h: * bmalloc/FreeList.cpp: Added. (bmalloc::FreeList::FreeList): (bmalloc::FreeList::~FreeList): (bmalloc::FreeList::clear): (bmalloc::FreeList::initializeList): (bmalloc::FreeList::initializeBump): (bmalloc::FreeList::contains const): * bmalloc/FreeList.h: Added. (bmalloc::FreeCell::scramble): (bmalloc::FreeCell::descramble): (bmalloc::FreeCell::setNext): (bmalloc::FreeCell::next const): (bmalloc::FreeList::allocationWillFail const): (bmalloc::FreeList::allocationWillSucceed const): (bmalloc::FreeList::originalSize const): (bmalloc::FreeList::head const): * bmalloc/FreeListInlines.h: Added. (bmalloc::FreeList::allocate): (bmalloc::FreeList::forEach const): * bmalloc/IsoAllocator.h: Added. * bmalloc/IsoAllocatorInlines.h: Added. (bmalloc::IsoAllocator<Config>::IsoAllocator): (bmalloc::IsoAllocator<Config>::~IsoAllocator): (bmalloc::IsoAllocator<Config>::allocate): (bmalloc::IsoAllocator<Config>::allocateSlow): (bmalloc::IsoAllocator<Config>::scavenge): * bmalloc/IsoConfig.h: Added. * bmalloc/IsoDeallocator.h: Added. * bmalloc/IsoDeallocatorInlines.h: Added. (bmalloc::IsoDeallocator<Config>::IsoDeallocator): (bmalloc::IsoDeallocator<Config>::~IsoDeallocator): (bmalloc::IsoDeallocator<Config>::deallocate): (bmalloc::IsoDeallocator<Config>::scavenge): * bmalloc/IsoDirectory.h: Added. (bmalloc::IsoDirectoryBaseBase::IsoDirectoryBaseBase): (bmalloc::IsoDirectoryBaseBase::~IsoDirectoryBaseBase): (bmalloc::IsoDirectoryBase::heap): * bmalloc/IsoDirectoryInlines.h: Added. (bmalloc::IsoDirectoryBase<Config>::IsoDirectoryBase): (bmalloc::passedNumPages>::IsoDirectory): (bmalloc::passedNumPages>::takeFirstEligible): (bmalloc::passedNumPages>::didBecome): (bmalloc::passedNumPages>::didDecommit): (bmalloc::passedNumPages>::scavenge): (bmalloc::passedNumPages>::forEachCommittedPage): * bmalloc/IsoDirectoryPage.h: Added. (bmalloc::IsoDirectoryPage::index const): * bmalloc/IsoDirectoryPageInlines.h: Added. (bmalloc::IsoDirectoryPage<Config>::IsoDirectoryPage): (bmalloc::IsoDirectoryPage<Config>::pageFor): * bmalloc/IsoHeap.h: Added. (bmalloc::api::IsoHeap::allocatorOffset): (bmalloc::api::IsoHeap::setAllocatorOffset): (bmalloc::api::IsoHeap::deallocatorOffset): (bmalloc::api::IsoHeap::setDeallocatorOffset): * bmalloc/IsoHeapImpl.cpp: Added. (bmalloc::IsoHeapImplBase::IsoHeapImplBase): (bmalloc::IsoHeapImplBase::~IsoHeapImplBase): (bmalloc::IsoHeapImplBase::scavengeNow): (bmalloc::IsoHeapImplBase::finishScavenging): * bmalloc/IsoHeapImpl.h: Added. * bmalloc/IsoHeapImplInlines.h: Added. (bmalloc::IsoHeapImpl<Config>::IsoHeapImpl): (bmalloc::IsoHeapImpl<Config>::takeFirstEligible): (bmalloc::IsoHeapImpl<Config>::didBecomeEligible): (bmalloc::IsoHeapImpl<Config>::scavenge): (bmalloc::IsoHeapImpl<Config>::allocatorOffset): (bmalloc::IsoHeapImpl<Config>::deallocatorOffset): (bmalloc::IsoHeapImpl<Config>::numLiveObjects): (bmalloc::IsoHeapImpl<Config>::numCommittedPages): (bmalloc::IsoHeapImpl<Config>::forEachDirectory): (bmalloc::IsoHeapImpl<Config>::forEachCommittedPage): (bmalloc::IsoHeapImpl<Config>::forEachLiveObject): * bmalloc/IsoHeapInlines.h: Added. (bmalloc::api::IsoHeap<Type>::allocate): (bmalloc::api::IsoHeap<Type>::tryAllocate): (bmalloc::api::IsoHeap<Type>::deallocate): (bmalloc::api::IsoHeap<Type>::scavenge): (bmalloc::api::IsoHeap<Type>::isInitialized): (bmalloc::api::IsoHeap<Type>::impl): * bmalloc/IsoPage.h: Added. (bmalloc::IsoPage::index const): (bmalloc::IsoPage::directory): (bmalloc::IsoPage::isInUseForAllocation const): (bmalloc::IsoPage::indexOfFirstObject): * bmalloc/IsoPageInlines.h: Added. (bmalloc::IsoPage<Config>::tryCreate): (bmalloc::IsoPage<Config>::IsoPage): (bmalloc::IsoPage<Config>::free): (bmalloc::IsoPage<Config>::startAllocating): (bmalloc::IsoPage<Config>::stopAllocating): (bmalloc::IsoPage<Config>::forEachLiveObject): * bmalloc/IsoPageTrigger.h: Added. * bmalloc/IsoTLS.cpp: Added. (bmalloc::IsoTLS::scavenge): (bmalloc::IsoTLS::IsoTLS): (bmalloc::IsoTLS::ensureEntries): (bmalloc::IsoTLS::destructor): (bmalloc::IsoTLS::sizeForCapacity): (bmalloc::IsoTLS::capacityForSize): (bmalloc::IsoTLS::size): (bmalloc::IsoTLS::forEachEntry): * bmalloc/IsoTLS.h: Added. * bmalloc/IsoTLSAllocatorEntry.h: Added. * bmalloc/IsoTLSAllocatorEntryInlines.h: Added. (bmalloc::IsoTLSAllocatorEntry<Config>::IsoTLSAllocatorEntry): (bmalloc::IsoTLSAllocatorEntry<Config>::~IsoTLSAllocatorEntry): (bmalloc::IsoTLSAllocatorEntry<Config>::construct): * bmalloc/IsoTLSDeallocatorEntry.h: Added. * bmalloc/IsoTLSDeallocatorEntryInlines.h: Added. (bmalloc::IsoTLSDeallocatorEntry<Config>::IsoTLSDeallocatorEntry): (bmalloc::IsoTLSDeallocatorEntry<Config>::~IsoTLSDeallocatorEntry): (bmalloc::IsoTLSDeallocatorEntry<Config>::construct): * bmalloc/IsoTLSEntry.cpp: Added. (bmalloc::IsoTLSEntry::IsoTLSEntry): (bmalloc::IsoTLSEntry::~IsoTLSEntry): * bmalloc/IsoTLSEntry.h: Added. (bmalloc::IsoTLSEntry::offset const): (bmalloc::IsoTLSEntry::alignment const): (bmalloc::IsoTLSEntry::size const): (bmalloc::IsoTLSEntry::extent const): * bmalloc/IsoTLSEntryInlines.h: Added. (bmalloc::IsoTLSEntry::walkUpToInclusive): (bmalloc::DefaultIsoTLSEntry<EntryType>::DefaultIsoTLSEntry): (bmalloc::DefaultIsoTLSEntry<EntryType>::~DefaultIsoTLSEntry): (bmalloc::DefaultIsoTLSEntry<EntryType>::move): (bmalloc::DefaultIsoTLSEntry<EntryType>::destruct): (bmalloc::DefaultIsoTLSEntry<EntryType>::scavenge): * bmalloc/IsoTLSInlines.h: Added. (bmalloc::IsoTLS::allocate): (bmalloc::IsoTLS::deallocate): (bmalloc::IsoTLS::scavenge): (bmalloc::IsoTLS::allocator): (bmalloc::IsoTLS::deallocator): (bmalloc::IsoTLS::get): (bmalloc::IsoTLS::set): (bmalloc::IsoTLS::ensureHeap): (bmalloc::IsoTLS::ensureHeapAndEntries): * bmalloc/IsoTLSLayout.cpp: Added. (bmalloc::IsoTLSLayout::IsoTLSLayout): (bmalloc::IsoTLSLayout::add): * bmalloc/IsoTLSLayout.h: Added. (bmalloc::IsoTLSLayout::head const): * bmalloc/PerHeapKind.h: * bmalloc/PerProcess.h: (bmalloc::PerProcess<T>::getFastCase): * bmalloc/Scavenger.cpp: (bmalloc::Scavenger::scavenge): * bmalloc/Scavenger.h: * bmalloc/bmalloc.h: (bmalloc::api::scavengeThisThread): * test: Added. * test/testbmalloc.cpp: Added. (hiddenTruthBecauseNoReturnIsStupid): (usage): (assertEmptyPointerSet): (assertHasObjects): (assertHasOnlyObjects): (assertClean): (testIsoSimple): (testIsoSimpleScavengeBeforeDealloc): (testIsoFlipFlopFragmentedPages): (testIsoFlipFlopFragmentedPagesScavengeInMiddle): (BisoMalloced::BisoMalloced): (testBisoMalloced): (BisoMallocedInline::BisoMallocedInline): (testBisoMallocedInline): (run): (main): Source/WebCore: No new tests because no new change in behavior. Though, the bmalloc change has a unit test. Adopting IsoHeap means dropping in macros in both the .h and .cpp file of each class that we opt in. It's not pretty, but it helps ensure speedy allocation since it means that we never have to do any kind of switch or dynamic lookup to find the right allocator for a type. This change is perf-neutral on MotionMark, PLT3, and membuster. * Sources.txt: * html/shadow/SliderThumbElement.cpp: * html/shadow/SliderThumbElement.h: * html/shadow/mac/ImageControlsButtonElementMac.cpp: * html/shadow/mac/ImageControlsRootElementMac.cpp: * rendering/RenderAttachment.cpp: * rendering/RenderAttachment.h: * rendering/RenderBlock.cpp: * rendering/RenderBlock.h: * rendering/RenderBlockFlow.cpp: * rendering/RenderBlockFlow.h: * rendering/RenderBox.cpp: * rendering/RenderBox.h: * rendering/RenderBoxModelObject.cpp: * rendering/RenderBoxModelObject.h: * rendering/RenderButton.cpp: * rendering/RenderButton.h: * rendering/RenderCombineText.cpp: * rendering/RenderCombineText.h: * rendering/RenderCounter.cpp: * rendering/RenderCounter.h: * rendering/RenderDeprecatedFlexibleBox.cpp: * rendering/RenderDeprecatedFlexibleBox.h: * rendering/RenderDetailsMarker.cpp: * rendering/RenderDetailsMarker.h: * rendering/RenderElement.cpp: * rendering/RenderElement.h: * rendering/RenderEmbeddedObject.cpp: * rendering/RenderEmbeddedObject.h: * rendering/RenderFileUploadControl.cpp: * rendering/RenderFileUploadControl.h: * rendering/RenderFlexibleBox.cpp: * rendering/RenderFlexibleBox.h: * rendering/RenderFragmentContainer.cpp: * rendering/RenderFragmentContainer.h: * rendering/RenderFragmentContainerSet.cpp: * rendering/RenderFragmentContainerSet.h: * rendering/RenderFragmentedFlow.cpp: * rendering/RenderFragmentedFlow.h: * rendering/RenderFrameBase.cpp: * rendering/RenderFrameBase.h: * rendering/RenderFrameSet.cpp: * rendering/RenderFrameSet.h: * rendering/RenderFullScreen.cpp: * rendering/RenderFullScreen.h: * rendering/RenderGrid.cpp: * rendering/RenderGrid.h: * rendering/RenderHTMLCanvas.cpp: * rendering/RenderHTMLCanvas.h: * rendering/RenderImage.cpp: * rendering/RenderImage.h: * rendering/RenderImageResourceStyleImage.cpp: * rendering/RenderImageResourceStyleImage.h: * rendering/RenderInline.cpp: * rendering/RenderInline.h: * rendering/RenderLayerModelObject.cpp: * rendering/RenderLayerModelObject.h: * rendering/RenderLineBreak.cpp: * rendering/RenderLineBreak.h: * rendering/RenderListBox.cpp: * rendering/RenderListBox.h: * rendering/RenderListItem.cpp: * rendering/RenderListItem.h: * rendering/RenderListMarker.cpp: * rendering/RenderListMarker.h: * rendering/RenderMedia.cpp: * rendering/RenderMedia.h: * rendering/RenderMediaControlElements.cpp: * rendering/RenderMediaControlElements.h: * rendering/RenderMenuList.cpp: * rendering/RenderMenuList.h: * rendering/RenderMeter.cpp: * rendering/RenderMeter.h: * rendering/RenderMultiColumnFlow.cpp: * rendering/RenderMultiColumnFlow.h: * rendering/RenderMultiColumnSet.cpp: * rendering/RenderMultiColumnSet.h: * rendering/RenderMultiColumnSpannerPlaceholder.cpp: * rendering/RenderMultiColumnSpannerPlaceholder.h: * rendering/RenderObject.cpp: * rendering/RenderObject.h: * rendering/RenderProgress.cpp: * rendering/RenderProgress.h: * rendering/RenderQuote.cpp: * rendering/RenderQuote.h: * rendering/RenderReplaced.cpp: * rendering/RenderReplaced.h: * rendering/RenderReplica.cpp: * rendering/RenderReplica.h: * rendering/RenderRuby.cpp: * rendering/RenderRuby.h: * rendering/RenderRubyBase.cpp: * rendering/RenderRubyBase.h: * rendering/RenderRubyRun.cpp: * rendering/RenderRubyRun.h: * rendering/RenderRubyText.cpp: * rendering/RenderRubyText.h: * rendering/RenderScrollbarPart.cpp: * rendering/RenderScrollbarPart.h: * rendering/RenderSearchField.cpp: * rendering/RenderSearchField.h: * rendering/RenderSlider.cpp: * rendering/RenderSlider.h: * rendering/RenderTable.cpp: * rendering/RenderTable.h: * rendering/RenderTableCaption.cpp: * rendering/RenderTableCaption.h: * rendering/RenderTableCell.cpp: * rendering/RenderTableCell.h: * rendering/RenderTableCol.cpp: * rendering/RenderTableCol.h: * rendering/RenderTableRow.cpp: * rendering/RenderTableRow.h: * rendering/RenderTableSection.cpp: * rendering/RenderTableSection.h: * rendering/RenderText.cpp: * rendering/RenderText.h: * rendering/RenderTextControl.cpp: * rendering/RenderTextControl.h: * rendering/RenderTextControlMultiLine.cpp: * rendering/RenderTextControlMultiLine.h: * rendering/RenderTextControlSingleLine.cpp: * rendering/RenderTextControlSingleLine.h: * rendering/RenderTextFragment.cpp: * rendering/RenderTextFragment.h: * rendering/RenderVTTCue.cpp: * rendering/RenderVTTCue.h: * rendering/RenderVideo.cpp: * rendering/RenderVideo.h: * rendering/RenderView.cpp: * rendering/RenderView.h: * rendering/RenderWidget.cpp: * rendering/RenderWidget.h: * rendering/mathml/RenderMathMLBlock.cpp: * rendering/mathml/RenderMathMLBlock.h: * rendering/mathml/RenderMathMLFenced.cpp: * rendering/mathml/RenderMathMLFenced.h: * rendering/mathml/RenderMathMLFencedOperator.cpp: * rendering/mathml/RenderMathMLFencedOperator.h: * rendering/mathml/RenderMathMLFraction.cpp: * rendering/mathml/RenderMathMLFraction.h: * rendering/mathml/RenderMathMLMath.cpp: * rendering/mathml/RenderMathMLMath.h: * rendering/mathml/RenderMathMLMenclose.cpp: * rendering/mathml/RenderMathMLMenclose.h: * rendering/mathml/RenderMathMLOperator.cpp: * rendering/mathml/RenderMathMLOperator.h: * rendering/mathml/RenderMathMLPadded.cpp: * rendering/mathml/RenderMathMLPadded.h: * rendering/mathml/RenderMathMLRoot.cpp: * rendering/mathml/RenderMathMLRoot.h: * rendering/mathml/RenderMathMLRow.cpp: * rendering/mathml/RenderMathMLRow.h: * rendering/mathml/RenderMathMLScripts.cpp: * rendering/mathml/RenderMathMLScripts.h: * rendering/mathml/RenderMathMLSpace.cpp: * rendering/mathml/RenderMathMLSpace.h: * rendering/mathml/RenderMathMLToken.cpp: * rendering/mathml/RenderMathMLToken.h: * rendering/mathml/RenderMathMLUnderOver.cpp: * rendering/mathml/RenderMathMLUnderOver.h: * rendering/svg/RenderSVGBlock.cpp: * rendering/svg/RenderSVGBlock.h: * rendering/svg/RenderSVGContainer.cpp: * rendering/svg/RenderSVGContainer.h: * rendering/svg/RenderSVGEllipse.cpp: * rendering/svg/RenderSVGEllipse.h: * rendering/svg/RenderSVGForeignObject.cpp: * rendering/svg/RenderSVGForeignObject.h: * rendering/svg/RenderSVGGradientStop.cpp: * rendering/svg/RenderSVGGradientStop.h: * rendering/svg/RenderSVGHiddenContainer.cpp: * rendering/svg/RenderSVGHiddenContainer.h: * rendering/svg/RenderSVGImage.cpp: * rendering/svg/RenderSVGImage.h: * rendering/svg/RenderSVGInline.cpp: * rendering/svg/RenderSVGInline.h: * rendering/svg/RenderSVGInlineText.cpp: * rendering/svg/RenderSVGInlineText.h: * rendering/svg/RenderSVGModelObject.cpp: * rendering/svg/RenderSVGModelObject.h: * rendering/svg/RenderSVGPath.cpp: * rendering/svg/RenderSVGPath.h: * rendering/svg/RenderSVGRect.cpp: * rendering/svg/RenderSVGRect.h: * rendering/svg/RenderSVGResourceClipper.cpp: * rendering/svg/RenderSVGResourceClipper.h: * rendering/svg/RenderSVGResourceContainer.cpp: * rendering/svg/RenderSVGResourceContainer.h: * rendering/svg/RenderSVGResourceFilter.cpp: * rendering/svg/RenderSVGResourceFilter.h: * rendering/svg/RenderSVGResourceFilterPrimitive.cpp: * rendering/svg/RenderSVGResourceFilterPrimitive.h: * rendering/svg/RenderSVGResourceGradient.cpp: * rendering/svg/RenderSVGResourceGradient.h: * rendering/svg/RenderSVGResourceLinearGradient.cpp: * rendering/svg/RenderSVGResourceLinearGradient.h: * rendering/svg/RenderSVGResourceMarker.cpp: * rendering/svg/RenderSVGResourceMarker.h: * rendering/svg/RenderSVGResourceMasker.cpp: * rendering/svg/RenderSVGResourceMasker.h: * rendering/svg/RenderSVGResourcePattern.cpp: * rendering/svg/RenderSVGResourcePattern.h: * rendering/svg/RenderSVGResourceRadialGradient.cpp: * rendering/svg/RenderSVGResourceRadialGradient.h: * rendering/svg/RenderSVGRoot.cpp: * rendering/svg/RenderSVGRoot.h: * rendering/svg/RenderSVGShape.cpp: * rendering/svg/RenderSVGShape.h: * rendering/svg/RenderSVGTSpan.cpp: Added. * rendering/svg/RenderSVGTSpan.h: * rendering/svg/RenderSVGText.cpp: * rendering/svg/RenderSVGText.h: * rendering/svg/RenderSVGTextPath.cpp: * rendering/svg/RenderSVGTextPath.h: * rendering/svg/RenderSVGTransformableContainer.cpp: * rendering/svg/RenderSVGTransformableContainer.h: * rendering/svg/RenderSVGViewportContainer.cpp: * rendering/svg/RenderSVGViewportContainer.h: Source/WTF: This just adds an easy way of using the bmalloc IsoHeap API in WebKit. If bmalloc is not enabled, these macros will just make the object FastMalloced. * WTF.xcodeproj/project.pbxproj: * wtf/FastTLS.h: * wtf/IsoMalloc.h: Added. * wtf/IsoMallocInlines.h: Added. Canonical link: https://commits.webkit.org/195445@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@224537 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-11-07 19:21:52 +00:00
// __PTK_FRAMEWORK_JAVASCRIPTCORE_KEY0-1 is taken by bmalloc, so WTF's KEY0 maps to the
// system's KEY2.
#define WTF_FAST_TLS_KEY0 __PTK_FRAMEWORK_JAVASCRIPTCORE_KEY2
#define WTF_FAST_TLS_KEY1 __PTK_FRAMEWORK_JAVASCRIPTCORE_KEY3
The JITs should be able to emit fast TLS loads https://bugs.webkit.org/show_bug.cgi?id=169483 Reviewed by Keith Miller. Source/JavaScriptCore: Added loadFromTLS32/64/Ptr to the MacroAssembler and added a B3 test for this. * assembler/ARM64Assembler.h: (JSC::ARM64Assembler::mrs_TPIDRRO_EL0): * assembler/MacroAssembler.h: (JSC::MacroAssembler::loadFromTLSPtr): * assembler/MacroAssemblerARM64.h: (JSC::MacroAssemblerARM64::loadFromTLS32): (JSC::MacroAssemblerARM64::loadFromTLS64): * assembler/MacroAssemblerX86Common.h: (JSC::MacroAssemblerX86Common::loadFromTLS32): * assembler/MacroAssemblerX86_64.h: (JSC::MacroAssemblerX86_64::loadFromTLS64): * assembler/X86Assembler.h: (JSC::X86Assembler::adcl_im): (JSC::X86Assembler::addl_mr): (JSC::X86Assembler::addl_im): (JSC::X86Assembler::andl_im): (JSC::X86Assembler::orl_im): (JSC::X86Assembler::orl_rm): (JSC::X86Assembler::subl_im): (JSC::X86Assembler::cmpb_im): (JSC::X86Assembler::cmpl_rm): (JSC::X86Assembler::cmpl_im): (JSC::X86Assembler::testb_im): (JSC::X86Assembler::movb_i8m): (JSC::X86Assembler::movb_rm): (JSC::X86Assembler::movl_mr): (JSC::X86Assembler::movq_mr): (JSC::X86Assembler::movsxd_rr): (JSC::X86Assembler::gs): (JSC::X86Assembler::X86InstructionFormatter::SingleInstructionBufferWriter::memoryModRM): * b3/testb3.cpp: (JSC::B3::testFastTLS): (JSC::B3::run): Source/WTF: Consolidated what we know about fast TLS in FastTLS.h. * WTF.xcodeproj/project.pbxproj: * wtf/CMakeLists.txt: * wtf/FastTLS.h: Added. (WTF::loadFastTLS): (WTF::fastTLSOffsetForKey): * wtf/Platform.h: * wtf/WTFThreadData.cpp: (WTF::WTFThreadData::createAndRegisterForGetspecificDirect): * wtf/WTFThreadData.h: (WTF::wtfThreadData): Canonical link: https://commits.webkit.org/186504@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@213753 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-03-11 07:17:48 +00:00
// NOTE: We should manage our use of these keys here. If you want to use a key for something,
// put a #define in here to give your key a symbolic name. This ensures that we don't
// accidentally use the same key for more than one thing.
#define WTF_THREAD_DATA_KEY WTF_FAST_TLS_KEY0
WebAssembly: store state in TLS instead of on VM https://bugs.webkit.org/show_bug.cgi?id=169611 Reviewed by Filip Pizlo. Source/JavaScriptCore: Using thread-local storage instead of VM makes code more position independent. We used to store the WebAssembly top Instance (the latest one in the call stack) on VM, now we instead store it in TLS. This top Instance is used to access a bunch of state such as Memory location, size, table (for call_indirect), etc. Instead of calling it "top", which is confusing, we now just call it WasmContext. Making the code PIC means future patches will be able to postMessage and structured clone into IDB without having to recompile the code. This wasn't possible before because we hard-coded the address of VM at compilation time. That doesn't work between workers, and doesn't work across reloads (which IDB is intended to do). It'll also potentially make code faster once we start tuning what's in TLS, what's in which of the 4 free slots, and what's in pinned registers. I'm leaving this tuning for later because there's lower lying fruit for us to pick. * CMakeLists.txt: * JavaScriptCore.xcodeproj/project.pbxproj: * assembler/AbstractMacroAssembler.h: * assembler/AllowMacroScratchRegisterUsageIf.h: Copied from assembler/AllowMacroScratchRegisterUsage.h. (JSC::AllowMacroScratchRegisterUsageIf::AllowMacroScratchRegisterUsageIf): (JSC::AllowMacroScratchRegisterUsageIf::~AllowMacroScratchRegisterUsageIf): * assembler/MacroAssembler.h: (JSC::MacroAssembler::storeToTLSPtr): we previously didn't have the code required to store to TLS, only to load * assembler/MacroAssemblerARM64.h: (JSC::MacroAssemblerARM64::loadFromTLSPtrNeedsMacroScratchRegister): (JSC::MacroAssemblerARM64::storeToTLS32): (JSC::MacroAssemblerARM64::storeToTLS64): (JSC::MacroAssemblerARM64::storeToTLSPtrNeedsMacroScratchRegister): * assembler/MacroAssemblerX86Common.h: (JSC::MacroAssemblerX86Common::loadFromTLSPtrNeedsMacroScratchRegister): (JSC::MacroAssemblerX86Common::storeToTLS32): (JSC::MacroAssemblerX86Common::storeToTLSPtrNeedsMacroScratchRegister): * assembler/MacroAssemblerX86_64.h: (JSC::MacroAssemblerX86_64::loadFromTLS64): was loading 32-bit instead of 64-bit (JSC::MacroAssemblerX86_64::storeToTLS64): * assembler/X86Assembler.h: (JSC::X86Assembler::movl_rm): (JSC::X86Assembler::movq_rm): * b3/testb3.cpp: (JSC::B3::testFastTLSLoad): (JSC::B3::testFastTLSStore): (JSC::B3::run): * jit/AssemblyHelpers.h: (JSC::AssemblyHelpers::loadWasmContext): (JSC::AssemblyHelpers::storeWasmContext): (JSC::AssemblyHelpers::loadWasmContextNeedsMacroScratchRegister): (JSC::AssemblyHelpers::storeWasmContextNeedsMacroScratchRegister): * jit/Repatch.cpp: (JSC::webAssemblyOwner): * jit/ThunkGenerators.cpp: (JSC::throwExceptionFromWasmThunkGenerator): * runtime/Options.h: * runtime/VM.cpp: (JSC::VM::VM): * runtime/VM.h: * wasm/WasmB3IRGenerator.cpp: (JSC::Wasm::loadWasmContext): (JSC::Wasm::storeWasmContext): (JSC::Wasm::B3IRGenerator::B3IRGenerator): (JSC::Wasm::getMemoryBaseAndSize): (JSC::Wasm::restoreWebAssemblyGlobalState): (JSC::Wasm::createJSToWasmWrapper): (JSC::Wasm::parseAndCompile): * wasm/WasmBinding.cpp: (JSC::Wasm::materializeImportJSCell): (JSC::Wasm::wasmToJs): (JSC::Wasm::wasmToWasm): * wasm/WasmContext.cpp: Added. (JSC::loadWasmContext): (JSC::storeWasmContext): * wasm/WasmContext.h: Added. Replaces "top" JSWebAssemblyInstance. * wasm/js/WebAssemblyFunction.cpp: (JSC::callWebAssemblyFunction): * wasm/js/WebAssemblyInstanceConstructor.h: Source/WTF: * wtf/FastTLS.h: reserve one key for WebAssembly, delete a bunch of dead code which clang couldn't compile (it's valid GCC assembly which LLVM dislikes). Canonical link: https://commits.webkit.org/187009@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@214384 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-03-24 23:25:16 +00:00
#define WTF_WASM_CONTEXT_KEY WTF_FAST_TLS_KEY1
#define WTF_TESTING_KEY WTF_WASM_CONTEXT_KEY // So far, this key is only used in places that don't do WebAssembly, so it's OK that they share the same key.
The JITs should be able to emit fast TLS loads https://bugs.webkit.org/show_bug.cgi?id=169483 Reviewed by Keith Miller. Source/JavaScriptCore: Added loadFromTLS32/64/Ptr to the MacroAssembler and added a B3 test for this. * assembler/ARM64Assembler.h: (JSC::ARM64Assembler::mrs_TPIDRRO_EL0): * assembler/MacroAssembler.h: (JSC::MacroAssembler::loadFromTLSPtr): * assembler/MacroAssemblerARM64.h: (JSC::MacroAssemblerARM64::loadFromTLS32): (JSC::MacroAssemblerARM64::loadFromTLS64): * assembler/MacroAssemblerX86Common.h: (JSC::MacroAssemblerX86Common::loadFromTLS32): * assembler/MacroAssemblerX86_64.h: (JSC::MacroAssemblerX86_64::loadFromTLS64): * assembler/X86Assembler.h: (JSC::X86Assembler::adcl_im): (JSC::X86Assembler::addl_mr): (JSC::X86Assembler::addl_im): (JSC::X86Assembler::andl_im): (JSC::X86Assembler::orl_im): (JSC::X86Assembler::orl_rm): (JSC::X86Assembler::subl_im): (JSC::X86Assembler::cmpb_im): (JSC::X86Assembler::cmpl_rm): (JSC::X86Assembler::cmpl_im): (JSC::X86Assembler::testb_im): (JSC::X86Assembler::movb_i8m): (JSC::X86Assembler::movb_rm): (JSC::X86Assembler::movl_mr): (JSC::X86Assembler::movq_mr): (JSC::X86Assembler::movsxd_rr): (JSC::X86Assembler::gs): (JSC::X86Assembler::X86InstructionFormatter::SingleInstructionBufferWriter::memoryModRM): * b3/testb3.cpp: (JSC::B3::testFastTLS): (JSC::B3::run): Source/WTF: Consolidated what we know about fast TLS in FastTLS.h. * WTF.xcodeproj/project.pbxproj: * wtf/CMakeLists.txt: * wtf/FastTLS.h: Added. (WTF::loadFastTLS): (WTF::fastTLSOffsetForKey): * wtf/Platform.h: * wtf/WTFThreadData.cpp: (WTF::WTFThreadData::createAndRegisterForGetspecificDirect): * wtf/WTFThreadData.h: (WTF::wtfThreadData): Canonical link: https://commits.webkit.org/186504@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@213753 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-03-11 07:17:48 +00:00
#if ENABLE(FAST_TLS_JIT)
inline unsigned fastTLSOffsetForKey(unsigned long slot)
{
return slot * sizeof(void*);
}
WebAssembly: store state in TLS instead of on VM https://bugs.webkit.org/show_bug.cgi?id=169611 Reviewed by Filip Pizlo. Source/JavaScriptCore: Using thread-local storage instead of VM makes code more position independent. We used to store the WebAssembly top Instance (the latest one in the call stack) on VM, now we instead store it in TLS. This top Instance is used to access a bunch of state such as Memory location, size, table (for call_indirect), etc. Instead of calling it "top", which is confusing, we now just call it WasmContext. Making the code PIC means future patches will be able to postMessage and structured clone into IDB without having to recompile the code. This wasn't possible before because we hard-coded the address of VM at compilation time. That doesn't work between workers, and doesn't work across reloads (which IDB is intended to do). It'll also potentially make code faster once we start tuning what's in TLS, what's in which of the 4 free slots, and what's in pinned registers. I'm leaving this tuning for later because there's lower lying fruit for us to pick. * CMakeLists.txt: * JavaScriptCore.xcodeproj/project.pbxproj: * assembler/AbstractMacroAssembler.h: * assembler/AllowMacroScratchRegisterUsageIf.h: Copied from assembler/AllowMacroScratchRegisterUsage.h. (JSC::AllowMacroScratchRegisterUsageIf::AllowMacroScratchRegisterUsageIf): (JSC::AllowMacroScratchRegisterUsageIf::~AllowMacroScratchRegisterUsageIf): * assembler/MacroAssembler.h: (JSC::MacroAssembler::storeToTLSPtr): we previously didn't have the code required to store to TLS, only to load * assembler/MacroAssemblerARM64.h: (JSC::MacroAssemblerARM64::loadFromTLSPtrNeedsMacroScratchRegister): (JSC::MacroAssemblerARM64::storeToTLS32): (JSC::MacroAssemblerARM64::storeToTLS64): (JSC::MacroAssemblerARM64::storeToTLSPtrNeedsMacroScratchRegister): * assembler/MacroAssemblerX86Common.h: (JSC::MacroAssemblerX86Common::loadFromTLSPtrNeedsMacroScratchRegister): (JSC::MacroAssemblerX86Common::storeToTLS32): (JSC::MacroAssemblerX86Common::storeToTLSPtrNeedsMacroScratchRegister): * assembler/MacroAssemblerX86_64.h: (JSC::MacroAssemblerX86_64::loadFromTLS64): was loading 32-bit instead of 64-bit (JSC::MacroAssemblerX86_64::storeToTLS64): * assembler/X86Assembler.h: (JSC::X86Assembler::movl_rm): (JSC::X86Assembler::movq_rm): * b3/testb3.cpp: (JSC::B3::testFastTLSLoad): (JSC::B3::testFastTLSStore): (JSC::B3::run): * jit/AssemblyHelpers.h: (JSC::AssemblyHelpers::loadWasmContext): (JSC::AssemblyHelpers::storeWasmContext): (JSC::AssemblyHelpers::loadWasmContextNeedsMacroScratchRegister): (JSC::AssemblyHelpers::storeWasmContextNeedsMacroScratchRegister): * jit/Repatch.cpp: (JSC::webAssemblyOwner): * jit/ThunkGenerators.cpp: (JSC::throwExceptionFromWasmThunkGenerator): * runtime/Options.h: * runtime/VM.cpp: (JSC::VM::VM): * runtime/VM.h: * wasm/WasmB3IRGenerator.cpp: (JSC::Wasm::loadWasmContext): (JSC::Wasm::storeWasmContext): (JSC::Wasm::B3IRGenerator::B3IRGenerator): (JSC::Wasm::getMemoryBaseAndSize): (JSC::Wasm::restoreWebAssemblyGlobalState): (JSC::Wasm::createJSToWasmWrapper): (JSC::Wasm::parseAndCompile): * wasm/WasmBinding.cpp: (JSC::Wasm::materializeImportJSCell): (JSC::Wasm::wasmToJs): (JSC::Wasm::wasmToWasm): * wasm/WasmContext.cpp: Added. (JSC::loadWasmContext): (JSC::storeWasmContext): * wasm/WasmContext.h: Added. Replaces "top" JSWebAssemblyInstance. * wasm/js/WebAssemblyFunction.cpp: (JSC::callWebAssemblyFunction): * wasm/js/WebAssemblyInstanceConstructor.h: Source/WTF: * wtf/FastTLS.h: reserve one key for WebAssembly, delete a bunch of dead code which clang couldn't compile (it's valid GCC assembly which LLVM dislikes). Canonical link: https://commits.webkit.org/187009@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@214384 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-03-24 23:25:16 +00:00
#endif
The JITs should be able to emit fast TLS loads https://bugs.webkit.org/show_bug.cgi?id=169483 Reviewed by Keith Miller. Source/JavaScriptCore: Added loadFromTLS32/64/Ptr to the MacroAssembler and added a B3 test for this. * assembler/ARM64Assembler.h: (JSC::ARM64Assembler::mrs_TPIDRRO_EL0): * assembler/MacroAssembler.h: (JSC::MacroAssembler::loadFromTLSPtr): * assembler/MacroAssemblerARM64.h: (JSC::MacroAssemblerARM64::loadFromTLS32): (JSC::MacroAssemblerARM64::loadFromTLS64): * assembler/MacroAssemblerX86Common.h: (JSC::MacroAssemblerX86Common::loadFromTLS32): * assembler/MacroAssemblerX86_64.h: (JSC::MacroAssemblerX86_64::loadFromTLS64): * assembler/X86Assembler.h: (JSC::X86Assembler::adcl_im): (JSC::X86Assembler::addl_mr): (JSC::X86Assembler::addl_im): (JSC::X86Assembler::andl_im): (JSC::X86Assembler::orl_im): (JSC::X86Assembler::orl_rm): (JSC::X86Assembler::subl_im): (JSC::X86Assembler::cmpb_im): (JSC::X86Assembler::cmpl_rm): (JSC::X86Assembler::cmpl_im): (JSC::X86Assembler::testb_im): (JSC::X86Assembler::movb_i8m): (JSC::X86Assembler::movb_rm): (JSC::X86Assembler::movl_mr): (JSC::X86Assembler::movq_mr): (JSC::X86Assembler::movsxd_rr): (JSC::X86Assembler::gs): (JSC::X86Assembler::X86InstructionFormatter::SingleInstructionBufferWriter::memoryModRM): * b3/testb3.cpp: (JSC::B3::testFastTLS): (JSC::B3::run): Source/WTF: Consolidated what we know about fast TLS in FastTLS.h. * WTF.xcodeproj/project.pbxproj: * wtf/CMakeLists.txt: * wtf/FastTLS.h: Added. (WTF::loadFastTLS): (WTF::fastTLSOffsetForKey): * wtf/Platform.h: * wtf/WTFThreadData.cpp: (WTF::WTFThreadData::createAndRegisterForGetspecificDirect): * wtf/WTFThreadData.h: (WTF::wtfThreadData): Canonical link: https://commits.webkit.org/186504@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@213753 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-03-11 07:17:48 +00:00
} // namespace WTF
#if ENABLE(FAST_TLS_JIT)
WebAssembly: store state in TLS instead of on VM https://bugs.webkit.org/show_bug.cgi?id=169611 Reviewed by Filip Pizlo. Source/JavaScriptCore: Using thread-local storage instead of VM makes code more position independent. We used to store the WebAssembly top Instance (the latest one in the call stack) on VM, now we instead store it in TLS. This top Instance is used to access a bunch of state such as Memory location, size, table (for call_indirect), etc. Instead of calling it "top", which is confusing, we now just call it WasmContext. Making the code PIC means future patches will be able to postMessage and structured clone into IDB without having to recompile the code. This wasn't possible before because we hard-coded the address of VM at compilation time. That doesn't work between workers, and doesn't work across reloads (which IDB is intended to do). It'll also potentially make code faster once we start tuning what's in TLS, what's in which of the 4 free slots, and what's in pinned registers. I'm leaving this tuning for later because there's lower lying fruit for us to pick. * CMakeLists.txt: * JavaScriptCore.xcodeproj/project.pbxproj: * assembler/AbstractMacroAssembler.h: * assembler/AllowMacroScratchRegisterUsageIf.h: Copied from assembler/AllowMacroScratchRegisterUsage.h. (JSC::AllowMacroScratchRegisterUsageIf::AllowMacroScratchRegisterUsageIf): (JSC::AllowMacroScratchRegisterUsageIf::~AllowMacroScratchRegisterUsageIf): * assembler/MacroAssembler.h: (JSC::MacroAssembler::storeToTLSPtr): we previously didn't have the code required to store to TLS, only to load * assembler/MacroAssemblerARM64.h: (JSC::MacroAssemblerARM64::loadFromTLSPtrNeedsMacroScratchRegister): (JSC::MacroAssemblerARM64::storeToTLS32): (JSC::MacroAssemblerARM64::storeToTLS64): (JSC::MacroAssemblerARM64::storeToTLSPtrNeedsMacroScratchRegister): * assembler/MacroAssemblerX86Common.h: (JSC::MacroAssemblerX86Common::loadFromTLSPtrNeedsMacroScratchRegister): (JSC::MacroAssemblerX86Common::storeToTLS32): (JSC::MacroAssemblerX86Common::storeToTLSPtrNeedsMacroScratchRegister): * assembler/MacroAssemblerX86_64.h: (JSC::MacroAssemblerX86_64::loadFromTLS64): was loading 32-bit instead of 64-bit (JSC::MacroAssemblerX86_64::storeToTLS64): * assembler/X86Assembler.h: (JSC::X86Assembler::movl_rm): (JSC::X86Assembler::movq_rm): * b3/testb3.cpp: (JSC::B3::testFastTLSLoad): (JSC::B3::testFastTLSStore): (JSC::B3::run): * jit/AssemblyHelpers.h: (JSC::AssemblyHelpers::loadWasmContext): (JSC::AssemblyHelpers::storeWasmContext): (JSC::AssemblyHelpers::loadWasmContextNeedsMacroScratchRegister): (JSC::AssemblyHelpers::storeWasmContextNeedsMacroScratchRegister): * jit/Repatch.cpp: (JSC::webAssemblyOwner): * jit/ThunkGenerators.cpp: (JSC::throwExceptionFromWasmThunkGenerator): * runtime/Options.h: * runtime/VM.cpp: (JSC::VM::VM): * runtime/VM.h: * wasm/WasmB3IRGenerator.cpp: (JSC::Wasm::loadWasmContext): (JSC::Wasm::storeWasmContext): (JSC::Wasm::B3IRGenerator::B3IRGenerator): (JSC::Wasm::getMemoryBaseAndSize): (JSC::Wasm::restoreWebAssemblyGlobalState): (JSC::Wasm::createJSToWasmWrapper): (JSC::Wasm::parseAndCompile): * wasm/WasmBinding.cpp: (JSC::Wasm::materializeImportJSCell): (JSC::Wasm::wasmToJs): (JSC::Wasm::wasmToWasm): * wasm/WasmContext.cpp: Added. (JSC::loadWasmContext): (JSC::storeWasmContext): * wasm/WasmContext.h: Added. Replaces "top" JSWebAssemblyInstance. * wasm/js/WebAssemblyFunction.cpp: (JSC::callWebAssemblyFunction): * wasm/js/WebAssemblyInstanceConstructor.h: Source/WTF: * wtf/FastTLS.h: reserve one key for WebAssembly, delete a bunch of dead code which clang couldn't compile (it's valid GCC assembly which LLVM dislikes). Canonical link: https://commits.webkit.org/187009@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@214384 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-03-24 23:25:16 +00:00
using WTF::fastTLSOffsetForKey;
The JITs should be able to emit fast TLS loads https://bugs.webkit.org/show_bug.cgi?id=169483 Reviewed by Keith Miller. Source/JavaScriptCore: Added loadFromTLS32/64/Ptr to the MacroAssembler and added a B3 test for this. * assembler/ARM64Assembler.h: (JSC::ARM64Assembler::mrs_TPIDRRO_EL0): * assembler/MacroAssembler.h: (JSC::MacroAssembler::loadFromTLSPtr): * assembler/MacroAssemblerARM64.h: (JSC::MacroAssemblerARM64::loadFromTLS32): (JSC::MacroAssemblerARM64::loadFromTLS64): * assembler/MacroAssemblerX86Common.h: (JSC::MacroAssemblerX86Common::loadFromTLS32): * assembler/MacroAssemblerX86_64.h: (JSC::MacroAssemblerX86_64::loadFromTLS64): * assembler/X86Assembler.h: (JSC::X86Assembler::adcl_im): (JSC::X86Assembler::addl_mr): (JSC::X86Assembler::addl_im): (JSC::X86Assembler::andl_im): (JSC::X86Assembler::orl_im): (JSC::X86Assembler::orl_rm): (JSC::X86Assembler::subl_im): (JSC::X86Assembler::cmpb_im): (JSC::X86Assembler::cmpl_rm): (JSC::X86Assembler::cmpl_im): (JSC::X86Assembler::testb_im): (JSC::X86Assembler::movb_i8m): (JSC::X86Assembler::movb_rm): (JSC::X86Assembler::movl_mr): (JSC::X86Assembler::movq_mr): (JSC::X86Assembler::movsxd_rr): (JSC::X86Assembler::gs): (JSC::X86Assembler::X86InstructionFormatter::SingleInstructionBufferWriter::memoryModRM): * b3/testb3.cpp: (JSC::B3::testFastTLS): (JSC::B3::run): Source/WTF: Consolidated what we know about fast TLS in FastTLS.h. * WTF.xcodeproj/project.pbxproj: * wtf/CMakeLists.txt: * wtf/FastTLS.h: Added. (WTF::loadFastTLS): (WTF::fastTLSOffsetForKey): * wtf/Platform.h: * wtf/WTFThreadData.cpp: (WTF::WTFThreadData::createAndRegisterForGetspecificDirect): * wtf/WTFThreadData.h: (WTF::wtfThreadData): Canonical link: https://commits.webkit.org/186504@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@213753 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-03-11 07:17:48 +00:00
#endif
#endif // HAVE(FAST_TLS)