haikuwebkit/Source/WTF/wtf/ConcurrentBuffer.h

115 lines
3.8 KiB
C
Raw Permalink Normal View History

It should be possible to flag a cell for unconditional finalization https://bugs.webkit.org/show_bug.cgi?id=180636 Reviewed by Saam Barati. Source/JavaScriptCore: UnconditionalFinalizers were annoying - you had to allocate them and you had to manage a global linked list - but they had some nice properties: - You only did the hardest work (creating the UnconditionalFinalizer) on first GC where you survived and needed it. -> Just needing it wasn't enough. -> Just surviving wasn't enough. The new API based on IsoSubspaces meant that just surviving was enough to cause unconditional finalizer logic to be invoked. I think that's not great. InferredType got around this by making InferredStructure a cell, but this was a gross hack. For one, it meant that InferredStructure would survive during the GC in which its finalizer obviated the need for its existence. It's not really an idiom I want us to repeat because it sounds like the sort of thing that turns out to be subtly broken. We really need to have a way of indicating when you have entered into the state that requires your unconditional finalizer to be invoked. Basically, we want to be able to track the set of objects that need unconditional finalizers. Only the subset of that set that overlaps with the set of marked objects needs to be accurate. The easiest way to do this is a hierarchy of bitvectors: one to say which MarkedBlocks have objects that have unconditional finalizers, and another level to say which atoms within a MarkedBlock have unconditional finalizers. This change introduces IsoCellSet, which couples itself to the MarkedAllocator of some IsoSubspace to allow maintaining a set of objects (well, cells - you could do this with auxiliaries) that belong to that IsoSubspace. It'll have undefined behavior if you try to add/remove/contains an object that isn't in that IsoSubspace. For objects in that subspace, you can add/remove/contains and forEachMarkedCell. The cost of each IsoCellSet is at worst about 0.8% increase in size to every object in the subspace that the set is attached to. So, it makes sense to have a handful per subspace max. This change only needs one per subspace, but you could imagine more if we do this for WeakReferenceHarvester. To absolutely minimize the possibility that this incurs costs, the add/remove/contains functions can be used from any thread so long as forEachMarkedCell isn't running. This means that InferredType only needs to add itself to the set during visitChildren. Thus, it needs to both survive and need it for the hardest work to take place. The work of adding does involve a gnarly load chain that ends in a CAS: load block handle from block, load index, load segment, load bitvector, load bit -> if not set, then CAS. That's five dependent loads! However, it's perfect for running in parallel since the only write operations are to widely dispersed cache lines that contain the bits underlying the set. The best part is how forEachMarkedCell works. That skips blocks that don't have any objects that need unconditional finalizers, and only touches the memory of marked objects that have the unconditional finalizer bit set. It will walk those objects in roughly address order. I previously found that this speeds up walking over a lot of objects when I made similar changes for DOM GC (calling visitAdditionalChildren via forEachMarkedCell rather than by walking a HashSet). This change makes InferredStructure be a malloc object again, but now it's in an IsoHeap. My expectation for this change is that it's perf-neutral. Long-term, it gives us a path forward for eliminating UnconditionalFinalizer and WeakReferenceHarvester while using IsoSubspace in more places. * JavaScriptCore.xcodeproj/project.pbxproj: * Sources.txt: * heap/AtomIndices.h: Added. (JSC::AtomIndices::AtomIndices): * heap/Heap.cpp: (JSC::Heap::finalizeUnconditionalFinalizers): * heap/Heap.h: * heap/IsoCellSet.cpp: Added. (JSC::IsoCellSet::IsoCellSet): (JSC::IsoCellSet::~IsoCellSet): (JSC::IsoCellSet::addSlow): (JSC::IsoCellSet::didResizeBits): (JSC::IsoCellSet::didRemoveBlock): (JSC::IsoCellSet::sweepToFreeList): * heap/IsoCellSet.h: Added. * heap/IsoCellSetInlines.h: Added. (JSC::IsoCellSet::add): (JSC::IsoCellSet::remove): (JSC::IsoCellSet::contains const): (JSC::IsoCellSet::forEachMarkedCell): * heap/IsoSubspace.cpp: (JSC::IsoSubspace::didResizeBits): (JSC::IsoSubspace::didRemoveBlock): (JSC::IsoSubspace::didBeginSweepingToFreeList): * heap/IsoSubspace.h: * heap/MarkedAllocator.cpp: (JSC::MarkedAllocator::addBlock): (JSC::MarkedAllocator::removeBlock): * heap/MarkedAllocator.h: * heap/MarkedAllocatorInlines.h: * heap/MarkedBlock.cpp: (JSC::MarkedBlock::Handle::sweep): (JSC::MarkedBlock::Handle::isEmpty): Deleted. * heap/MarkedBlock.h: (JSC::MarkedBlock::marks const): (JSC::MarkedBlock::Handle::newlyAllocated const): * heap/MarkedBlockInlines.h: (JSC::MarkedBlock::Handle::isAllocated): (JSC::MarkedBlock::Handle::isEmpty): (JSC::MarkedBlock::Handle::emptyMode): (JSC::MarkedBlock::Handle::forEachMarkedCell): * heap/Subspace.cpp: (JSC::Subspace::didResizeBits): (JSC::Subspace::didRemoveBlock): (JSC::Subspace::didBeginSweepingToFreeList): * heap/Subspace.h: * heap/SubspaceInlines.h: (JSC::Subspace::forEachMarkedCell): * runtime/InferredStructure.cpp: (JSC::InferredStructure::InferredStructure): (JSC::InferredStructure::create): Deleted. (JSC::InferredStructure::destroy): Deleted. (JSC::InferredStructure::createStructure): Deleted. (JSC::InferredStructure::visitChildren): Deleted. (JSC::InferredStructure::finalizeUnconditionally): Deleted. (JSC::InferredStructure::finishCreation): Deleted. * runtime/InferredStructure.h: * runtime/InferredStructureWatchpoint.cpp: (JSC::InferredStructureWatchpoint::fireInternal): * runtime/InferredType.cpp: (JSC::InferredType::visitChildren): (JSC::InferredType::willStoreValueSlow): (JSC::InferredType::makeTopSlow): (JSC::InferredType::set): (JSC::InferredType::removeStructure): (JSC::InferredType::finalizeUnconditionally): * runtime/InferredType.h: * runtime/VM.cpp: (JSC::VM::VM): * runtime/VM.h: Source/WTF: This adds ConcurrentVector, which is like SegmentedVector, but wastes some space to allow resizing to proceed concurrently to access. It's not possible to resize concurrently to resizing, concurrent read/writes aren't protected from racing if they access the same element, and who knows what you'll get if you iterate up to size() while someone else append()s. The key insight is to stash all prior copies of the spine, so that nobody crashes trying to access a stale spine. I'm going to want to do the same thing for FastBitVector, by creating a segmented WordOwner class. That would require repeating the dance of having a spine that can resize while stashing old versions. So, the spine resizing logic is abstracted behind ConcurrentBuffer. You could use that as a kind of "concurrent vector" for immutable data. That's how ConcurrentVector uses it: it's an immutable array of segment pointers. * WTF.xcodeproj/project.pbxproj: * wtf/ConcurrentBuffer.h: Added. (WTF::ConcurrentBuffer::ConcurrentBuffer): (WTF::ConcurrentBuffer::~ConcurrentBuffer): (WTF::ConcurrentBuffer::growExact): (WTF::ConcurrentBuffer::grow): (WTF::ConcurrentBuffer::array const): (WTF::ConcurrentBuffer::operator[]): (WTF::ConcurrentBuffer::operator[] const): (WTF::ConcurrentBuffer::createArray): * wtf/ConcurrentVector.h: Added. (WTF::ConcurrentVectorIterator::~ConcurrentVectorIterator): (WTF::ConcurrentVectorIterator::operator* const): (WTF::ConcurrentVectorIterator::operator-> const): (WTF::ConcurrentVectorIterator::operator++): (WTF::ConcurrentVectorIterator::operator== const): (WTF::ConcurrentVectorIterator::operator!= const): (WTF::ConcurrentVectorIterator::operator=): (WTF::ConcurrentVectorIterator::ConcurrentVectorIterator): (WTF::ConcurrentVector::~ConcurrentVector): (WTF::ConcurrentVector::size const): (WTF::ConcurrentVector::isEmpty const): (WTF::ConcurrentVector::at): (WTF::ConcurrentVector::at const): (WTF::ConcurrentVector::operator[]): (WTF::ConcurrentVector::operator[] const): (WTF::ConcurrentVector::first): (WTF::ConcurrentVector::first const): (WTF::ConcurrentVector::last): (WTF::ConcurrentVector::last const): (WTF::ConcurrentVector::takeLast): (WTF::ConcurrentVector::append): (WTF::ConcurrentVector::alloc): (WTF::ConcurrentVector::removeLast): (WTF::ConcurrentVector::grow): (WTF::ConcurrentVector::begin): (WTF::ConcurrentVector::end): (WTF::ConcurrentVector::segmentExistsFor): (WTF::ConcurrentVector::segmentFor): (WTF::ConcurrentVector::subscriptFor): (WTF::ConcurrentVector::ensureSegmentsFor): (WTF::ConcurrentVector::ensureSegment): (WTF::ConcurrentVector::allocateSegment): Canonical link: https://commits.webkit.org/196644@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@225831 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-12-13 02:35:54 +00:00
/*
* Copyright (C) 2017 Apple Inc. All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
* are met:
* 1. Redistributions of source code must retain the above copyright
* notice, this list of conditions and the following disclaimer.
* 2. Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
*
* THIS SOFTWARE IS PROVIDED BY APPLE INC. ``AS IS'' AND ANY
* EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
* PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR
* CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
* EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
* PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
* PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
* OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
* (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
* OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
#pragma once
#include <wtf/Atomics.h>
#include <wtf/FastMalloc.h>
#include <wtf/HashFunctions.h>
#include <wtf/Lock.h>
#include <wtf/Noncopyable.h>
#include <wtf/Vector.h>
namespace WTF {
Experiment: create lots of different malloc zones for easier accounting of memory use https://bugs.webkit.org/show_bug.cgi?id=186422 Patch by Yusuke Suzuki <ysuzuki@apple.com> and Simon Fraser <simon.fraser@apple.com> on 2020-01-02 Reviewed by Saam Barati. Source/bmalloc: * bmalloc/BPlatform.h: * bmalloc/Environment.cpp: (bmalloc::Environment::computeIsDebugHeapEnabled): * bmalloc/IsoHeap.h: (bmalloc::api::IsoHeap::IsoHeap): * bmalloc/IsoHeapInlines.h: (bmalloc::api::IsoHeap<Type>::IsoHeap): * bmalloc/IsoTLSInlines.h: (bmalloc::IsoTLS::allocateSlow): (bmalloc::IsoTLS::deallocateSlow): Source/JavaScriptCore: * JavaScriptCore.xcodeproj/project.pbxproj: * Sources.txt: * assembler/AssemblerBuffer.cpp: Copied from Source/JavaScriptCore/bytecode/InstructionStream.cpp. * assembler/AssemblerBuffer.h: (JSC::AssemblerData::AssemblerData): (JSC::AssemblerData::operator=): (JSC::AssemblerData::~AssemblerData): (JSC::AssemblerData::grow): * bytecode/AccessCase.cpp: * bytecode/AccessCase.h: * bytecode/BytecodeBasicBlock.cpp: * bytecode/BytecodeBasicBlock.h: * bytecode/CodeBlock.cpp: * bytecode/CodeBlock.h: * bytecode/InstructionStream.cpp: * bytecode/InstructionStream.h: * bytecode/PolymorphicAccess.cpp: * bytecode/PolymorphicAccess.h: * bytecode/UnlinkedMetadataTable.cpp: (JSC::UnlinkedMetadataTable::finalize): * bytecode/UnlinkedMetadataTable.h: * bytecode/UnlinkedMetadataTableInlines.h: (JSC::UnlinkedMetadataTable::UnlinkedMetadataTable): (JSC::UnlinkedMetadataTable::~UnlinkedMetadataTable): (JSC::UnlinkedMetadataTable::link): (JSC::UnlinkedMetadataTable::unlink): * bytecode/ValueProfile.h: (JSC::ValueProfileAndVirtualRegisterBuffer::ValueProfileAndVirtualRegisterBuffer): * bytecode/Watchpoint.cpp: * bytecode/Watchpoint.h: * dfg/DFGBasicBlock.cpp: * dfg/DFGBasicBlock.h: * dfg/DFGNode.cpp: * dfg/DFGNode.h: * dfg/DFGSpeculativeJIT.cpp: * dfg/DFGSpeculativeJIT.h: * heap/BlockDirectory.cpp: * heap/BlockDirectory.h: * heap/FastMallocAlignedMemoryAllocator.cpp: (JSC::FastMallocAlignedMemoryAllocator::FastMallocAlignedMemoryAllocator): (JSC::FastMallocAlignedMemoryAllocator::tryAllocateAlignedMemory): (JSC::FastMallocAlignedMemoryAllocator::freeAlignedMemory): (JSC::FastMallocAlignedMemoryAllocator::tryAllocateMemory): (JSC::FastMallocAlignedMemoryAllocator::freeMemory): (JSC::FastMallocAlignedMemoryAllocator::tryReallocateMemory): * heap/FastMallocAlignedMemoryAllocator.h: * heap/GCSegmentedArray.cpp: Copied from Source/JavaScriptCore/parser/SourceProviderCache.cpp. * heap/GCSegmentedArray.h: * heap/GCSegmentedArrayInlines.h: (JSC::GCArraySegment<T>::create): (JSC::GCArraySegment<T>::destroy): * heap/GigacageAlignedMemoryAllocator.cpp: (JSC::GigacageAlignedMemoryAllocator::GigacageAlignedMemoryAllocator): (JSC::GigacageAlignedMemoryAllocator::tryAllocateAlignedMemory): (JSC::GigacageAlignedMemoryAllocator::freeAlignedMemory): (JSC::GigacageAlignedMemoryAllocator::tryAllocateMemory): (JSC::GigacageAlignedMemoryAllocator::freeMemory): (JSC::GigacageAlignedMemoryAllocator::tryReallocateMemory): * heap/GigacageAlignedMemoryAllocator.h: * heap/IsoAlignedMemoryAllocator.cpp: (JSC::IsoAlignedMemoryAllocator::IsoAlignedMemoryAllocator): (JSC::IsoAlignedMemoryAllocator::~IsoAlignedMemoryAllocator): (JSC::IsoAlignedMemoryAllocator::tryAllocateAlignedMemory): (JSC::IsoAlignedMemoryAllocator::freeAlignedMemory): (JSC::IsoAlignedMemoryAllocator::tryAllocateMemory): (JSC::IsoAlignedMemoryAllocator::freeMemory): * heap/IsoAlignedMemoryAllocator.h: * heap/IsoSubspace.cpp: (JSC::IsoSubspace::IsoSubspace): * heap/MarkedBlock.cpp: * heap/MarkedBlock.h: * heap/WeakBlock.cpp: (JSC::WeakBlock::create): (JSC::WeakBlock::destroy): * heap/WeakBlock.h: * jit/JITCode.cpp: * jit/JITCode.h: * jit/RegisterAtOffsetList.cpp: * jit/RegisterAtOffsetList.h: * parser/Nodes.cpp: * parser/Nodes.h: * parser/ParserArena.cpp: (JSC::ParserArena::deallocateObjects): (JSC::ParserArena::allocateFreeablePool): * parser/ParserArena.h: * parser/SourceProvider.cpp: * parser/SourceProvider.h: * parser/SourceProviderCache.cpp: * parser/SourceProviderCache.h: * parser/SourceProviderCacheItem.h: (JSC::SourceProviderCacheItem::create): * runtime/CachePayload.cpp: (JSC::CachePayload::makeMallocPayload): * runtime/CachePayload.h: * runtime/CachedBytecode.h: (JSC::CachedBytecode::create): * runtime/CachedTypes.cpp: (JSC::Encoder::release): (JSC::Encoder::Page::Page): (JSC::CachedVector::encode): (JSC::CachedVector::decode const): (JSC::CachedInstructionStream::decode const): * runtime/PropertyMapHashTable.h: (JSC::PropertyTable::rehash): * runtime/PropertyTable.cpp: (JSC::PropertyTable::PropertyTable): (JSC::PropertyTable::~PropertyTable): * runtime/SymbolTable.cpp: * runtime/SymbolTable.h: * runtime/VM.cpp: (JSC::VM::~VM): * runtime/VM.h: (JSC::ScratchBuffer::create): (JSC::VM::exceptionFuzzingBuffer): * wasm/WasmInstance.cpp: (JSC::Wasm::Instance::Instance): * wasm/WasmInstance.h: * wasm/WasmTable.cpp: (JSC::Wasm::Table::Table): (JSC::Wasm::FuncRefTable::FuncRefTable): * wasm/WasmTable.h: Source/WebCore: * Sources.txt: * WebCore.xcodeproj/project.pbxproj: * bindings/js/SerializedScriptValue.cpp: * bindings/js/SerializedScriptValue.h: * css/CSSFontFace.cpp: * css/CSSFontFace.h: * css/CSSSelector.cpp: * css/CSSSelector.h: * css/CSSValue.cpp: * css/CSSValue.h: * css/StyleProperties.cpp: (WebCore::ImmutableStyleProperties::create): * css/StyleProperties.h: * css/StyleRule.cpp: * css/StyleRule.h: * dom/ElementData.cpp: (WebCore::ShareableElementData::createWithAttributes): (WebCore::UniqueElementData::makeShareableCopy const): * dom/ElementData.h: * dom/NodeRareData.cpp: * dom/NodeRareData.h: * dom/QualifiedName.cpp: * dom/QualifiedName.h: * html/parser/HTMLDocumentParser.cpp: * html/parser/HTMLDocumentParser.h: * loader/DocumentLoader.cpp: * loader/DocumentLoader.h: * loader/ResourceLoader.cpp: * loader/ResourceLoader.h: * loader/cache/CachedResource.cpp: * loader/cache/CachedResource.h: * page/PerformanceEntry.cpp: * page/PerformanceEntry.h: * platform/graphics/Font.cpp: * platform/graphics/Font.h: * platform/graphics/FontCascadeFonts.cpp: * platform/graphics/FontCascadeFonts.h: * platform/graphics/Region.cpp: * platform/graphics/Region.h: * platform/graphics/avfoundation/objc/MediaSampleAVFObjC.mm: (WebCore::releaseUint8Vector): * platform/graphics/cg/ImageBufferCG.cpp: (WebCore::ImageBuffer::ImageBuffer): * platform/graphics/nicosia/NicosiaBuffer.cpp: (Nicosia::Buffer::Buffer): * platform/network/ResourceHandle.cpp: * platform/network/ResourceHandleInternal.h: * platform/network/cf/FormDataStreamCFNet.cpp: (WebCore::closeCurrentStream): (WebCore::advanceCurrentStream): * rendering/RenderLayer.cpp: * rendering/RenderLayer.h: * rendering/TableLayout.cpp: Copied from Source/JavaScriptCore/parser/SourceProviderCache.cpp. * rendering/TableLayout.h: * rendering/style/RenderStyle.cpp: * rendering/style/RenderStyle.h: * rendering/style/SVGRenderStyle.cpp: * rendering/style/SVGRenderStyle.h: * rendering/style/SVGRenderStyleDefs.cpp: * rendering/style/SVGRenderStyleDefs.h: * rendering/style/StyleBoxData.cpp: * rendering/style/StyleBoxData.h: * rendering/style/StyleInheritedData.cpp: * rendering/style/StyleInheritedData.h: * rendering/style/StyleRareInheritedData.cpp: * rendering/style/StyleRareInheritedData.h: * rendering/style/StyleRareNonInheritedData.cpp: * rendering/style/StyleRareNonInheritedData.h: * rendering/style/StyleSurroundData.cpp: * rendering/style/StyleSurroundData.h: * rendering/style/StyleTransformData.cpp: * rendering/style/StyleTransformData.h: * style/StyleTreeResolver.cpp: * style/StyleTreeResolver.h: * svg/animation/SMILTimeContainer.cpp: * svg/animation/SMILTimeContainer.h: Source/WebKit: * Shared/ShareableBitmap.cpp: (WebKit::ShareableBitmap::create): (WebKit::ShareableBitmap::~ShareableBitmap): * UIProcess/mac/LegacySessionStateCoding.cpp: (WebKit::HistoryEntryDataEncoder::HistoryEntryDataEncoder): (WebKit::HistoryEntryDataEncoder::finishEncoding): (WebKit::encodeSessionHistoryEntryData): (WebKit::encodeLegacySessionState): Source/WTF: This patch introduces ENABLE(MALLOC_HEAP_BREAKDOWN). If this is enabled, we allocate malloc_zone per malloc kind. This offers the way to investigate the usage of memory per kind by using vmmap, like the following. VIRTUAL RESIDENT DIRTY SWAPPED ALLOCATION BYTES DIRTY+SWAP REGION MALLOC ZONE SIZE SIZE SIZE SIZE COUNT ALLOCATED FRAG SIZE % FRAG COUNT =========== ======= ========= ========= ========= ========= ========= ========= ====== ====== StringImpl_0x116efd000 188.0M 69.3M 30.9M 0K 139456 18.0M 12.9M 42% 34 DefaultMallocZone_0x10f487000 176.0M 53.9M 14.1M 0K 115956 9955K 4497K 32% 22 Vector_0x116eff000 162.0M 56.3M 55.3M 0K 140715 17.3M 37.9M 69% 36 MetadataTable_0x11843b000 152.0M 17.5M 17.5M 0K 14200 2353K 15.2M 87% 26 WebKit Using System Malloc_0x114cbe000 150.0M 31.6M 21.8M 0K 87422 16.7M 5278K 24% 23 InstructionStream_0x118469000 150.0M 5764K 5764K 0K 14470 4688K 1076K 19% 24 AssemblerData_0x117ee6000 150.0M 1928K 1928K 0K 1 16 1928K 100% 24 To achieve this goal without making very large change, we put a template type in various containers. For example, Vector will take Malloc parameter (the default one is FastMalloc allocator). If ENABLE(MALLOC_HEAP_BREAKDOWN) is enabled, we change this to specific VectorMalloc allocator, and vmmap can show memory usage of this allocator. This patch also supports malloc_zone per IsoHeap. So we can see memory allocation per IsoHeap in vmmap. To use this feature, we need to flip two compile time flags, ENABLE(MALLOC_HEAP_BREAKDOWN) in WTF and BENABLE_MALLOC_HEAP_BREAKDOWN in bmalloc. And use `vmmap $PID` to dump malloc zones. To allocate objects of a class with a specific malloc-zone, use WTF_MAKE_FAST_ALLOCATED_WITH_HEAP_IDENTIFIER(HeapIdentifier) for the class, and define allocator by DECLARE_ALLOCATOR_WITH_HEAP_IDENTIFIER(HeapIdentifier) in a header and DEFINE_ALLOCATOR_WITH_HEAP_IDENTIFIER(HeapIdentifier) in a cpp file. This patch also introduce callstack collector for malloc. Vector, HashMap etc. are used to allocate various things, but the above malloc_zone feature only tells thing like "Vector takes XXX MB memory". But what we want to know in this case is what Vector is consuming memory. We collect StackShot for each malloc call, and combine these information to tell which callsite is consuming much memory, which tell us that what Vector is consuming memory. * WTF.xcodeproj/project.pbxproj: * wtf/Bag.cpp: Copied from Source/JavaScriptCore/parser/SourceProviderCache.cpp. * wtf/Bag.h: (WTF::Private::BagNode::BagNode): Deleted. * wtf/BitVector.cpp: (WTF::BitVector::OutOfLineBits::create): (WTF::BitVector::OutOfLineBits::destroy): * wtf/CMakeLists.txt: * wtf/ConcurrentBuffer.cpp: Copied from Source/JavaScriptCore/parser/SourceProviderCache.cpp. * wtf/ConcurrentBuffer.h: * wtf/DebugHeap.cpp: Copied from Source/JavaScriptCore/runtime/CachePayload.cpp. (WTF::DebugHeap::DebugHeap): (WTF::DebugHeap::malloc): (WTF::DebugHeap::calloc): (WTF::DebugHeap::memalign): (WTF::DebugHeap::realloc): (WTF::DebugHeap::free): * wtf/DebugHeap.h: Added. * wtf/FastBitVector.cpp: (WTF::FastBitVectorWordOwner::setEqualsSlow): (WTF::FastBitVectorWordOwner::resizeSlow): * wtf/FastBitVector.h: (WTF::FastBitVectorWordOwner::~FastBitVectorWordOwner): * wtf/FastMalloc.cpp: (WTF::fastMallocDumpMallocStats): (WTF::AvoidRecordingScope::AvoidRecordingScope): (WTF::AvoidRecordingScope::~AvoidRecordingScope): (WTF::MallocCallTracker::MallocSiteData::MallocSiteData): (WTF::MallocCallTracker::singleton): (WTF::MallocCallTracker::MallocCallTracker): (WTF::MallocCallTracker::recordMalloc): (WTF::MallocCallTracker::recordRealloc): (WTF::MallocCallTracker::recordFree): (WTF::MallocCallTracker::dumpStats): (WTF::fastMalloc): (WTF::fastRealloc): (WTF::fastFree): (WTF::fastAlignedMalloc): (WTF::tryFastAlignedMalloc): (WTF::fastAlignedFree): * wtf/FastMalloc.h: (WTF::FastMalloc::zeroedMalloc): (WTF::FastMalloc::tryZeroedMalloc): * wtf/Forward.h: * wtf/HashTable.cpp: * wtf/HashTable.h: (WTF::KeyTraits>::allocateTable): (WTF::KeyTraits>::deallocateTable): (WTF::KeyTraits>::rehash): * wtf/MallocPtr.h: (WTF::MallocPtr::MallocPtr): (WTF::MallocPtr::malloc): (WTF::MallocPtr::zeroedMalloc): (WTF::MallocPtr::tryMalloc): (WTF::MallocPtr::tryZeroedMalloc): (WTF::adoptMallocPtr): * wtf/MetaAllocator.cpp: (WTF::MetaAllocator::allocFreeSpaceNode): (WTF::MetaAllocator::freeFreeSpaceNode): * wtf/MetaAllocatorHandle.h: * wtf/Platform.h: * wtf/RefCountedArray.cpp: Copied from Source/JavaScriptCore/bytecode/InstructionStream.cpp. * wtf/RefCountedArray.h: (WTF::RefCountedArray::RefCountedArray): (WTF::RefCountedArray::~RefCountedArray): (WTF::RefCountedArray::assign): * wtf/SegmentedVector.cpp: Copied from Source/JavaScriptCore/bytecode/InstructionStream.cpp. * wtf/SegmentedVector.h: * wtf/SmallPtrSet.cpp: Copied from Source/JavaScriptCore/bytecode/InstructionStream.cpp. * wtf/SmallPtrSet.h: (WTF::SmallPtrSet::~SmallPtrSet): (WTF::SmallPtrSet::grow): * wtf/UniqueArray.cpp: Copied from Source/JavaScriptCore/bytecode/InstructionStream.cpp. * wtf/UniqueArray.h: (WTF::UniqueArrayFree::operator() const): (WTF::UniqueArrayFree<T::operator() const): * wtf/Vector.cpp: Copied from Source/JavaScriptCore/bytecode/InstructionStream.cpp. * wtf/Vector.h: (WTF::VectorBufferBase::allocateBuffer): (WTF::VectorBufferBase::tryAllocateBuffer): (WTF::VectorBufferBase::reallocateBuffer): (WTF::VectorBufferBase::deallocateBuffer): (WTF::VectorBufferBase::releaseBuffer): (WTF::VectorBuffer::releaseBuffer): (WTF::Vector::swap): (WTF::Malloc>::Vector): (WTF::=): (WTF::Malloc>::contains const): (WTF::Malloc>::findMatching const): (WTF::Malloc>::find const): (WTF::Malloc>::reverseFind const): (WTF::Malloc>::appendIfNotContains): (WTF::Malloc>::fill): (WTF::Malloc>::appendRange): (WTF::Malloc>::expandCapacity): (WTF::Malloc>::tryExpandCapacity): (WTF::Malloc>::resize): (WTF::Malloc>::resizeToFit): (WTF::Malloc>::shrink): (WTF::Malloc>::grow): (WTF::Malloc>::asanSetInitialBufferSizeTo): (WTF::Malloc>::asanSetBufferSizeToFullCapacity): (WTF::Malloc>::asanBufferSizeWillChangeTo): (WTF::Malloc>::reserveCapacity): (WTF::Malloc>::tryReserveCapacity): (WTF::Malloc>::reserveInitialCapacity): (WTF::Malloc>::shrinkCapacity): (WTF::Malloc>::append): (WTF::Malloc>::tryAppend): (WTF::Malloc>::constructAndAppend): (WTF::Malloc>::tryConstructAndAppend): (WTF::Malloc>::appendSlowCase): (WTF::Malloc>::constructAndAppendSlowCase): (WTF::Malloc>::tryConstructAndAppendSlowCase): (WTF::Malloc>::uncheckedAppend): (WTF::Malloc>::uncheckedConstructAndAppend): (WTF::Malloc>::appendVector): (WTF::Malloc>::insert): (WTF::Malloc>::insertVector): (WTF::Malloc>::remove): (WTF::Malloc>::removeFirst): (WTF::Malloc>::removeFirstMatching): (WTF::Malloc>::removeAll): (WTF::Malloc>::removeAllMatching): (WTF::Malloc>::reverse): (WTF::Malloc>::map const): (WTF::Malloc>::releaseBuffer): (WTF::Malloc>::checkConsistency): (WTF::swap): (WTF::operator==): (WTF::operator!=): (WTF::Malloc>::isolatedCopy const): (WTF::removeRepeatedElements): (WTF::minCapacity>::Vector): Deleted. (WTF::minCapacity>::contains const): Deleted. (WTF::minCapacity>::findMatching const): Deleted. (WTF::minCapacity>::find const): Deleted. (WTF::minCapacity>::reverseFind const): Deleted. (WTF::minCapacity>::appendIfNotContains): Deleted. (WTF::minCapacity>::fill): Deleted. (WTF::minCapacity>::appendRange): Deleted. (WTF::minCapacity>::expandCapacity): Deleted. (WTF::minCapacity>::tryExpandCapacity): Deleted. (WTF::minCapacity>::resize): Deleted. (WTF::minCapacity>::resizeToFit): Deleted. (WTF::minCapacity>::shrink): Deleted. (WTF::minCapacity>::grow): Deleted. (WTF::minCapacity>::asanSetInitialBufferSizeTo): Deleted. (WTF::minCapacity>::asanSetBufferSizeToFullCapacity): Deleted. (WTF::minCapacity>::asanBufferSizeWillChangeTo): Deleted. (WTF::minCapacity>::reserveCapacity): Deleted. (WTF::minCapacity>::tryReserveCapacity): Deleted. (WTF::minCapacity>::reserveInitialCapacity): Deleted. (WTF::minCapacity>::shrinkCapacity): Deleted. (WTF::minCapacity>::append): Deleted. (WTF::minCapacity>::tryAppend): Deleted. (WTF::minCapacity>::constructAndAppend): Deleted. (WTF::minCapacity>::tryConstructAndAppend): Deleted. (WTF::minCapacity>::appendSlowCase): Deleted. (WTF::minCapacity>::constructAndAppendSlowCase): Deleted. (WTF::minCapacity>::tryConstructAndAppendSlowCase): Deleted. (WTF::minCapacity>::uncheckedAppend): Deleted. (WTF::minCapacity>::uncheckedConstructAndAppend): Deleted. (WTF::minCapacity>::appendVector): Deleted. (WTF::minCapacity>::insert): Deleted. (WTF::minCapacity>::insertVector): Deleted. (WTF::minCapacity>::remove): Deleted. (WTF::minCapacity>::removeFirst): Deleted. (WTF::minCapacity>::removeFirstMatching): Deleted. (WTF::minCapacity>::removeAll): Deleted. (WTF::minCapacity>::removeAllMatching): Deleted. (WTF::minCapacity>::reverse): Deleted. (WTF::minCapacity>::map const): Deleted. (WTF::minCapacity>::releaseBuffer): Deleted. (WTF::minCapacity>::checkConsistency): Deleted. (WTF::minCapacity>::isolatedCopy const): Deleted. * wtf/text/CString.cpp: (WTF::CStringBuffer::createUninitialized): * wtf/text/CString.h: * wtf/text/StringBuffer.cpp: Copied from Source/JavaScriptCore/bytecode/InstructionStream.cpp. * wtf/text/StringBuffer.h: (WTF::StringBuffer::StringBuffer): (WTF::StringBuffer::~StringBuffer): (WTF::StringBuffer::resize): (WTF::StringBuffer::release): * wtf/text/StringImpl.cpp: (WTF::StringImpl::~StringImpl): (WTF::StringImpl::destroy): (WTF::StringImpl::createUninitializedInternalNonEmpty): (WTF::StringImpl::reallocateInternal): * wtf/text/StringImpl.h: (WTF::StringImpl::StringImpl): (WTF::StringImpl::createSubstringSharingImpl): (WTF::StringImpl::tryCreateUninitialized): (WTF::StringImpl::adopt): * wtf/text/cf/StringImplCF.cpp: (WTF::StringWrapperCFAllocator::allocate): (WTF::StringWrapperCFAllocator::reallocate): (WTF::StringWrapperCFAllocator::deallocate): Canonical link: https://commits.webkit.org/218863@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@253987 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2020-01-03 02:36:43 +00:00
DECLARE_ALLOCATOR_WITH_HEAP_IDENTIFIER(ConcurrentBuffer);
It should be possible to flag a cell for unconditional finalization https://bugs.webkit.org/show_bug.cgi?id=180636 Reviewed by Saam Barati. Source/JavaScriptCore: UnconditionalFinalizers were annoying - you had to allocate them and you had to manage a global linked list - but they had some nice properties: - You only did the hardest work (creating the UnconditionalFinalizer) on first GC where you survived and needed it. -> Just needing it wasn't enough. -> Just surviving wasn't enough. The new API based on IsoSubspaces meant that just surviving was enough to cause unconditional finalizer logic to be invoked. I think that's not great. InferredType got around this by making InferredStructure a cell, but this was a gross hack. For one, it meant that InferredStructure would survive during the GC in which its finalizer obviated the need for its existence. It's not really an idiom I want us to repeat because it sounds like the sort of thing that turns out to be subtly broken. We really need to have a way of indicating when you have entered into the state that requires your unconditional finalizer to be invoked. Basically, we want to be able to track the set of objects that need unconditional finalizers. Only the subset of that set that overlaps with the set of marked objects needs to be accurate. The easiest way to do this is a hierarchy of bitvectors: one to say which MarkedBlocks have objects that have unconditional finalizers, and another level to say which atoms within a MarkedBlock have unconditional finalizers. This change introduces IsoCellSet, which couples itself to the MarkedAllocator of some IsoSubspace to allow maintaining a set of objects (well, cells - you could do this with auxiliaries) that belong to that IsoSubspace. It'll have undefined behavior if you try to add/remove/contains an object that isn't in that IsoSubspace. For objects in that subspace, you can add/remove/contains and forEachMarkedCell. The cost of each IsoCellSet is at worst about 0.8% increase in size to every object in the subspace that the set is attached to. So, it makes sense to have a handful per subspace max. This change only needs one per subspace, but you could imagine more if we do this for WeakReferenceHarvester. To absolutely minimize the possibility that this incurs costs, the add/remove/contains functions can be used from any thread so long as forEachMarkedCell isn't running. This means that InferredType only needs to add itself to the set during visitChildren. Thus, it needs to both survive and need it for the hardest work to take place. The work of adding does involve a gnarly load chain that ends in a CAS: load block handle from block, load index, load segment, load bitvector, load bit -> if not set, then CAS. That's five dependent loads! However, it's perfect for running in parallel since the only write operations are to widely dispersed cache lines that contain the bits underlying the set. The best part is how forEachMarkedCell works. That skips blocks that don't have any objects that need unconditional finalizers, and only touches the memory of marked objects that have the unconditional finalizer bit set. It will walk those objects in roughly address order. I previously found that this speeds up walking over a lot of objects when I made similar changes for DOM GC (calling visitAdditionalChildren via forEachMarkedCell rather than by walking a HashSet). This change makes InferredStructure be a malloc object again, but now it's in an IsoHeap. My expectation for this change is that it's perf-neutral. Long-term, it gives us a path forward for eliminating UnconditionalFinalizer and WeakReferenceHarvester while using IsoSubspace in more places. * JavaScriptCore.xcodeproj/project.pbxproj: * Sources.txt: * heap/AtomIndices.h: Added. (JSC::AtomIndices::AtomIndices): * heap/Heap.cpp: (JSC::Heap::finalizeUnconditionalFinalizers): * heap/Heap.h: * heap/IsoCellSet.cpp: Added. (JSC::IsoCellSet::IsoCellSet): (JSC::IsoCellSet::~IsoCellSet): (JSC::IsoCellSet::addSlow): (JSC::IsoCellSet::didResizeBits): (JSC::IsoCellSet::didRemoveBlock): (JSC::IsoCellSet::sweepToFreeList): * heap/IsoCellSet.h: Added. * heap/IsoCellSetInlines.h: Added. (JSC::IsoCellSet::add): (JSC::IsoCellSet::remove): (JSC::IsoCellSet::contains const): (JSC::IsoCellSet::forEachMarkedCell): * heap/IsoSubspace.cpp: (JSC::IsoSubspace::didResizeBits): (JSC::IsoSubspace::didRemoveBlock): (JSC::IsoSubspace::didBeginSweepingToFreeList): * heap/IsoSubspace.h: * heap/MarkedAllocator.cpp: (JSC::MarkedAllocator::addBlock): (JSC::MarkedAllocator::removeBlock): * heap/MarkedAllocator.h: * heap/MarkedAllocatorInlines.h: * heap/MarkedBlock.cpp: (JSC::MarkedBlock::Handle::sweep): (JSC::MarkedBlock::Handle::isEmpty): Deleted. * heap/MarkedBlock.h: (JSC::MarkedBlock::marks const): (JSC::MarkedBlock::Handle::newlyAllocated const): * heap/MarkedBlockInlines.h: (JSC::MarkedBlock::Handle::isAllocated): (JSC::MarkedBlock::Handle::isEmpty): (JSC::MarkedBlock::Handle::emptyMode): (JSC::MarkedBlock::Handle::forEachMarkedCell): * heap/Subspace.cpp: (JSC::Subspace::didResizeBits): (JSC::Subspace::didRemoveBlock): (JSC::Subspace::didBeginSweepingToFreeList): * heap/Subspace.h: * heap/SubspaceInlines.h: (JSC::Subspace::forEachMarkedCell): * runtime/InferredStructure.cpp: (JSC::InferredStructure::InferredStructure): (JSC::InferredStructure::create): Deleted. (JSC::InferredStructure::destroy): Deleted. (JSC::InferredStructure::createStructure): Deleted. (JSC::InferredStructure::visitChildren): Deleted. (JSC::InferredStructure::finalizeUnconditionally): Deleted. (JSC::InferredStructure::finishCreation): Deleted. * runtime/InferredStructure.h: * runtime/InferredStructureWatchpoint.cpp: (JSC::InferredStructureWatchpoint::fireInternal): * runtime/InferredType.cpp: (JSC::InferredType::visitChildren): (JSC::InferredType::willStoreValueSlow): (JSC::InferredType::makeTopSlow): (JSC::InferredType::set): (JSC::InferredType::removeStructure): (JSC::InferredType::finalizeUnconditionally): * runtime/InferredType.h: * runtime/VM.cpp: (JSC::VM::VM): * runtime/VM.h: Source/WTF: This adds ConcurrentVector, which is like SegmentedVector, but wastes some space to allow resizing to proceed concurrently to access. It's not possible to resize concurrently to resizing, concurrent read/writes aren't protected from racing if they access the same element, and who knows what you'll get if you iterate up to size() while someone else append()s. The key insight is to stash all prior copies of the spine, so that nobody crashes trying to access a stale spine. I'm going to want to do the same thing for FastBitVector, by creating a segmented WordOwner class. That would require repeating the dance of having a spine that can resize while stashing old versions. So, the spine resizing logic is abstracted behind ConcurrentBuffer. You could use that as a kind of "concurrent vector" for immutable data. That's how ConcurrentVector uses it: it's an immutable array of segment pointers. * WTF.xcodeproj/project.pbxproj: * wtf/ConcurrentBuffer.h: Added. (WTF::ConcurrentBuffer::ConcurrentBuffer): (WTF::ConcurrentBuffer::~ConcurrentBuffer): (WTF::ConcurrentBuffer::growExact): (WTF::ConcurrentBuffer::grow): (WTF::ConcurrentBuffer::array const): (WTF::ConcurrentBuffer::operator[]): (WTF::ConcurrentBuffer::operator[] const): (WTF::ConcurrentBuffer::createArray): * wtf/ConcurrentVector.h: Added. (WTF::ConcurrentVectorIterator::~ConcurrentVectorIterator): (WTF::ConcurrentVectorIterator::operator* const): (WTF::ConcurrentVectorIterator::operator-> const): (WTF::ConcurrentVectorIterator::operator++): (WTF::ConcurrentVectorIterator::operator== const): (WTF::ConcurrentVectorIterator::operator!= const): (WTF::ConcurrentVectorIterator::operator=): (WTF::ConcurrentVectorIterator::ConcurrentVectorIterator): (WTF::ConcurrentVector::~ConcurrentVector): (WTF::ConcurrentVector::size const): (WTF::ConcurrentVector::isEmpty const): (WTF::ConcurrentVector::at): (WTF::ConcurrentVector::at const): (WTF::ConcurrentVector::operator[]): (WTF::ConcurrentVector::operator[] const): (WTF::ConcurrentVector::first): (WTF::ConcurrentVector::first const): (WTF::ConcurrentVector::last): (WTF::ConcurrentVector::last const): (WTF::ConcurrentVector::takeLast): (WTF::ConcurrentVector::append): (WTF::ConcurrentVector::alloc): (WTF::ConcurrentVector::removeLast): (WTF::ConcurrentVector::grow): (WTF::ConcurrentVector::begin): (WTF::ConcurrentVector::end): (WTF::ConcurrentVector::segmentExistsFor): (WTF::ConcurrentVector::segmentFor): (WTF::ConcurrentVector::subscriptFor): (WTF::ConcurrentVector::ensureSegmentsFor): (WTF::ConcurrentVector::ensureSegment): (WTF::ConcurrentVector::allocateSegment): Canonical link: https://commits.webkit.org/196644@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@225831 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-12-13 02:35:54 +00:00
// ConcurrentBuffer is suitable for when you plan to store immutable data and sometimes append to it.
// It supports storing data that is not copy-constructable but bit-copyable.
template<typename T>
[WTF][JSC] Make JSC and WTF aggressively-fast-malloced https://bugs.webkit.org/show_bug.cgi?id=200611 Reviewed by Saam Barati. Source/JavaScriptCore: This patch aggressively puts many classes into FastMalloc. In JSC side, we grep `std::make_unique` etc. to find potentially system-malloc-allocated classes. After this patch, all the JSC related allocations in JetStream2 cli is done from bmalloc. In the future, it would be nice that we add `WTF::makeUnique<T>` helper function and throw a compile error if `T` is not FastMalloc annotated[1]. Putting WebKit classes in FastMalloc has many benefits. 1. Simply, it is fast. 2. vmmap can tell the amount of memory used for WebKit. 3. bmalloc can isolate WebKit memory allocation from the rest of the world. This is useful since we can know more about what component is corrupting the memory from the memory corruption crash. [1]: https://bugs.webkit.org/show_bug.cgi?id=200620 * API/ObjCCallbackFunction.mm: * assembler/AbstractMacroAssembler.h: * b3/B3PhiChildren.h: * b3/air/AirAllocateRegistersAndStackAndGenerateCode.h: * b3/air/AirDisassembler.h: * bytecode/AccessCaseSnippetParams.h: * bytecode/CallVariant.h: * bytecode/DeferredSourceDump.h: * bytecode/ExecutionCounter.h: * bytecode/GetByIdStatus.h: * bytecode/GetByIdVariant.h: * bytecode/InByIdStatus.h: * bytecode/InByIdVariant.h: * bytecode/InstanceOfStatus.h: * bytecode/InstanceOfVariant.h: * bytecode/PutByIdStatus.h: * bytecode/PutByIdVariant.h: * bytecode/ValueProfile.h: * dfg/DFGAbstractInterpreter.h: * dfg/DFGByteCodeParser.cpp: (JSC::DFG::ByteCodeParser::newVariableAccessData): * dfg/DFGFlowIndexing.h: * dfg/DFGFlowMap.h: * dfg/DFGLiveCatchVariablePreservationPhase.cpp: (JSC::DFG::LiveCatchVariablePreservationPhase::newVariableAccessData): * dfg/DFGMaximalFlushInsertionPhase.cpp: (JSC::DFG::MaximalFlushInsertionPhase::newVariableAccessData): * dfg/DFGOSRExit.h: * dfg/DFGSpeculativeJIT.h: * dfg/DFGVariableAccessData.h: * disassembler/ARM64/A64DOpcode.h: * inspector/remote/socket/RemoteInspectorMessageParser.h: * inspector/remote/socket/RemoteInspectorSocket.h: * inspector/remote/socket/RemoteInspectorSocketEndpoint.h: * jit/PCToCodeOriginMap.h: * runtime/BasicBlockLocation.h: * runtime/DoublePredictionFuzzerAgent.h: * runtime/JSRunLoopTimer.h: * runtime/PromiseDeferredTimer.h: (JSC::PromiseDeferredTimer::create): PromiseDeferredTimer should be allocated as `Ref<>` instead of `std::unique_ptr` since it is inheriting ThreadSafeRefCounted<>. Holding such a class with std::unique_ptr could lead to potentially dangerous operations (like, someone holds it with Ref<> while it is deleted by std::unique_ptr<>). * runtime/RandomizingFuzzerAgent.h: * runtime/SymbolTable.h: * runtime/VM.cpp: (JSC::VM::VM): * runtime/VM.h: * tools/JSDollarVM.cpp: * tools/SigillCrashAnalyzer.cpp: * wasm/WasmFormat.h: * wasm/WasmMemory.cpp: * wasm/WasmSignature.h: * yarr/YarrJIT.h: Source/WebCore: Changed the accessor since we changed std::unique_ptr to Ref for this field. No behavior change. * bindings/js/WorkerScriptController.cpp: (WebCore::WorkerScriptController::addTimerSetNotification): (WebCore::WorkerScriptController::removeTimerSetNotification): Source/WTF: WTF has many data structures, in particular, containers. And these containers can be allocated like `std::make_unique<Container>()`. Without WTF_MAKE_FAST_ALLOCATED, this container itself is allocated from the system malloc. This patch attaches WTF_MAKE_FAST_ALLOCATED more aggressively not to allocate them from the system malloc. And we add some `final` to containers and classes that would be never inherited. * wtf/Assertions.cpp: * wtf/Atomics.h: * wtf/AutodrainedPool.h: * wtf/Bag.h: (WTF::Bag::Bag): Deleted. (WTF::Bag::~Bag): Deleted. (WTF::Bag::clear): Deleted. (WTF::Bag::add): Deleted. (WTF::Bag::iterator::iterator): Deleted. (WTF::Bag::iterator::operator! const): Deleted. (WTF::Bag::iterator::operator* const): Deleted. (WTF::Bag::iterator::operator++): Deleted. (WTF::Bag::iterator::operator== const): Deleted. (WTF::Bag::iterator::operator!= const): Deleted. (WTF::Bag::begin): Deleted. (WTF::Bag::begin const): Deleted. (WTF::Bag::end const): Deleted. (WTF::Bag::isEmpty const): Deleted. (WTF::Bag::unwrappedHead const): Deleted. * wtf/BitVector.h: (WTF::BitVector::BitVector): Deleted. (WTF::BitVector::~BitVector): Deleted. (WTF::BitVector::operator=): Deleted. (WTF::BitVector::size const): Deleted. (WTF::BitVector::ensureSize): Deleted. (WTF::BitVector::quickGet const): Deleted. (WTF::BitVector::quickSet): Deleted. (WTF::BitVector::quickClear): Deleted. (WTF::BitVector::get const): Deleted. (WTF::BitVector::contains const): Deleted. (WTF::BitVector::set): Deleted. (WTF::BitVector::add): Deleted. (WTF::BitVector::ensureSizeAndSet): Deleted. (WTF::BitVector::clear): Deleted. (WTF::BitVector::remove): Deleted. (WTF::BitVector::merge): Deleted. (WTF::BitVector::filter): Deleted. (WTF::BitVector::exclude): Deleted. (WTF::BitVector::bitCount const): Deleted. (WTF::BitVector::isEmpty const): Deleted. (WTF::BitVector::findBit const): Deleted. (WTF::BitVector::isEmptyValue const): Deleted. (WTF::BitVector::isDeletedValue const): Deleted. (WTF::BitVector::isEmptyOrDeletedValue const): Deleted. (WTF::BitVector::operator== const): Deleted. (WTF::BitVector::hash const): Deleted. (WTF::BitVector::iterator::iterator): Deleted. (WTF::BitVector::iterator::operator* const): Deleted. (WTF::BitVector::iterator::operator++): Deleted. (WTF::BitVector::iterator::isAtEnd const): Deleted. (WTF::BitVector::iterator::operator== const): Deleted. (WTF::BitVector::iterator::operator!= const): Deleted. (WTF::BitVector::begin const): Deleted. (WTF::BitVector::end const): Deleted. (WTF::BitVector::bitsInPointer): Deleted. (WTF::BitVector::maxInlineBits): Deleted. (WTF::BitVector::byteCount): Deleted. (WTF::BitVector::makeInlineBits): Deleted. (WTF::BitVector::cleanseInlineBits): Deleted. (WTF::BitVector::bitCount): Deleted. (WTF::BitVector::findBitFast const): Deleted. (WTF::BitVector::findBitSimple const): Deleted. (WTF::BitVector::OutOfLineBits::numBits const): Deleted. (WTF::BitVector::OutOfLineBits::numWords const): Deleted. (WTF::BitVector::OutOfLineBits::bits): Deleted. (WTF::BitVector::OutOfLineBits::bits const): Deleted. (WTF::BitVector::OutOfLineBits::OutOfLineBits): Deleted. (WTF::BitVector::isInline const): Deleted. (WTF::BitVector::outOfLineBits const): Deleted. (WTF::BitVector::outOfLineBits): Deleted. (WTF::BitVector::bits): Deleted. (WTF::BitVector::bits const): Deleted. * wtf/Bitmap.h: (WTF::Bitmap::size): Deleted. (WTF::Bitmap::iterator::iterator): Deleted. (WTF::Bitmap::iterator::operator* const): Deleted. (WTF::Bitmap::iterator::operator++): Deleted. (WTF::Bitmap::iterator::operator== const): Deleted. (WTF::Bitmap::iterator::operator!= const): Deleted. (WTF::Bitmap::begin const): Deleted. (WTF::Bitmap::end const): Deleted. * wtf/Box.h: * wtf/BumpPointerAllocator.h: * wtf/CPUTime.h: * wtf/CheckedBoolean.h: * wtf/CommaPrinter.h: (WTF::CommaPrinter::CommaPrinter): Deleted. (WTF::CommaPrinter::dump const): Deleted. (WTF::CommaPrinter::didPrint const): Deleted. * wtf/CompactPointerTuple.h: (WTF::CompactPointerTuple::encodeType): Deleted. (WTF::CompactPointerTuple::decodeType): Deleted. (WTF::CompactPointerTuple::CompactPointerTuple): Deleted. (WTF::CompactPointerTuple::pointer const): Deleted. (WTF::CompactPointerTuple::setPointer): Deleted. (WTF::CompactPointerTuple::type const): Deleted. (WTF::CompactPointerTuple::setType): Deleted. * wtf/CompilationThread.h: (WTF::CompilationScope::CompilationScope): Deleted. (WTF::CompilationScope::~CompilationScope): Deleted. (WTF::CompilationScope::leaveEarly): Deleted. * wtf/CompletionHandler.h: (WTF::CompletionHandler<Out): (WTF::Detail::CallableWrapper<CompletionHandler<Out): (WTF::CompletionHandlerCallingScope::CompletionHandlerCallingScope): Deleted. (WTF::CompletionHandlerCallingScope::~CompletionHandlerCallingScope): Deleted. (WTF::CompletionHandlerCallingScope::CompletionHandler<void): Deleted. * wtf/ConcurrentBuffer.h: (WTF::ConcurrentBuffer::ConcurrentBuffer): Deleted. (WTF::ConcurrentBuffer::~ConcurrentBuffer): Deleted. (WTF::ConcurrentBuffer::growExact): Deleted. (WTF::ConcurrentBuffer::grow): Deleted. (WTF::ConcurrentBuffer::array const): Deleted. (WTF::ConcurrentBuffer::operator[]): Deleted. (WTF::ConcurrentBuffer::operator[] const): Deleted. (WTF::ConcurrentBuffer::createArray): Deleted. * wtf/ConcurrentPtrHashSet.h: (WTF::ConcurrentPtrHashSet::contains): Deleted. (WTF::ConcurrentPtrHashSet::add): Deleted. (WTF::ConcurrentPtrHashSet::size const): Deleted. (WTF::ConcurrentPtrHashSet::Table::maxLoad const): Deleted. (WTF::ConcurrentPtrHashSet::hash): Deleted. (WTF::ConcurrentPtrHashSet::cast): Deleted. (WTF::ConcurrentPtrHashSet::containsImpl const): Deleted. (WTF::ConcurrentPtrHashSet::addImpl): Deleted. * wtf/ConcurrentVector.h: (WTF::ConcurrentVector::~ConcurrentVector): Deleted. (WTF::ConcurrentVector::size const): Deleted. (WTF::ConcurrentVector::isEmpty const): Deleted. (WTF::ConcurrentVector::at): Deleted. (WTF::ConcurrentVector::at const): Deleted. (WTF::ConcurrentVector::operator[]): Deleted. (WTF::ConcurrentVector::operator[] const): Deleted. (WTF::ConcurrentVector::first): Deleted. (WTF::ConcurrentVector::first const): Deleted. (WTF::ConcurrentVector::last): Deleted. (WTF::ConcurrentVector::last const): Deleted. (WTF::ConcurrentVector::takeLast): Deleted. (WTF::ConcurrentVector::append): Deleted. (WTF::ConcurrentVector::alloc): Deleted. (WTF::ConcurrentVector::removeLast): Deleted. (WTF::ConcurrentVector::grow): Deleted. (WTF::ConcurrentVector::begin): Deleted. (WTF::ConcurrentVector::end): Deleted. (WTF::ConcurrentVector::segmentExistsFor): Deleted. (WTF::ConcurrentVector::segmentFor): Deleted. (WTF::ConcurrentVector::subscriptFor): Deleted. (WTF::ConcurrentVector::ensureSegmentsFor): Deleted. (WTF::ConcurrentVector::ensureSegment): Deleted. (WTF::ConcurrentVector::allocateSegment): Deleted. * wtf/Condition.h: (WTF::Condition::waitUntil): Deleted. (WTF::Condition::waitFor): Deleted. (WTF::Condition::wait): Deleted. (WTF::Condition::notifyOne): Deleted. (WTF::Condition::notifyAll): Deleted. * wtf/CountingLock.h: (WTF::CountingLock::LockHooks::lockHook): Deleted. (WTF::CountingLock::LockHooks::unlockHook): Deleted. (WTF::CountingLock::LockHooks::parkHook): Deleted. (WTF::CountingLock::LockHooks::handoffHook): Deleted. (WTF::CountingLock::tryLock): Deleted. (WTF::CountingLock::lock): Deleted. (WTF::CountingLock::unlock): Deleted. (WTF::CountingLock::isHeld const): Deleted. (WTF::CountingLock::isLocked const): Deleted. (WTF::CountingLock::Count::operator bool const): Deleted. (WTF::CountingLock::Count::operator== const): Deleted. (WTF::CountingLock::Count::operator!= const): Deleted. (WTF::CountingLock::tryOptimisticRead): Deleted. (WTF::CountingLock::validate): Deleted. (WTF::CountingLock::doOptimizedRead): Deleted. (WTF::CountingLock::tryOptimisticFencelessRead): Deleted. (WTF::CountingLock::fencelessValidate): Deleted. (WTF::CountingLock::doOptimizedFencelessRead): Deleted. (WTF::CountingLock::getCount): Deleted. * wtf/CrossThreadQueue.h: * wtf/CrossThreadTask.h: * wtf/CryptographicallyRandomNumber.cpp: * wtf/DataMutex.h: * wtf/DateMath.h: * wtf/Deque.h: (WTF::Deque::size const): Deleted. (WTF::Deque::isEmpty const): Deleted. (WTF::Deque::begin): Deleted. (WTF::Deque::end): Deleted. (WTF::Deque::begin const): Deleted. (WTF::Deque::end const): Deleted. (WTF::Deque::rbegin): Deleted. (WTF::Deque::rend): Deleted. (WTF::Deque::rbegin const): Deleted. (WTF::Deque::rend const): Deleted. (WTF::Deque::first): Deleted. (WTF::Deque::first const): Deleted. (WTF::Deque::last): Deleted. (WTF::Deque::last const): Deleted. (WTF::Deque::append): Deleted. * wtf/Dominators.h: * wtf/DoublyLinkedList.h: * wtf/Expected.h: * wtf/FastBitVector.h: * wtf/FileMetadata.h: * wtf/FileSystem.h: * wtf/GraphNodeWorklist.h: * wtf/GregorianDateTime.h: (WTF::GregorianDateTime::GregorianDateTime): Deleted. (WTF::GregorianDateTime::year const): Deleted. (WTF::GregorianDateTime::month const): Deleted. (WTF::GregorianDateTime::yearDay const): Deleted. (WTF::GregorianDateTime::monthDay const): Deleted. (WTF::GregorianDateTime::weekDay const): Deleted. (WTF::GregorianDateTime::hour const): Deleted. (WTF::GregorianDateTime::minute const): Deleted. (WTF::GregorianDateTime::second const): Deleted. (WTF::GregorianDateTime::utcOffset const): Deleted. (WTF::GregorianDateTime::isDST const): Deleted. (WTF::GregorianDateTime::setYear): Deleted. (WTF::GregorianDateTime::setMonth): Deleted. (WTF::GregorianDateTime::setYearDay): Deleted. (WTF::GregorianDateTime::setMonthDay): Deleted. (WTF::GregorianDateTime::setWeekDay): Deleted. (WTF::GregorianDateTime::setHour): Deleted. (WTF::GregorianDateTime::setMinute): Deleted. (WTF::GregorianDateTime::setSecond): Deleted. (WTF::GregorianDateTime::setUtcOffset): Deleted. (WTF::GregorianDateTime::setIsDST): Deleted. (WTF::GregorianDateTime::operator tm const): Deleted. (WTF::GregorianDateTime::copyFrom): Deleted. * wtf/HashTable.h: * wtf/Hasher.h: * wtf/HexNumber.h: * wtf/Indenter.h: * wtf/IndexMap.h: * wtf/IndexSet.h: * wtf/IndexSparseSet.h: * wtf/IndexedContainerIterator.h: * wtf/Insertion.h: * wtf/IteratorAdaptors.h: * wtf/IteratorRange.h: * wtf/KeyValuePair.h: * wtf/ListHashSet.h: (WTF::ListHashSet::begin): Deleted. (WTF::ListHashSet::end): Deleted. (WTF::ListHashSet::begin const): Deleted. (WTF::ListHashSet::end const): Deleted. (WTF::ListHashSet::random): Deleted. (WTF::ListHashSet::random const): Deleted. (WTF::ListHashSet::rbegin): Deleted. (WTF::ListHashSet::rend): Deleted. (WTF::ListHashSet::rbegin const): Deleted. (WTF::ListHashSet::rend const): Deleted. * wtf/Liveness.h: * wtf/LocklessBag.h: (WTF::LocklessBag::LocklessBag): Deleted. (WTF::LocklessBag::add): Deleted. (WTF::LocklessBag::iterate): Deleted. (WTF::LocklessBag::consumeAll): Deleted. (WTF::LocklessBag::consumeAllWithNode): Deleted. (WTF::LocklessBag::~LocklessBag): Deleted. * wtf/LoggingHashID.h: * wtf/MD5.h: * wtf/MachSendRight.h: * wtf/MainThreadData.h: * wtf/Markable.h: * wtf/MediaTime.h: * wtf/MemoryPressureHandler.h: * wtf/MessageQueue.h: (WTF::MessageQueue::MessageQueue): Deleted. * wtf/MetaAllocator.h: * wtf/MonotonicTime.h: (WTF::MonotonicTime::MonotonicTime): Deleted. (WTF::MonotonicTime::fromRawSeconds): Deleted. (WTF::MonotonicTime::infinity): Deleted. (WTF::MonotonicTime::nan): Deleted. (WTF::MonotonicTime::secondsSinceEpoch const): Deleted. (WTF::MonotonicTime::approximateMonotonicTime const): Deleted. (WTF::MonotonicTime::operator bool const): Deleted. (WTF::MonotonicTime::operator+ const): Deleted. (WTF::MonotonicTime::operator- const): Deleted. (WTF::MonotonicTime::operator% const): Deleted. (WTF::MonotonicTime::operator+=): Deleted. (WTF::MonotonicTime::operator-=): Deleted. (WTF::MonotonicTime::operator== const): Deleted. (WTF::MonotonicTime::operator!= const): Deleted. (WTF::MonotonicTime::operator< const): Deleted. (WTF::MonotonicTime::operator> const): Deleted. (WTF::MonotonicTime::operator<= const): Deleted. (WTF::MonotonicTime::operator>= const): Deleted. (WTF::MonotonicTime::isolatedCopy const): Deleted. (WTF::MonotonicTime::encode const): Deleted. (WTF::MonotonicTime::decode): Deleted. * wtf/NaturalLoops.h: * wtf/NoLock.h: * wtf/OSAllocator.h: * wtf/OptionSet.h: * wtf/Optional.h: * wtf/OrderMaker.h: * wtf/Packed.h: (WTF::alignof): * wtf/PackedIntVector.h: (WTF::PackedIntVector::PackedIntVector): Deleted. (WTF::PackedIntVector::operator=): Deleted. (WTF::PackedIntVector::size const): Deleted. (WTF::PackedIntVector::ensureSize): Deleted. (WTF::PackedIntVector::resize): Deleted. (WTF::PackedIntVector::clearAll): Deleted. (WTF::PackedIntVector::get const): Deleted. (WTF::PackedIntVector::set): Deleted. (WTF::PackedIntVector::mask): Deleted. * wtf/PageBlock.h: * wtf/ParallelJobsOpenMP.h: * wtf/ParkingLot.h: * wtf/PriorityQueue.h: (WTF::PriorityQueue::size const): Deleted. (WTF::PriorityQueue::isEmpty const): Deleted. (WTF::PriorityQueue::enqueue): Deleted. (WTF::PriorityQueue::peek const): Deleted. (WTF::PriorityQueue::dequeue): Deleted. (WTF::PriorityQueue::decreaseKey): Deleted. (WTF::PriorityQueue::increaseKey): Deleted. (WTF::PriorityQueue::begin const): Deleted. (WTF::PriorityQueue::end const): Deleted. (WTF::PriorityQueue::isValidHeap const): Deleted. (WTF::PriorityQueue::parentOf): Deleted. (WTF::PriorityQueue::leftChildOf): Deleted. (WTF::PriorityQueue::rightChildOf): Deleted. (WTF::PriorityQueue::siftUp): Deleted. (WTF::PriorityQueue::siftDown): Deleted. * wtf/RandomDevice.h: * wtf/Range.h: * wtf/RangeSet.h: (WTF::RangeSet::RangeSet): Deleted. (WTF::RangeSet::~RangeSet): Deleted. (WTF::RangeSet::add): Deleted. (WTF::RangeSet::contains const): Deleted. (WTF::RangeSet::overlaps const): Deleted. (WTF::RangeSet::clear): Deleted. (WTF::RangeSet::dump const): Deleted. (WTF::RangeSet::dumpRaw const): Deleted. (WTF::RangeSet::begin const): Deleted. (WTF::RangeSet::end const): Deleted. (WTF::RangeSet::addAll): Deleted. (WTF::RangeSet::compact): Deleted. (WTF::RangeSet::overlapsNonEmpty): Deleted. (WTF::RangeSet::subsumesNonEmpty): Deleted. (WTF::RangeSet::findRange const): Deleted. * wtf/RecursableLambda.h: * wtf/RedBlackTree.h: (WTF::RedBlackTree::Node::successor const): Deleted. (WTF::RedBlackTree::Node::predecessor const): Deleted. (WTF::RedBlackTree::Node::successor): Deleted. (WTF::RedBlackTree::Node::predecessor): Deleted. (WTF::RedBlackTree::Node::reset): Deleted. (WTF::RedBlackTree::Node::parent const): Deleted. (WTF::RedBlackTree::Node::setParent): Deleted. (WTF::RedBlackTree::Node::left const): Deleted. (WTF::RedBlackTree::Node::setLeft): Deleted. (WTF::RedBlackTree::Node::right const): Deleted. (WTF::RedBlackTree::Node::setRight): Deleted. (WTF::RedBlackTree::Node::color const): Deleted. (WTF::RedBlackTree::Node::setColor): Deleted. (WTF::RedBlackTree::RedBlackTree): Deleted. (WTF::RedBlackTree::insert): Deleted. (WTF::RedBlackTree::remove): Deleted. (WTF::RedBlackTree::findExact const): Deleted. (WTF::RedBlackTree::findLeastGreaterThanOrEqual const): Deleted. (WTF::RedBlackTree::findGreatestLessThanOrEqual const): Deleted. (WTF::RedBlackTree::first const): Deleted. (WTF::RedBlackTree::last const): Deleted. (WTF::RedBlackTree::size): Deleted. (WTF::RedBlackTree::isEmpty): Deleted. (WTF::RedBlackTree::treeMinimum): Deleted. (WTF::RedBlackTree::treeMaximum): Deleted. (WTF::RedBlackTree::treeInsert): Deleted. (WTF::RedBlackTree::leftRotate): Deleted. (WTF::RedBlackTree::rightRotate): Deleted. (WTF::RedBlackTree::removeFixup): Deleted. * wtf/ResourceUsage.h: * wtf/RunLoop.cpp: * wtf/RunLoopTimer.h: * wtf/SHA1.h: * wtf/Seconds.h: (WTF::Seconds::Seconds): Deleted. (WTF::Seconds::value const): Deleted. (WTF::Seconds::minutes const): Deleted. (WTF::Seconds::seconds const): Deleted. (WTF::Seconds::milliseconds const): Deleted. (WTF::Seconds::microseconds const): Deleted. (WTF::Seconds::nanoseconds const): Deleted. (WTF::Seconds::minutesAs const): Deleted. (WTF::Seconds::secondsAs const): Deleted. (WTF::Seconds::millisecondsAs const): Deleted. (WTF::Seconds::microsecondsAs const): Deleted. (WTF::Seconds::nanosecondsAs const): Deleted. (WTF::Seconds::fromMinutes): Deleted. (WTF::Seconds::fromHours): Deleted. (WTF::Seconds::fromMilliseconds): Deleted. (WTF::Seconds::fromMicroseconds): Deleted. (WTF::Seconds::fromNanoseconds): Deleted. (WTF::Seconds::infinity): Deleted. (WTF::Seconds::nan): Deleted. (WTF::Seconds::operator bool const): Deleted. (WTF::Seconds::operator+ const): Deleted. (WTF::Seconds::operator- const): Deleted. (WTF::Seconds::operator* const): Deleted. (WTF::Seconds::operator/ const): Deleted. (WTF::Seconds::operator% const): Deleted. (WTF::Seconds::operator+=): Deleted. (WTF::Seconds::operator-=): Deleted. (WTF::Seconds::operator*=): Deleted. (WTF::Seconds::operator/=): Deleted. (WTF::Seconds::operator%=): Deleted. (WTF::Seconds::operator== const): Deleted. (WTF::Seconds::operator!= const): Deleted. (WTF::Seconds::operator< const): Deleted. (WTF::Seconds::operator> const): Deleted. (WTF::Seconds::operator<= const): Deleted. (WTF::Seconds::operator>= const): Deleted. (WTF::Seconds::isolatedCopy const): Deleted. (WTF::Seconds::encode const): Deleted. (WTF::Seconds::decode): Deleted. * wtf/SegmentedVector.h: (WTF::SegmentedVector::~SegmentedVector): Deleted. (WTF::SegmentedVector::size const): Deleted. (WTF::SegmentedVector::isEmpty const): Deleted. (WTF::SegmentedVector::at): Deleted. (WTF::SegmentedVector::at const): Deleted. (WTF::SegmentedVector::operator[]): Deleted. (WTF::SegmentedVector::operator[] const): Deleted. (WTF::SegmentedVector::first): Deleted. (WTF::SegmentedVector::first const): Deleted. (WTF::SegmentedVector::last): Deleted. (WTF::SegmentedVector::last const): Deleted. (WTF::SegmentedVector::takeLast): Deleted. (WTF::SegmentedVector::append): Deleted. (WTF::SegmentedVector::alloc): Deleted. (WTF::SegmentedVector::removeLast): Deleted. (WTF::SegmentedVector::grow): Deleted. (WTF::SegmentedVector::clear): Deleted. (WTF::SegmentedVector::begin): Deleted. (WTF::SegmentedVector::end): Deleted. (WTF::SegmentedVector::shrinkToFit): Deleted. (WTF::SegmentedVector::deleteAllSegments): Deleted. (WTF::SegmentedVector::segmentExistsFor): Deleted. (WTF::SegmentedVector::segmentFor): Deleted. (WTF::SegmentedVector::subscriptFor): Deleted. (WTF::SegmentedVector::ensureSegmentsFor): Deleted. (WTF::SegmentedVector::ensureSegment): Deleted. (WTF::SegmentedVector::allocateSegment): Deleted. * wtf/SetForScope.h: * wtf/SingleRootGraph.h: * wtf/SinglyLinkedList.h: * wtf/SmallPtrSet.h: * wtf/SpanningTree.h: * wtf/Spectrum.h: * wtf/StackBounds.h: * wtf/StackShot.h: * wtf/StackShotProfiler.h: * wtf/StackStats.h: * wtf/StackTrace.h: * wtf/StreamBuffer.h: * wtf/SynchronizedFixedQueue.h: (WTF::SynchronizedFixedQueue::create): Deleted. (WTF::SynchronizedFixedQueue::open): Deleted. (WTF::SynchronizedFixedQueue::close): Deleted. (WTF::SynchronizedFixedQueue::isOpen): Deleted. (WTF::SynchronizedFixedQueue::enqueue): Deleted. (WTF::SynchronizedFixedQueue::dequeue): Deleted. (WTF::SynchronizedFixedQueue::SynchronizedFixedQueue): Deleted. * wtf/SystemTracing.h: * wtf/ThreadGroup.h: (WTF::ThreadGroup::create): Deleted. (WTF::ThreadGroup::threads const): Deleted. (WTF::ThreadGroup::getLock): Deleted. (WTF::ThreadGroup::weakFromThis): Deleted. * wtf/ThreadSpecific.h: * wtf/ThreadingPrimitives.h: (WTF::Mutex::impl): Deleted. * wtf/TimeWithDynamicClockType.h: (WTF::TimeWithDynamicClockType::TimeWithDynamicClockType): Deleted. (WTF::TimeWithDynamicClockType::fromRawSeconds): Deleted. (WTF::TimeWithDynamicClockType::secondsSinceEpoch const): Deleted. (WTF::TimeWithDynamicClockType::clockType const): Deleted. (WTF::TimeWithDynamicClockType::withSameClockAndRawSeconds const): Deleted. (WTF::TimeWithDynamicClockType::operator bool const): Deleted. (WTF::TimeWithDynamicClockType::operator+ const): Deleted. (WTF::TimeWithDynamicClockType::operator- const): Deleted. (WTF::TimeWithDynamicClockType::operator+=): Deleted. (WTF::TimeWithDynamicClockType::operator-=): Deleted. (WTF::TimeWithDynamicClockType::operator== const): Deleted. (WTF::TimeWithDynamicClockType::operator!= const): Deleted. * wtf/TimingScope.h: * wtf/TinyLRUCache.h: * wtf/TinyPtrSet.h: * wtf/URLParser.cpp: * wtf/URLParser.h: * wtf/Unexpected.h: * wtf/Variant.h: * wtf/WTFSemaphore.h: (WTF::Semaphore::Semaphore): Deleted. (WTF::Semaphore::signal): Deleted. (WTF::Semaphore::waitUntil): Deleted. (WTF::Semaphore::waitFor): Deleted. (WTF::Semaphore::wait): Deleted. * wtf/WallTime.h: (WTF::WallTime::WallTime): Deleted. (WTF::WallTime::fromRawSeconds): Deleted. (WTF::WallTime::infinity): Deleted. (WTF::WallTime::nan): Deleted. (WTF::WallTime::secondsSinceEpoch const): Deleted. (WTF::WallTime::approximateWallTime const): Deleted. (WTF::WallTime::operator bool const): Deleted. (WTF::WallTime::operator+ const): Deleted. (WTF::WallTime::operator- const): Deleted. (WTF::WallTime::operator+=): Deleted. (WTF::WallTime::operator-=): Deleted. (WTF::WallTime::operator== const): Deleted. (WTF::WallTime::operator!= const): Deleted. (WTF::WallTime::operator< const): Deleted. (WTF::WallTime::operator> const): Deleted. (WTF::WallTime::operator<= const): Deleted. (WTF::WallTime::operator>= const): Deleted. (WTF::WallTime::isolatedCopy const): Deleted. * wtf/WeakHashSet.h: (WTF::WeakHashSet::WeakHashSetConstIterator::WeakHashSetConstIterator): Deleted. (WTF::WeakHashSet::WeakHashSetConstIterator::get const): Deleted. (WTF::WeakHashSet::WeakHashSetConstIterator::operator* const): Deleted. (WTF::WeakHashSet::WeakHashSetConstIterator::operator-> const): Deleted. (WTF::WeakHashSet::WeakHashSetConstIterator::operator++): Deleted. (WTF::WeakHashSet::WeakHashSetConstIterator::skipEmptyBuckets): Deleted. (WTF::WeakHashSet::WeakHashSetConstIterator::operator== const): Deleted. (WTF::WeakHashSet::WeakHashSetConstIterator::operator!= const): Deleted. (WTF::WeakHashSet::WeakHashSet): Deleted. (WTF::WeakHashSet::begin const): Deleted. (WTF::WeakHashSet::end const): Deleted. (WTF::WeakHashSet::add): Deleted. (WTF::WeakHashSet::remove): Deleted. (WTF::WeakHashSet::contains const): Deleted. (WTF::WeakHashSet::capacity const): Deleted. (WTF::WeakHashSet::computesEmpty const): Deleted. (WTF::WeakHashSet::hasNullReferences const): Deleted. (WTF::WeakHashSet::computeSize const): Deleted. (WTF::WeakHashSet::checkConsistency const): Deleted. * wtf/WeakRandom.h: (WTF::WeakRandom::WeakRandom): Deleted. (WTF::WeakRandom::setSeed): Deleted. (WTF::WeakRandom::seed const): Deleted. (WTF::WeakRandom::get): Deleted. (WTF::WeakRandom::getUint32): Deleted. (WTF::WeakRandom::lowOffset): Deleted. (WTF::WeakRandom::highOffset): Deleted. (WTF::WeakRandom::nextState): Deleted. (WTF::WeakRandom::generate): Deleted. (WTF::WeakRandom::advance): Deleted. * wtf/WordLock.h: (WTF::WordLock::lock): Deleted. (WTF::WordLock::unlock): Deleted. (WTF::WordLock::isHeld const): Deleted. (WTF::WordLock::isLocked const): Deleted. (WTF::WordLock::isFullyReset const): Deleted. * wtf/generic/MainThreadGeneric.cpp: * wtf/glib/GMutexLocker.h: * wtf/linux/CurrentProcessMemoryStatus.h: * wtf/posix/ThreadingPOSIX.cpp: (WTF::Semaphore::Semaphore): Deleted. (WTF::Semaphore::~Semaphore): Deleted. (WTF::Semaphore::wait): Deleted. (WTF::Semaphore::post): Deleted. * wtf/text/ASCIILiteral.h: (WTF::ASCIILiteral::operator const char* const): Deleted. (WTF::ASCIILiteral::fromLiteralUnsafe): Deleted. (WTF::ASCIILiteral::null): Deleted. (WTF::ASCIILiteral::characters const): Deleted. (WTF::ASCIILiteral::ASCIILiteral): Deleted. * wtf/text/AtomString.h: (WTF::AtomString::operator=): Deleted. (WTF::AtomString::isHashTableDeletedValue const): Deleted. (WTF::AtomString::existingHash const): Deleted. (WTF::AtomString::operator const String& const): Deleted. (WTF::AtomString::string const): Deleted. (WTF::AtomString::impl const): Deleted. (WTF::AtomString::is8Bit const): Deleted. (WTF::AtomString::characters8 const): Deleted. (WTF::AtomString::characters16 const): Deleted. (WTF::AtomString::length const): Deleted. (WTF::AtomString::operator[] const): Deleted. (WTF::AtomString::contains const): Deleted. (WTF::AtomString::containsIgnoringASCIICase const): Deleted. (WTF::AtomString::find const): Deleted. (WTF::AtomString::findIgnoringASCIICase const): Deleted. (WTF::AtomString::startsWith const): Deleted. (WTF::AtomString::startsWithIgnoringASCIICase const): Deleted. (WTF::AtomString::endsWith const): Deleted. (WTF::AtomString::endsWithIgnoringASCIICase const): Deleted. (WTF::AtomString::toInt const): Deleted. (WTF::AtomString::toDouble const): Deleted. (WTF::AtomString::toFloat const): Deleted. (WTF::AtomString::percentage const): Deleted. (WTF::AtomString::isNull const): Deleted. (WTF::AtomString::isEmpty const): Deleted. (WTF::AtomString::operator NSString * const): Deleted. * wtf/text/AtomStringImpl.h: (WTF::AtomStringImpl::lookUp): Deleted. (WTF::AtomStringImpl::add): Deleted. (WTF::AtomStringImpl::addWithStringTableProvider): Deleted. * wtf/text/CString.h: (WTF::CStringBuffer::data): Deleted. (WTF::CStringBuffer::length const): Deleted. (WTF::CStringBuffer::CStringBuffer): Deleted. (WTF::CStringBuffer::mutableData): Deleted. (WTF::CString::CString): Deleted. (WTF::CString::data const): Deleted. (WTF::CString::length const): Deleted. (WTF::CString::isNull const): Deleted. (WTF::CString::buffer const): Deleted. (WTF::CString::isHashTableDeletedValue const): Deleted. * wtf/text/ExternalStringImpl.h: (WTF::ExternalStringImpl::freeExternalBuffer): Deleted. * wtf/text/LineBreakIteratorPoolICU.h: * wtf/text/NullTextBreakIterator.h: * wtf/text/OrdinalNumber.h: * wtf/text/StringBuffer.h: * wtf/text/StringBuilder.h: * wtf/text/StringConcatenateNumbers.h: * wtf/text/StringHasher.h: * wtf/text/StringImpl.h: * wtf/text/StringView.cpp: * wtf/text/StringView.h: (WTF::StringView::left const): Deleted. (WTF::StringView::right const): Deleted. (WTF::StringView::underlyingStringIsValid const): Deleted. (WTF::StringView::setUnderlyingString): Deleted. * wtf/text/SymbolImpl.h: (WTF::SymbolImpl::StaticSymbolImpl::StaticSymbolImpl): Deleted. (WTF::SymbolImpl::StaticSymbolImpl::operator SymbolImpl&): Deleted. (WTF::PrivateSymbolImpl::PrivateSymbolImpl): Deleted. (WTF::RegisteredSymbolImpl::symbolRegistry const): Deleted. (WTF::RegisteredSymbolImpl::clearSymbolRegistry): Deleted. (WTF::RegisteredSymbolImpl::RegisteredSymbolImpl): Deleted. * wtf/text/SymbolRegistry.h: * wtf/text/TextBreakIterator.h: * wtf/text/TextPosition.h: * wtf/text/TextStream.h: * wtf/text/WTFString.h: (WTF::String::swap): Deleted. (WTF::String::adopt): Deleted. (WTF::String::isNull const): Deleted. (WTF::String::isEmpty const): Deleted. (WTF::String::impl const): Deleted. (WTF::String::releaseImpl): Deleted. (WTF::String::length const): Deleted. (WTF::String::characters8 const): Deleted. (WTF::String::characters16 const): Deleted. (WTF::String::is8Bit const): Deleted. (WTF::String::sizeInBytes const): Deleted. (WTF::String::operator[] const): Deleted. (WTF::String::find const): Deleted. (WTF::String::findIgnoringASCIICase const): Deleted. (WTF::String::reverseFind const): Deleted. (WTF::String::contains const): Deleted. (WTF::String::containsIgnoringASCIICase const): Deleted. (WTF::String::startsWith const): Deleted. (WTF::String::startsWithIgnoringASCIICase const): Deleted. (WTF::String::hasInfixStartingAt const): Deleted. (WTF::String::endsWith const): Deleted. (WTF::String::endsWithIgnoringASCIICase const): Deleted. (WTF::String::hasInfixEndingAt const): Deleted. (WTF::String::append): Deleted. (WTF::String::left const): Deleted. (WTF::String::right const): Deleted. (WTF::String::createUninitialized): Deleted. (WTF::String::fromUTF8WithLatin1Fallback): Deleted. (WTF::String::isAllASCII const): Deleted. (WTF::String::isAllLatin1 const): Deleted. (WTF::String::isSpecialCharacter const): Deleted. (WTF::String::isHashTableDeletedValue const): Deleted. (WTF::String::hash const): Deleted. (WTF::String::existingHash const): Deleted. * wtf/text/cf/TextBreakIteratorCF.h: * wtf/text/icu/TextBreakIteratorICU.h: * wtf/text/icu/UTextProviderLatin1.h: * wtf/threads/BinarySemaphore.h: (WTF::BinarySemaphore::waitFor): Deleted. (WTF::BinarySemaphore::wait): Deleted. * wtf/unicode/Collator.h: * wtf/win/GDIObject.h: * wtf/win/PathWalker.h: * wtf/win/Win32Handle.h: Canonical link: https://commits.webkit.org/214396@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@248546 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2019-08-12 20:57:15 +00:00
class ConcurrentBuffer final {
It should be possible to flag a cell for unconditional finalization https://bugs.webkit.org/show_bug.cgi?id=180636 Reviewed by Saam Barati. Source/JavaScriptCore: UnconditionalFinalizers were annoying - you had to allocate them and you had to manage a global linked list - but they had some nice properties: - You only did the hardest work (creating the UnconditionalFinalizer) on first GC where you survived and needed it. -> Just needing it wasn't enough. -> Just surviving wasn't enough. The new API based on IsoSubspaces meant that just surviving was enough to cause unconditional finalizer logic to be invoked. I think that's not great. InferredType got around this by making InferredStructure a cell, but this was a gross hack. For one, it meant that InferredStructure would survive during the GC in which its finalizer obviated the need for its existence. It's not really an idiom I want us to repeat because it sounds like the sort of thing that turns out to be subtly broken. We really need to have a way of indicating when you have entered into the state that requires your unconditional finalizer to be invoked. Basically, we want to be able to track the set of objects that need unconditional finalizers. Only the subset of that set that overlaps with the set of marked objects needs to be accurate. The easiest way to do this is a hierarchy of bitvectors: one to say which MarkedBlocks have objects that have unconditional finalizers, and another level to say which atoms within a MarkedBlock have unconditional finalizers. This change introduces IsoCellSet, which couples itself to the MarkedAllocator of some IsoSubspace to allow maintaining a set of objects (well, cells - you could do this with auxiliaries) that belong to that IsoSubspace. It'll have undefined behavior if you try to add/remove/contains an object that isn't in that IsoSubspace. For objects in that subspace, you can add/remove/contains and forEachMarkedCell. The cost of each IsoCellSet is at worst about 0.8% increase in size to every object in the subspace that the set is attached to. So, it makes sense to have a handful per subspace max. This change only needs one per subspace, but you could imagine more if we do this for WeakReferenceHarvester. To absolutely minimize the possibility that this incurs costs, the add/remove/contains functions can be used from any thread so long as forEachMarkedCell isn't running. This means that InferredType only needs to add itself to the set during visitChildren. Thus, it needs to both survive and need it for the hardest work to take place. The work of adding does involve a gnarly load chain that ends in a CAS: load block handle from block, load index, load segment, load bitvector, load bit -> if not set, then CAS. That's five dependent loads! However, it's perfect for running in parallel since the only write operations are to widely dispersed cache lines that contain the bits underlying the set. The best part is how forEachMarkedCell works. That skips blocks that don't have any objects that need unconditional finalizers, and only touches the memory of marked objects that have the unconditional finalizer bit set. It will walk those objects in roughly address order. I previously found that this speeds up walking over a lot of objects when I made similar changes for DOM GC (calling visitAdditionalChildren via forEachMarkedCell rather than by walking a HashSet). This change makes InferredStructure be a malloc object again, but now it's in an IsoHeap. My expectation for this change is that it's perf-neutral. Long-term, it gives us a path forward for eliminating UnconditionalFinalizer and WeakReferenceHarvester while using IsoSubspace in more places. * JavaScriptCore.xcodeproj/project.pbxproj: * Sources.txt: * heap/AtomIndices.h: Added. (JSC::AtomIndices::AtomIndices): * heap/Heap.cpp: (JSC::Heap::finalizeUnconditionalFinalizers): * heap/Heap.h: * heap/IsoCellSet.cpp: Added. (JSC::IsoCellSet::IsoCellSet): (JSC::IsoCellSet::~IsoCellSet): (JSC::IsoCellSet::addSlow): (JSC::IsoCellSet::didResizeBits): (JSC::IsoCellSet::didRemoveBlock): (JSC::IsoCellSet::sweepToFreeList): * heap/IsoCellSet.h: Added. * heap/IsoCellSetInlines.h: Added. (JSC::IsoCellSet::add): (JSC::IsoCellSet::remove): (JSC::IsoCellSet::contains const): (JSC::IsoCellSet::forEachMarkedCell): * heap/IsoSubspace.cpp: (JSC::IsoSubspace::didResizeBits): (JSC::IsoSubspace::didRemoveBlock): (JSC::IsoSubspace::didBeginSweepingToFreeList): * heap/IsoSubspace.h: * heap/MarkedAllocator.cpp: (JSC::MarkedAllocator::addBlock): (JSC::MarkedAllocator::removeBlock): * heap/MarkedAllocator.h: * heap/MarkedAllocatorInlines.h: * heap/MarkedBlock.cpp: (JSC::MarkedBlock::Handle::sweep): (JSC::MarkedBlock::Handle::isEmpty): Deleted. * heap/MarkedBlock.h: (JSC::MarkedBlock::marks const): (JSC::MarkedBlock::Handle::newlyAllocated const): * heap/MarkedBlockInlines.h: (JSC::MarkedBlock::Handle::isAllocated): (JSC::MarkedBlock::Handle::isEmpty): (JSC::MarkedBlock::Handle::emptyMode): (JSC::MarkedBlock::Handle::forEachMarkedCell): * heap/Subspace.cpp: (JSC::Subspace::didResizeBits): (JSC::Subspace::didRemoveBlock): (JSC::Subspace::didBeginSweepingToFreeList): * heap/Subspace.h: * heap/SubspaceInlines.h: (JSC::Subspace::forEachMarkedCell): * runtime/InferredStructure.cpp: (JSC::InferredStructure::InferredStructure): (JSC::InferredStructure::create): Deleted. (JSC::InferredStructure::destroy): Deleted. (JSC::InferredStructure::createStructure): Deleted. (JSC::InferredStructure::visitChildren): Deleted. (JSC::InferredStructure::finalizeUnconditionally): Deleted. (JSC::InferredStructure::finishCreation): Deleted. * runtime/InferredStructure.h: * runtime/InferredStructureWatchpoint.cpp: (JSC::InferredStructureWatchpoint::fireInternal): * runtime/InferredType.cpp: (JSC::InferredType::visitChildren): (JSC::InferredType::willStoreValueSlow): (JSC::InferredType::makeTopSlow): (JSC::InferredType::set): (JSC::InferredType::removeStructure): (JSC::InferredType::finalizeUnconditionally): * runtime/InferredType.h: * runtime/VM.cpp: (JSC::VM::VM): * runtime/VM.h: Source/WTF: This adds ConcurrentVector, which is like SegmentedVector, but wastes some space to allow resizing to proceed concurrently to access. It's not possible to resize concurrently to resizing, concurrent read/writes aren't protected from racing if they access the same element, and who knows what you'll get if you iterate up to size() while someone else append()s. The key insight is to stash all prior copies of the spine, so that nobody crashes trying to access a stale spine. I'm going to want to do the same thing for FastBitVector, by creating a segmented WordOwner class. That would require repeating the dance of having a spine that can resize while stashing old versions. So, the spine resizing logic is abstracted behind ConcurrentBuffer. You could use that as a kind of "concurrent vector" for immutable data. That's how ConcurrentVector uses it: it's an immutable array of segment pointers. * WTF.xcodeproj/project.pbxproj: * wtf/ConcurrentBuffer.h: Added. (WTF::ConcurrentBuffer::ConcurrentBuffer): (WTF::ConcurrentBuffer::~ConcurrentBuffer): (WTF::ConcurrentBuffer::growExact): (WTF::ConcurrentBuffer::grow): (WTF::ConcurrentBuffer::array const): (WTF::ConcurrentBuffer::operator[]): (WTF::ConcurrentBuffer::operator[] const): (WTF::ConcurrentBuffer::createArray): * wtf/ConcurrentVector.h: Added. (WTF::ConcurrentVectorIterator::~ConcurrentVectorIterator): (WTF::ConcurrentVectorIterator::operator* const): (WTF::ConcurrentVectorIterator::operator-> const): (WTF::ConcurrentVectorIterator::operator++): (WTF::ConcurrentVectorIterator::operator== const): (WTF::ConcurrentVectorIterator::operator!= const): (WTF::ConcurrentVectorIterator::operator=): (WTF::ConcurrentVectorIterator::ConcurrentVectorIterator): (WTF::ConcurrentVector::~ConcurrentVector): (WTF::ConcurrentVector::size const): (WTF::ConcurrentVector::isEmpty const): (WTF::ConcurrentVector::at): (WTF::ConcurrentVector::at const): (WTF::ConcurrentVector::operator[]): (WTF::ConcurrentVector::operator[] const): (WTF::ConcurrentVector::first): (WTF::ConcurrentVector::first const): (WTF::ConcurrentVector::last): (WTF::ConcurrentVector::last const): (WTF::ConcurrentVector::takeLast): (WTF::ConcurrentVector::append): (WTF::ConcurrentVector::alloc): (WTF::ConcurrentVector::removeLast): (WTF::ConcurrentVector::grow): (WTF::ConcurrentVector::begin): (WTF::ConcurrentVector::end): (WTF::ConcurrentVector::segmentExistsFor): (WTF::ConcurrentVector::segmentFor): (WTF::ConcurrentVector::subscriptFor): (WTF::ConcurrentVector::ensureSegmentsFor): (WTF::ConcurrentVector::ensureSegment): (WTF::ConcurrentVector::allocateSegment): Canonical link: https://commits.webkit.org/196644@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@225831 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-12-13 02:35:54 +00:00
WTF_MAKE_NONCOPYABLE(ConcurrentBuffer);
WTF_MAKE_FAST_ALLOCATED;
public:
ConcurrentBuffer()
{
}
~ConcurrentBuffer()
{
if (Array* array = m_array) {
for (size_t i = 0; i < array->size; ++i)
array->data[i].~T();
}
for (Array* array : m_allArrays)
Experiment: create lots of different malloc zones for easier accounting of memory use https://bugs.webkit.org/show_bug.cgi?id=186422 Patch by Yusuke Suzuki <ysuzuki@apple.com> and Simon Fraser <simon.fraser@apple.com> on 2020-01-02 Reviewed by Saam Barati. Source/bmalloc: * bmalloc/BPlatform.h: * bmalloc/Environment.cpp: (bmalloc::Environment::computeIsDebugHeapEnabled): * bmalloc/IsoHeap.h: (bmalloc::api::IsoHeap::IsoHeap): * bmalloc/IsoHeapInlines.h: (bmalloc::api::IsoHeap<Type>::IsoHeap): * bmalloc/IsoTLSInlines.h: (bmalloc::IsoTLS::allocateSlow): (bmalloc::IsoTLS::deallocateSlow): Source/JavaScriptCore: * JavaScriptCore.xcodeproj/project.pbxproj: * Sources.txt: * assembler/AssemblerBuffer.cpp: Copied from Source/JavaScriptCore/bytecode/InstructionStream.cpp. * assembler/AssemblerBuffer.h: (JSC::AssemblerData::AssemblerData): (JSC::AssemblerData::operator=): (JSC::AssemblerData::~AssemblerData): (JSC::AssemblerData::grow): * bytecode/AccessCase.cpp: * bytecode/AccessCase.h: * bytecode/BytecodeBasicBlock.cpp: * bytecode/BytecodeBasicBlock.h: * bytecode/CodeBlock.cpp: * bytecode/CodeBlock.h: * bytecode/InstructionStream.cpp: * bytecode/InstructionStream.h: * bytecode/PolymorphicAccess.cpp: * bytecode/PolymorphicAccess.h: * bytecode/UnlinkedMetadataTable.cpp: (JSC::UnlinkedMetadataTable::finalize): * bytecode/UnlinkedMetadataTable.h: * bytecode/UnlinkedMetadataTableInlines.h: (JSC::UnlinkedMetadataTable::UnlinkedMetadataTable): (JSC::UnlinkedMetadataTable::~UnlinkedMetadataTable): (JSC::UnlinkedMetadataTable::link): (JSC::UnlinkedMetadataTable::unlink): * bytecode/ValueProfile.h: (JSC::ValueProfileAndVirtualRegisterBuffer::ValueProfileAndVirtualRegisterBuffer): * bytecode/Watchpoint.cpp: * bytecode/Watchpoint.h: * dfg/DFGBasicBlock.cpp: * dfg/DFGBasicBlock.h: * dfg/DFGNode.cpp: * dfg/DFGNode.h: * dfg/DFGSpeculativeJIT.cpp: * dfg/DFGSpeculativeJIT.h: * heap/BlockDirectory.cpp: * heap/BlockDirectory.h: * heap/FastMallocAlignedMemoryAllocator.cpp: (JSC::FastMallocAlignedMemoryAllocator::FastMallocAlignedMemoryAllocator): (JSC::FastMallocAlignedMemoryAllocator::tryAllocateAlignedMemory): (JSC::FastMallocAlignedMemoryAllocator::freeAlignedMemory): (JSC::FastMallocAlignedMemoryAllocator::tryAllocateMemory): (JSC::FastMallocAlignedMemoryAllocator::freeMemory): (JSC::FastMallocAlignedMemoryAllocator::tryReallocateMemory): * heap/FastMallocAlignedMemoryAllocator.h: * heap/GCSegmentedArray.cpp: Copied from Source/JavaScriptCore/parser/SourceProviderCache.cpp. * heap/GCSegmentedArray.h: * heap/GCSegmentedArrayInlines.h: (JSC::GCArraySegment<T>::create): (JSC::GCArraySegment<T>::destroy): * heap/GigacageAlignedMemoryAllocator.cpp: (JSC::GigacageAlignedMemoryAllocator::GigacageAlignedMemoryAllocator): (JSC::GigacageAlignedMemoryAllocator::tryAllocateAlignedMemory): (JSC::GigacageAlignedMemoryAllocator::freeAlignedMemory): (JSC::GigacageAlignedMemoryAllocator::tryAllocateMemory): (JSC::GigacageAlignedMemoryAllocator::freeMemory): (JSC::GigacageAlignedMemoryAllocator::tryReallocateMemory): * heap/GigacageAlignedMemoryAllocator.h: * heap/IsoAlignedMemoryAllocator.cpp: (JSC::IsoAlignedMemoryAllocator::IsoAlignedMemoryAllocator): (JSC::IsoAlignedMemoryAllocator::~IsoAlignedMemoryAllocator): (JSC::IsoAlignedMemoryAllocator::tryAllocateAlignedMemory): (JSC::IsoAlignedMemoryAllocator::freeAlignedMemory): (JSC::IsoAlignedMemoryAllocator::tryAllocateMemory): (JSC::IsoAlignedMemoryAllocator::freeMemory): * heap/IsoAlignedMemoryAllocator.h: * heap/IsoSubspace.cpp: (JSC::IsoSubspace::IsoSubspace): * heap/MarkedBlock.cpp: * heap/MarkedBlock.h: * heap/WeakBlock.cpp: (JSC::WeakBlock::create): (JSC::WeakBlock::destroy): * heap/WeakBlock.h: * jit/JITCode.cpp: * jit/JITCode.h: * jit/RegisterAtOffsetList.cpp: * jit/RegisterAtOffsetList.h: * parser/Nodes.cpp: * parser/Nodes.h: * parser/ParserArena.cpp: (JSC::ParserArena::deallocateObjects): (JSC::ParserArena::allocateFreeablePool): * parser/ParserArena.h: * parser/SourceProvider.cpp: * parser/SourceProvider.h: * parser/SourceProviderCache.cpp: * parser/SourceProviderCache.h: * parser/SourceProviderCacheItem.h: (JSC::SourceProviderCacheItem::create): * runtime/CachePayload.cpp: (JSC::CachePayload::makeMallocPayload): * runtime/CachePayload.h: * runtime/CachedBytecode.h: (JSC::CachedBytecode::create): * runtime/CachedTypes.cpp: (JSC::Encoder::release): (JSC::Encoder::Page::Page): (JSC::CachedVector::encode): (JSC::CachedVector::decode const): (JSC::CachedInstructionStream::decode const): * runtime/PropertyMapHashTable.h: (JSC::PropertyTable::rehash): * runtime/PropertyTable.cpp: (JSC::PropertyTable::PropertyTable): (JSC::PropertyTable::~PropertyTable): * runtime/SymbolTable.cpp: * runtime/SymbolTable.h: * runtime/VM.cpp: (JSC::VM::~VM): * runtime/VM.h: (JSC::ScratchBuffer::create): (JSC::VM::exceptionFuzzingBuffer): * wasm/WasmInstance.cpp: (JSC::Wasm::Instance::Instance): * wasm/WasmInstance.h: * wasm/WasmTable.cpp: (JSC::Wasm::Table::Table): (JSC::Wasm::FuncRefTable::FuncRefTable): * wasm/WasmTable.h: Source/WebCore: * Sources.txt: * WebCore.xcodeproj/project.pbxproj: * bindings/js/SerializedScriptValue.cpp: * bindings/js/SerializedScriptValue.h: * css/CSSFontFace.cpp: * css/CSSFontFace.h: * css/CSSSelector.cpp: * css/CSSSelector.h: * css/CSSValue.cpp: * css/CSSValue.h: * css/StyleProperties.cpp: (WebCore::ImmutableStyleProperties::create): * css/StyleProperties.h: * css/StyleRule.cpp: * css/StyleRule.h: * dom/ElementData.cpp: (WebCore::ShareableElementData::createWithAttributes): (WebCore::UniqueElementData::makeShareableCopy const): * dom/ElementData.h: * dom/NodeRareData.cpp: * dom/NodeRareData.h: * dom/QualifiedName.cpp: * dom/QualifiedName.h: * html/parser/HTMLDocumentParser.cpp: * html/parser/HTMLDocumentParser.h: * loader/DocumentLoader.cpp: * loader/DocumentLoader.h: * loader/ResourceLoader.cpp: * loader/ResourceLoader.h: * loader/cache/CachedResource.cpp: * loader/cache/CachedResource.h: * page/PerformanceEntry.cpp: * page/PerformanceEntry.h: * platform/graphics/Font.cpp: * platform/graphics/Font.h: * platform/graphics/FontCascadeFonts.cpp: * platform/graphics/FontCascadeFonts.h: * platform/graphics/Region.cpp: * platform/graphics/Region.h: * platform/graphics/avfoundation/objc/MediaSampleAVFObjC.mm: (WebCore::releaseUint8Vector): * platform/graphics/cg/ImageBufferCG.cpp: (WebCore::ImageBuffer::ImageBuffer): * platform/graphics/nicosia/NicosiaBuffer.cpp: (Nicosia::Buffer::Buffer): * platform/network/ResourceHandle.cpp: * platform/network/ResourceHandleInternal.h: * platform/network/cf/FormDataStreamCFNet.cpp: (WebCore::closeCurrentStream): (WebCore::advanceCurrentStream): * rendering/RenderLayer.cpp: * rendering/RenderLayer.h: * rendering/TableLayout.cpp: Copied from Source/JavaScriptCore/parser/SourceProviderCache.cpp. * rendering/TableLayout.h: * rendering/style/RenderStyle.cpp: * rendering/style/RenderStyle.h: * rendering/style/SVGRenderStyle.cpp: * rendering/style/SVGRenderStyle.h: * rendering/style/SVGRenderStyleDefs.cpp: * rendering/style/SVGRenderStyleDefs.h: * rendering/style/StyleBoxData.cpp: * rendering/style/StyleBoxData.h: * rendering/style/StyleInheritedData.cpp: * rendering/style/StyleInheritedData.h: * rendering/style/StyleRareInheritedData.cpp: * rendering/style/StyleRareInheritedData.h: * rendering/style/StyleRareNonInheritedData.cpp: * rendering/style/StyleRareNonInheritedData.h: * rendering/style/StyleSurroundData.cpp: * rendering/style/StyleSurroundData.h: * rendering/style/StyleTransformData.cpp: * rendering/style/StyleTransformData.h: * style/StyleTreeResolver.cpp: * style/StyleTreeResolver.h: * svg/animation/SMILTimeContainer.cpp: * svg/animation/SMILTimeContainer.h: Source/WebKit: * Shared/ShareableBitmap.cpp: (WebKit::ShareableBitmap::create): (WebKit::ShareableBitmap::~ShareableBitmap): * UIProcess/mac/LegacySessionStateCoding.cpp: (WebKit::HistoryEntryDataEncoder::HistoryEntryDataEncoder): (WebKit::HistoryEntryDataEncoder::finishEncoding): (WebKit::encodeSessionHistoryEntryData): (WebKit::encodeLegacySessionState): Source/WTF: This patch introduces ENABLE(MALLOC_HEAP_BREAKDOWN). If this is enabled, we allocate malloc_zone per malloc kind. This offers the way to investigate the usage of memory per kind by using vmmap, like the following. VIRTUAL RESIDENT DIRTY SWAPPED ALLOCATION BYTES DIRTY+SWAP REGION MALLOC ZONE SIZE SIZE SIZE SIZE COUNT ALLOCATED FRAG SIZE % FRAG COUNT =========== ======= ========= ========= ========= ========= ========= ========= ====== ====== StringImpl_0x116efd000 188.0M 69.3M 30.9M 0K 139456 18.0M 12.9M 42% 34 DefaultMallocZone_0x10f487000 176.0M 53.9M 14.1M 0K 115956 9955K 4497K 32% 22 Vector_0x116eff000 162.0M 56.3M 55.3M 0K 140715 17.3M 37.9M 69% 36 MetadataTable_0x11843b000 152.0M 17.5M 17.5M 0K 14200 2353K 15.2M 87% 26 WebKit Using System Malloc_0x114cbe000 150.0M 31.6M 21.8M 0K 87422 16.7M 5278K 24% 23 InstructionStream_0x118469000 150.0M 5764K 5764K 0K 14470 4688K 1076K 19% 24 AssemblerData_0x117ee6000 150.0M 1928K 1928K 0K 1 16 1928K 100% 24 To achieve this goal without making very large change, we put a template type in various containers. For example, Vector will take Malloc parameter (the default one is FastMalloc allocator). If ENABLE(MALLOC_HEAP_BREAKDOWN) is enabled, we change this to specific VectorMalloc allocator, and vmmap can show memory usage of this allocator. This patch also supports malloc_zone per IsoHeap. So we can see memory allocation per IsoHeap in vmmap. To use this feature, we need to flip two compile time flags, ENABLE(MALLOC_HEAP_BREAKDOWN) in WTF and BENABLE_MALLOC_HEAP_BREAKDOWN in bmalloc. And use `vmmap $PID` to dump malloc zones. To allocate objects of a class with a specific malloc-zone, use WTF_MAKE_FAST_ALLOCATED_WITH_HEAP_IDENTIFIER(HeapIdentifier) for the class, and define allocator by DECLARE_ALLOCATOR_WITH_HEAP_IDENTIFIER(HeapIdentifier) in a header and DEFINE_ALLOCATOR_WITH_HEAP_IDENTIFIER(HeapIdentifier) in a cpp file. This patch also introduce callstack collector for malloc. Vector, HashMap etc. are used to allocate various things, but the above malloc_zone feature only tells thing like "Vector takes XXX MB memory". But what we want to know in this case is what Vector is consuming memory. We collect StackShot for each malloc call, and combine these information to tell which callsite is consuming much memory, which tell us that what Vector is consuming memory. * WTF.xcodeproj/project.pbxproj: * wtf/Bag.cpp: Copied from Source/JavaScriptCore/parser/SourceProviderCache.cpp. * wtf/Bag.h: (WTF::Private::BagNode::BagNode): Deleted. * wtf/BitVector.cpp: (WTF::BitVector::OutOfLineBits::create): (WTF::BitVector::OutOfLineBits::destroy): * wtf/CMakeLists.txt: * wtf/ConcurrentBuffer.cpp: Copied from Source/JavaScriptCore/parser/SourceProviderCache.cpp. * wtf/ConcurrentBuffer.h: * wtf/DebugHeap.cpp: Copied from Source/JavaScriptCore/runtime/CachePayload.cpp. (WTF::DebugHeap::DebugHeap): (WTF::DebugHeap::malloc): (WTF::DebugHeap::calloc): (WTF::DebugHeap::memalign): (WTF::DebugHeap::realloc): (WTF::DebugHeap::free): * wtf/DebugHeap.h: Added. * wtf/FastBitVector.cpp: (WTF::FastBitVectorWordOwner::setEqualsSlow): (WTF::FastBitVectorWordOwner::resizeSlow): * wtf/FastBitVector.h: (WTF::FastBitVectorWordOwner::~FastBitVectorWordOwner): * wtf/FastMalloc.cpp: (WTF::fastMallocDumpMallocStats): (WTF::AvoidRecordingScope::AvoidRecordingScope): (WTF::AvoidRecordingScope::~AvoidRecordingScope): (WTF::MallocCallTracker::MallocSiteData::MallocSiteData): (WTF::MallocCallTracker::singleton): (WTF::MallocCallTracker::MallocCallTracker): (WTF::MallocCallTracker::recordMalloc): (WTF::MallocCallTracker::recordRealloc): (WTF::MallocCallTracker::recordFree): (WTF::MallocCallTracker::dumpStats): (WTF::fastMalloc): (WTF::fastRealloc): (WTF::fastFree): (WTF::fastAlignedMalloc): (WTF::tryFastAlignedMalloc): (WTF::fastAlignedFree): * wtf/FastMalloc.h: (WTF::FastMalloc::zeroedMalloc): (WTF::FastMalloc::tryZeroedMalloc): * wtf/Forward.h: * wtf/HashTable.cpp: * wtf/HashTable.h: (WTF::KeyTraits>::allocateTable): (WTF::KeyTraits>::deallocateTable): (WTF::KeyTraits>::rehash): * wtf/MallocPtr.h: (WTF::MallocPtr::MallocPtr): (WTF::MallocPtr::malloc): (WTF::MallocPtr::zeroedMalloc): (WTF::MallocPtr::tryMalloc): (WTF::MallocPtr::tryZeroedMalloc): (WTF::adoptMallocPtr): * wtf/MetaAllocator.cpp: (WTF::MetaAllocator::allocFreeSpaceNode): (WTF::MetaAllocator::freeFreeSpaceNode): * wtf/MetaAllocatorHandle.h: * wtf/Platform.h: * wtf/RefCountedArray.cpp: Copied from Source/JavaScriptCore/bytecode/InstructionStream.cpp. * wtf/RefCountedArray.h: (WTF::RefCountedArray::RefCountedArray): (WTF::RefCountedArray::~RefCountedArray): (WTF::RefCountedArray::assign): * wtf/SegmentedVector.cpp: Copied from Source/JavaScriptCore/bytecode/InstructionStream.cpp. * wtf/SegmentedVector.h: * wtf/SmallPtrSet.cpp: Copied from Source/JavaScriptCore/bytecode/InstructionStream.cpp. * wtf/SmallPtrSet.h: (WTF::SmallPtrSet::~SmallPtrSet): (WTF::SmallPtrSet::grow): * wtf/UniqueArray.cpp: Copied from Source/JavaScriptCore/bytecode/InstructionStream.cpp. * wtf/UniqueArray.h: (WTF::UniqueArrayFree::operator() const): (WTF::UniqueArrayFree<T::operator() const): * wtf/Vector.cpp: Copied from Source/JavaScriptCore/bytecode/InstructionStream.cpp. * wtf/Vector.h: (WTF::VectorBufferBase::allocateBuffer): (WTF::VectorBufferBase::tryAllocateBuffer): (WTF::VectorBufferBase::reallocateBuffer): (WTF::VectorBufferBase::deallocateBuffer): (WTF::VectorBufferBase::releaseBuffer): (WTF::VectorBuffer::releaseBuffer): (WTF::Vector::swap): (WTF::Malloc>::Vector): (WTF::=): (WTF::Malloc>::contains const): (WTF::Malloc>::findMatching const): (WTF::Malloc>::find const): (WTF::Malloc>::reverseFind const): (WTF::Malloc>::appendIfNotContains): (WTF::Malloc>::fill): (WTF::Malloc>::appendRange): (WTF::Malloc>::expandCapacity): (WTF::Malloc>::tryExpandCapacity): (WTF::Malloc>::resize): (WTF::Malloc>::resizeToFit): (WTF::Malloc>::shrink): (WTF::Malloc>::grow): (WTF::Malloc>::asanSetInitialBufferSizeTo): (WTF::Malloc>::asanSetBufferSizeToFullCapacity): (WTF::Malloc>::asanBufferSizeWillChangeTo): (WTF::Malloc>::reserveCapacity): (WTF::Malloc>::tryReserveCapacity): (WTF::Malloc>::reserveInitialCapacity): (WTF::Malloc>::shrinkCapacity): (WTF::Malloc>::append): (WTF::Malloc>::tryAppend): (WTF::Malloc>::constructAndAppend): (WTF::Malloc>::tryConstructAndAppend): (WTF::Malloc>::appendSlowCase): (WTF::Malloc>::constructAndAppendSlowCase): (WTF::Malloc>::tryConstructAndAppendSlowCase): (WTF::Malloc>::uncheckedAppend): (WTF::Malloc>::uncheckedConstructAndAppend): (WTF::Malloc>::appendVector): (WTF::Malloc>::insert): (WTF::Malloc>::insertVector): (WTF::Malloc>::remove): (WTF::Malloc>::removeFirst): (WTF::Malloc>::removeFirstMatching): (WTF::Malloc>::removeAll): (WTF::Malloc>::removeAllMatching): (WTF::Malloc>::reverse): (WTF::Malloc>::map const): (WTF::Malloc>::releaseBuffer): (WTF::Malloc>::checkConsistency): (WTF::swap): (WTF::operator==): (WTF::operator!=): (WTF::Malloc>::isolatedCopy const): (WTF::removeRepeatedElements): (WTF::minCapacity>::Vector): Deleted. (WTF::minCapacity>::contains const): Deleted. (WTF::minCapacity>::findMatching const): Deleted. (WTF::minCapacity>::find const): Deleted. (WTF::minCapacity>::reverseFind const): Deleted. (WTF::minCapacity>::appendIfNotContains): Deleted. (WTF::minCapacity>::fill): Deleted. (WTF::minCapacity>::appendRange): Deleted. (WTF::minCapacity>::expandCapacity): Deleted. (WTF::minCapacity>::tryExpandCapacity): Deleted. (WTF::minCapacity>::resize): Deleted. (WTF::minCapacity>::resizeToFit): Deleted. (WTF::minCapacity>::shrink): Deleted. (WTF::minCapacity>::grow): Deleted. (WTF::minCapacity>::asanSetInitialBufferSizeTo): Deleted. (WTF::minCapacity>::asanSetBufferSizeToFullCapacity): Deleted. (WTF::minCapacity>::asanBufferSizeWillChangeTo): Deleted. (WTF::minCapacity>::reserveCapacity): Deleted. (WTF::minCapacity>::tryReserveCapacity): Deleted. (WTF::minCapacity>::reserveInitialCapacity): Deleted. (WTF::minCapacity>::shrinkCapacity): Deleted. (WTF::minCapacity>::append): Deleted. (WTF::minCapacity>::tryAppend): Deleted. (WTF::minCapacity>::constructAndAppend): Deleted. (WTF::minCapacity>::tryConstructAndAppend): Deleted. (WTF::minCapacity>::appendSlowCase): Deleted. (WTF::minCapacity>::constructAndAppendSlowCase): Deleted. (WTF::minCapacity>::tryConstructAndAppendSlowCase): Deleted. (WTF::minCapacity>::uncheckedAppend): Deleted. (WTF::minCapacity>::uncheckedConstructAndAppend): Deleted. (WTF::minCapacity>::appendVector): Deleted. (WTF::minCapacity>::insert): Deleted. (WTF::minCapacity>::insertVector): Deleted. (WTF::minCapacity>::remove): Deleted. (WTF::minCapacity>::removeFirst): Deleted. (WTF::minCapacity>::removeFirstMatching): Deleted. (WTF::minCapacity>::removeAll): Deleted. (WTF::minCapacity>::removeAllMatching): Deleted. (WTF::minCapacity>::reverse): Deleted. (WTF::minCapacity>::map const): Deleted. (WTF::minCapacity>::releaseBuffer): Deleted. (WTF::minCapacity>::checkConsistency): Deleted. (WTF::minCapacity>::isolatedCopy const): Deleted. * wtf/text/CString.cpp: (WTF::CStringBuffer::createUninitialized): * wtf/text/CString.h: * wtf/text/StringBuffer.cpp: Copied from Source/JavaScriptCore/bytecode/InstructionStream.cpp. * wtf/text/StringBuffer.h: (WTF::StringBuffer::StringBuffer): (WTF::StringBuffer::~StringBuffer): (WTF::StringBuffer::resize): (WTF::StringBuffer::release): * wtf/text/StringImpl.cpp: (WTF::StringImpl::~StringImpl): (WTF::StringImpl::destroy): (WTF::StringImpl::createUninitializedInternalNonEmpty): (WTF::StringImpl::reallocateInternal): * wtf/text/StringImpl.h: (WTF::StringImpl::StringImpl): (WTF::StringImpl::createSubstringSharingImpl): (WTF::StringImpl::tryCreateUninitialized): (WTF::StringImpl::adopt): * wtf/text/cf/StringImplCF.cpp: (WTF::StringWrapperCFAllocator::allocate): (WTF::StringWrapperCFAllocator::reallocate): (WTF::StringWrapperCFAllocator::deallocate): Canonical link: https://commits.webkit.org/218863@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@253987 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2020-01-03 02:36:43 +00:00
ConcurrentBufferMalloc::free(array);
It should be possible to flag a cell for unconditional finalization https://bugs.webkit.org/show_bug.cgi?id=180636 Reviewed by Saam Barati. Source/JavaScriptCore: UnconditionalFinalizers were annoying - you had to allocate them and you had to manage a global linked list - but they had some nice properties: - You only did the hardest work (creating the UnconditionalFinalizer) on first GC where you survived and needed it. -> Just needing it wasn't enough. -> Just surviving wasn't enough. The new API based on IsoSubspaces meant that just surviving was enough to cause unconditional finalizer logic to be invoked. I think that's not great. InferredType got around this by making InferredStructure a cell, but this was a gross hack. For one, it meant that InferredStructure would survive during the GC in which its finalizer obviated the need for its existence. It's not really an idiom I want us to repeat because it sounds like the sort of thing that turns out to be subtly broken. We really need to have a way of indicating when you have entered into the state that requires your unconditional finalizer to be invoked. Basically, we want to be able to track the set of objects that need unconditional finalizers. Only the subset of that set that overlaps with the set of marked objects needs to be accurate. The easiest way to do this is a hierarchy of bitvectors: one to say which MarkedBlocks have objects that have unconditional finalizers, and another level to say which atoms within a MarkedBlock have unconditional finalizers. This change introduces IsoCellSet, which couples itself to the MarkedAllocator of some IsoSubspace to allow maintaining a set of objects (well, cells - you could do this with auxiliaries) that belong to that IsoSubspace. It'll have undefined behavior if you try to add/remove/contains an object that isn't in that IsoSubspace. For objects in that subspace, you can add/remove/contains and forEachMarkedCell. The cost of each IsoCellSet is at worst about 0.8% increase in size to every object in the subspace that the set is attached to. So, it makes sense to have a handful per subspace max. This change only needs one per subspace, but you could imagine more if we do this for WeakReferenceHarvester. To absolutely minimize the possibility that this incurs costs, the add/remove/contains functions can be used from any thread so long as forEachMarkedCell isn't running. This means that InferredType only needs to add itself to the set during visitChildren. Thus, it needs to both survive and need it for the hardest work to take place. The work of adding does involve a gnarly load chain that ends in a CAS: load block handle from block, load index, load segment, load bitvector, load bit -> if not set, then CAS. That's five dependent loads! However, it's perfect for running in parallel since the only write operations are to widely dispersed cache lines that contain the bits underlying the set. The best part is how forEachMarkedCell works. That skips blocks that don't have any objects that need unconditional finalizers, and only touches the memory of marked objects that have the unconditional finalizer bit set. It will walk those objects in roughly address order. I previously found that this speeds up walking over a lot of objects when I made similar changes for DOM GC (calling visitAdditionalChildren via forEachMarkedCell rather than by walking a HashSet). This change makes InferredStructure be a malloc object again, but now it's in an IsoHeap. My expectation for this change is that it's perf-neutral. Long-term, it gives us a path forward for eliminating UnconditionalFinalizer and WeakReferenceHarvester while using IsoSubspace in more places. * JavaScriptCore.xcodeproj/project.pbxproj: * Sources.txt: * heap/AtomIndices.h: Added. (JSC::AtomIndices::AtomIndices): * heap/Heap.cpp: (JSC::Heap::finalizeUnconditionalFinalizers): * heap/Heap.h: * heap/IsoCellSet.cpp: Added. (JSC::IsoCellSet::IsoCellSet): (JSC::IsoCellSet::~IsoCellSet): (JSC::IsoCellSet::addSlow): (JSC::IsoCellSet::didResizeBits): (JSC::IsoCellSet::didRemoveBlock): (JSC::IsoCellSet::sweepToFreeList): * heap/IsoCellSet.h: Added. * heap/IsoCellSetInlines.h: Added. (JSC::IsoCellSet::add): (JSC::IsoCellSet::remove): (JSC::IsoCellSet::contains const): (JSC::IsoCellSet::forEachMarkedCell): * heap/IsoSubspace.cpp: (JSC::IsoSubspace::didResizeBits): (JSC::IsoSubspace::didRemoveBlock): (JSC::IsoSubspace::didBeginSweepingToFreeList): * heap/IsoSubspace.h: * heap/MarkedAllocator.cpp: (JSC::MarkedAllocator::addBlock): (JSC::MarkedAllocator::removeBlock): * heap/MarkedAllocator.h: * heap/MarkedAllocatorInlines.h: * heap/MarkedBlock.cpp: (JSC::MarkedBlock::Handle::sweep): (JSC::MarkedBlock::Handle::isEmpty): Deleted. * heap/MarkedBlock.h: (JSC::MarkedBlock::marks const): (JSC::MarkedBlock::Handle::newlyAllocated const): * heap/MarkedBlockInlines.h: (JSC::MarkedBlock::Handle::isAllocated): (JSC::MarkedBlock::Handle::isEmpty): (JSC::MarkedBlock::Handle::emptyMode): (JSC::MarkedBlock::Handle::forEachMarkedCell): * heap/Subspace.cpp: (JSC::Subspace::didResizeBits): (JSC::Subspace::didRemoveBlock): (JSC::Subspace::didBeginSweepingToFreeList): * heap/Subspace.h: * heap/SubspaceInlines.h: (JSC::Subspace::forEachMarkedCell): * runtime/InferredStructure.cpp: (JSC::InferredStructure::InferredStructure): (JSC::InferredStructure::create): Deleted. (JSC::InferredStructure::destroy): Deleted. (JSC::InferredStructure::createStructure): Deleted. (JSC::InferredStructure::visitChildren): Deleted. (JSC::InferredStructure::finalizeUnconditionally): Deleted. (JSC::InferredStructure::finishCreation): Deleted. * runtime/InferredStructure.h: * runtime/InferredStructureWatchpoint.cpp: (JSC::InferredStructureWatchpoint::fireInternal): * runtime/InferredType.cpp: (JSC::InferredType::visitChildren): (JSC::InferredType::willStoreValueSlow): (JSC::InferredType::makeTopSlow): (JSC::InferredType::set): (JSC::InferredType::removeStructure): (JSC::InferredType::finalizeUnconditionally): * runtime/InferredType.h: * runtime/VM.cpp: (JSC::VM::VM): * runtime/VM.h: Source/WTF: This adds ConcurrentVector, which is like SegmentedVector, but wastes some space to allow resizing to proceed concurrently to access. It's not possible to resize concurrently to resizing, concurrent read/writes aren't protected from racing if they access the same element, and who knows what you'll get if you iterate up to size() while someone else append()s. The key insight is to stash all prior copies of the spine, so that nobody crashes trying to access a stale spine. I'm going to want to do the same thing for FastBitVector, by creating a segmented WordOwner class. That would require repeating the dance of having a spine that can resize while stashing old versions. So, the spine resizing logic is abstracted behind ConcurrentBuffer. You could use that as a kind of "concurrent vector" for immutable data. That's how ConcurrentVector uses it: it's an immutable array of segment pointers. * WTF.xcodeproj/project.pbxproj: * wtf/ConcurrentBuffer.h: Added. (WTF::ConcurrentBuffer::ConcurrentBuffer): (WTF::ConcurrentBuffer::~ConcurrentBuffer): (WTF::ConcurrentBuffer::growExact): (WTF::ConcurrentBuffer::grow): (WTF::ConcurrentBuffer::array const): (WTF::ConcurrentBuffer::operator[]): (WTF::ConcurrentBuffer::operator[] const): (WTF::ConcurrentBuffer::createArray): * wtf/ConcurrentVector.h: Added. (WTF::ConcurrentVectorIterator::~ConcurrentVectorIterator): (WTF::ConcurrentVectorIterator::operator* const): (WTF::ConcurrentVectorIterator::operator-> const): (WTF::ConcurrentVectorIterator::operator++): (WTF::ConcurrentVectorIterator::operator== const): (WTF::ConcurrentVectorIterator::operator!= const): (WTF::ConcurrentVectorIterator::operator=): (WTF::ConcurrentVectorIterator::ConcurrentVectorIterator): (WTF::ConcurrentVector::~ConcurrentVector): (WTF::ConcurrentVector::size const): (WTF::ConcurrentVector::isEmpty const): (WTF::ConcurrentVector::at): (WTF::ConcurrentVector::at const): (WTF::ConcurrentVector::operator[]): (WTF::ConcurrentVector::operator[] const): (WTF::ConcurrentVector::first): (WTF::ConcurrentVector::first const): (WTF::ConcurrentVector::last): (WTF::ConcurrentVector::last const): (WTF::ConcurrentVector::takeLast): (WTF::ConcurrentVector::append): (WTF::ConcurrentVector::alloc): (WTF::ConcurrentVector::removeLast): (WTF::ConcurrentVector::grow): (WTF::ConcurrentVector::begin): (WTF::ConcurrentVector::end): (WTF::ConcurrentVector::segmentExistsFor): (WTF::ConcurrentVector::segmentFor): (WTF::ConcurrentVector::subscriptFor): (WTF::ConcurrentVector::ensureSegmentsFor): (WTF::ConcurrentVector::ensureSegment): (WTF::ConcurrentVector::allocateSegment): Canonical link: https://commits.webkit.org/196644@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@225831 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-12-13 02:35:54 +00:00
}
// Growing is not concurrent. This assumes you are holding some other lock before you do this.
void growExact(size_t newSize)
{
Array* array = m_array;
if (array && newSize <= array->size)
return;
Array* newArray = createArray(newSize);
// This allows us to do ConcurrentBuffer<std::unique_ptr<>>.
// static_cast<void*> avoids triggering -Wclass-memaccess.
It should be possible to flag a cell for unconditional finalization https://bugs.webkit.org/show_bug.cgi?id=180636 Reviewed by Saam Barati. Source/JavaScriptCore: UnconditionalFinalizers were annoying - you had to allocate them and you had to manage a global linked list - but they had some nice properties: - You only did the hardest work (creating the UnconditionalFinalizer) on first GC where you survived and needed it. -> Just needing it wasn't enough. -> Just surviving wasn't enough. The new API based on IsoSubspaces meant that just surviving was enough to cause unconditional finalizer logic to be invoked. I think that's not great. InferredType got around this by making InferredStructure a cell, but this was a gross hack. For one, it meant that InferredStructure would survive during the GC in which its finalizer obviated the need for its existence. It's not really an idiom I want us to repeat because it sounds like the sort of thing that turns out to be subtly broken. We really need to have a way of indicating when you have entered into the state that requires your unconditional finalizer to be invoked. Basically, we want to be able to track the set of objects that need unconditional finalizers. Only the subset of that set that overlaps with the set of marked objects needs to be accurate. The easiest way to do this is a hierarchy of bitvectors: one to say which MarkedBlocks have objects that have unconditional finalizers, and another level to say which atoms within a MarkedBlock have unconditional finalizers. This change introduces IsoCellSet, which couples itself to the MarkedAllocator of some IsoSubspace to allow maintaining a set of objects (well, cells - you could do this with auxiliaries) that belong to that IsoSubspace. It'll have undefined behavior if you try to add/remove/contains an object that isn't in that IsoSubspace. For objects in that subspace, you can add/remove/contains and forEachMarkedCell. The cost of each IsoCellSet is at worst about 0.8% increase in size to every object in the subspace that the set is attached to. So, it makes sense to have a handful per subspace max. This change only needs one per subspace, but you could imagine more if we do this for WeakReferenceHarvester. To absolutely minimize the possibility that this incurs costs, the add/remove/contains functions can be used from any thread so long as forEachMarkedCell isn't running. This means that InferredType only needs to add itself to the set during visitChildren. Thus, it needs to both survive and need it for the hardest work to take place. The work of adding does involve a gnarly load chain that ends in a CAS: load block handle from block, load index, load segment, load bitvector, load bit -> if not set, then CAS. That's five dependent loads! However, it's perfect for running in parallel since the only write operations are to widely dispersed cache lines that contain the bits underlying the set. The best part is how forEachMarkedCell works. That skips blocks that don't have any objects that need unconditional finalizers, and only touches the memory of marked objects that have the unconditional finalizer bit set. It will walk those objects in roughly address order. I previously found that this speeds up walking over a lot of objects when I made similar changes for DOM GC (calling visitAdditionalChildren via forEachMarkedCell rather than by walking a HashSet). This change makes InferredStructure be a malloc object again, but now it's in an IsoHeap. My expectation for this change is that it's perf-neutral. Long-term, it gives us a path forward for eliminating UnconditionalFinalizer and WeakReferenceHarvester while using IsoSubspace in more places. * JavaScriptCore.xcodeproj/project.pbxproj: * Sources.txt: * heap/AtomIndices.h: Added. (JSC::AtomIndices::AtomIndices): * heap/Heap.cpp: (JSC::Heap::finalizeUnconditionalFinalizers): * heap/Heap.h: * heap/IsoCellSet.cpp: Added. (JSC::IsoCellSet::IsoCellSet): (JSC::IsoCellSet::~IsoCellSet): (JSC::IsoCellSet::addSlow): (JSC::IsoCellSet::didResizeBits): (JSC::IsoCellSet::didRemoveBlock): (JSC::IsoCellSet::sweepToFreeList): * heap/IsoCellSet.h: Added. * heap/IsoCellSetInlines.h: Added. (JSC::IsoCellSet::add): (JSC::IsoCellSet::remove): (JSC::IsoCellSet::contains const): (JSC::IsoCellSet::forEachMarkedCell): * heap/IsoSubspace.cpp: (JSC::IsoSubspace::didResizeBits): (JSC::IsoSubspace::didRemoveBlock): (JSC::IsoSubspace::didBeginSweepingToFreeList): * heap/IsoSubspace.h: * heap/MarkedAllocator.cpp: (JSC::MarkedAllocator::addBlock): (JSC::MarkedAllocator::removeBlock): * heap/MarkedAllocator.h: * heap/MarkedAllocatorInlines.h: * heap/MarkedBlock.cpp: (JSC::MarkedBlock::Handle::sweep): (JSC::MarkedBlock::Handle::isEmpty): Deleted. * heap/MarkedBlock.h: (JSC::MarkedBlock::marks const): (JSC::MarkedBlock::Handle::newlyAllocated const): * heap/MarkedBlockInlines.h: (JSC::MarkedBlock::Handle::isAllocated): (JSC::MarkedBlock::Handle::isEmpty): (JSC::MarkedBlock::Handle::emptyMode): (JSC::MarkedBlock::Handle::forEachMarkedCell): * heap/Subspace.cpp: (JSC::Subspace::didResizeBits): (JSC::Subspace::didRemoveBlock): (JSC::Subspace::didBeginSweepingToFreeList): * heap/Subspace.h: * heap/SubspaceInlines.h: (JSC::Subspace::forEachMarkedCell): * runtime/InferredStructure.cpp: (JSC::InferredStructure::InferredStructure): (JSC::InferredStructure::create): Deleted. (JSC::InferredStructure::destroy): Deleted. (JSC::InferredStructure::createStructure): Deleted. (JSC::InferredStructure::visitChildren): Deleted. (JSC::InferredStructure::finalizeUnconditionally): Deleted. (JSC::InferredStructure::finishCreation): Deleted. * runtime/InferredStructure.h: * runtime/InferredStructureWatchpoint.cpp: (JSC::InferredStructureWatchpoint::fireInternal): * runtime/InferredType.cpp: (JSC::InferredType::visitChildren): (JSC::InferredType::willStoreValueSlow): (JSC::InferredType::makeTopSlow): (JSC::InferredType::set): (JSC::InferredType::removeStructure): (JSC::InferredType::finalizeUnconditionally): * runtime/InferredType.h: * runtime/VM.cpp: (JSC::VM::VM): * runtime/VM.h: Source/WTF: This adds ConcurrentVector, which is like SegmentedVector, but wastes some space to allow resizing to proceed concurrently to access. It's not possible to resize concurrently to resizing, concurrent read/writes aren't protected from racing if they access the same element, and who knows what you'll get if you iterate up to size() while someone else append()s. The key insight is to stash all prior copies of the spine, so that nobody crashes trying to access a stale spine. I'm going to want to do the same thing for FastBitVector, by creating a segmented WordOwner class. That would require repeating the dance of having a spine that can resize while stashing old versions. So, the spine resizing logic is abstracted behind ConcurrentBuffer. You could use that as a kind of "concurrent vector" for immutable data. That's how ConcurrentVector uses it: it's an immutable array of segment pointers. * WTF.xcodeproj/project.pbxproj: * wtf/ConcurrentBuffer.h: Added. (WTF::ConcurrentBuffer::ConcurrentBuffer): (WTF::ConcurrentBuffer::~ConcurrentBuffer): (WTF::ConcurrentBuffer::growExact): (WTF::ConcurrentBuffer::grow): (WTF::ConcurrentBuffer::array const): (WTF::ConcurrentBuffer::operator[]): (WTF::ConcurrentBuffer::operator[] const): (WTF::ConcurrentBuffer::createArray): * wtf/ConcurrentVector.h: Added. (WTF::ConcurrentVectorIterator::~ConcurrentVectorIterator): (WTF::ConcurrentVectorIterator::operator* const): (WTF::ConcurrentVectorIterator::operator-> const): (WTF::ConcurrentVectorIterator::operator++): (WTF::ConcurrentVectorIterator::operator== const): (WTF::ConcurrentVectorIterator::operator!= const): (WTF::ConcurrentVectorIterator::operator=): (WTF::ConcurrentVectorIterator::ConcurrentVectorIterator): (WTF::ConcurrentVector::~ConcurrentVector): (WTF::ConcurrentVector::size const): (WTF::ConcurrentVector::isEmpty const): (WTF::ConcurrentVector::at): (WTF::ConcurrentVector::at const): (WTF::ConcurrentVector::operator[]): (WTF::ConcurrentVector::operator[] const): (WTF::ConcurrentVector::first): (WTF::ConcurrentVector::first const): (WTF::ConcurrentVector::last): (WTF::ConcurrentVector::last const): (WTF::ConcurrentVector::takeLast): (WTF::ConcurrentVector::append): (WTF::ConcurrentVector::alloc): (WTF::ConcurrentVector::removeLast): (WTF::ConcurrentVector::grow): (WTF::ConcurrentVector::begin): (WTF::ConcurrentVector::end): (WTF::ConcurrentVector::segmentExistsFor): (WTF::ConcurrentVector::segmentFor): (WTF::ConcurrentVector::subscriptFor): (WTF::ConcurrentVector::ensureSegmentsFor): (WTF::ConcurrentVector::ensureSegment): (WTF::ConcurrentVector::allocateSegment): Canonical link: https://commits.webkit.org/196644@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@225831 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-12-13 02:35:54 +00:00
if (array)
memcpy(static_cast<void*>(newArray->data), array->data, sizeof(T) * array->size);
It should be possible to flag a cell for unconditional finalization https://bugs.webkit.org/show_bug.cgi?id=180636 Reviewed by Saam Barati. Source/JavaScriptCore: UnconditionalFinalizers were annoying - you had to allocate them and you had to manage a global linked list - but they had some nice properties: - You only did the hardest work (creating the UnconditionalFinalizer) on first GC where you survived and needed it. -> Just needing it wasn't enough. -> Just surviving wasn't enough. The new API based on IsoSubspaces meant that just surviving was enough to cause unconditional finalizer logic to be invoked. I think that's not great. InferredType got around this by making InferredStructure a cell, but this was a gross hack. For one, it meant that InferredStructure would survive during the GC in which its finalizer obviated the need for its existence. It's not really an idiom I want us to repeat because it sounds like the sort of thing that turns out to be subtly broken. We really need to have a way of indicating when you have entered into the state that requires your unconditional finalizer to be invoked. Basically, we want to be able to track the set of objects that need unconditional finalizers. Only the subset of that set that overlaps with the set of marked objects needs to be accurate. The easiest way to do this is a hierarchy of bitvectors: one to say which MarkedBlocks have objects that have unconditional finalizers, and another level to say which atoms within a MarkedBlock have unconditional finalizers. This change introduces IsoCellSet, which couples itself to the MarkedAllocator of some IsoSubspace to allow maintaining a set of objects (well, cells - you could do this with auxiliaries) that belong to that IsoSubspace. It'll have undefined behavior if you try to add/remove/contains an object that isn't in that IsoSubspace. For objects in that subspace, you can add/remove/contains and forEachMarkedCell. The cost of each IsoCellSet is at worst about 0.8% increase in size to every object in the subspace that the set is attached to. So, it makes sense to have a handful per subspace max. This change only needs one per subspace, but you could imagine more if we do this for WeakReferenceHarvester. To absolutely minimize the possibility that this incurs costs, the add/remove/contains functions can be used from any thread so long as forEachMarkedCell isn't running. This means that InferredType only needs to add itself to the set during visitChildren. Thus, it needs to both survive and need it for the hardest work to take place. The work of adding does involve a gnarly load chain that ends in a CAS: load block handle from block, load index, load segment, load bitvector, load bit -> if not set, then CAS. That's five dependent loads! However, it's perfect for running in parallel since the only write operations are to widely dispersed cache lines that contain the bits underlying the set. The best part is how forEachMarkedCell works. That skips blocks that don't have any objects that need unconditional finalizers, and only touches the memory of marked objects that have the unconditional finalizer bit set. It will walk those objects in roughly address order. I previously found that this speeds up walking over a lot of objects when I made similar changes for DOM GC (calling visitAdditionalChildren via forEachMarkedCell rather than by walking a HashSet). This change makes InferredStructure be a malloc object again, but now it's in an IsoHeap. My expectation for this change is that it's perf-neutral. Long-term, it gives us a path forward for eliminating UnconditionalFinalizer and WeakReferenceHarvester while using IsoSubspace in more places. * JavaScriptCore.xcodeproj/project.pbxproj: * Sources.txt: * heap/AtomIndices.h: Added. (JSC::AtomIndices::AtomIndices): * heap/Heap.cpp: (JSC::Heap::finalizeUnconditionalFinalizers): * heap/Heap.h: * heap/IsoCellSet.cpp: Added. (JSC::IsoCellSet::IsoCellSet): (JSC::IsoCellSet::~IsoCellSet): (JSC::IsoCellSet::addSlow): (JSC::IsoCellSet::didResizeBits): (JSC::IsoCellSet::didRemoveBlock): (JSC::IsoCellSet::sweepToFreeList): * heap/IsoCellSet.h: Added. * heap/IsoCellSetInlines.h: Added. (JSC::IsoCellSet::add): (JSC::IsoCellSet::remove): (JSC::IsoCellSet::contains const): (JSC::IsoCellSet::forEachMarkedCell): * heap/IsoSubspace.cpp: (JSC::IsoSubspace::didResizeBits): (JSC::IsoSubspace::didRemoveBlock): (JSC::IsoSubspace::didBeginSweepingToFreeList): * heap/IsoSubspace.h: * heap/MarkedAllocator.cpp: (JSC::MarkedAllocator::addBlock): (JSC::MarkedAllocator::removeBlock): * heap/MarkedAllocator.h: * heap/MarkedAllocatorInlines.h: * heap/MarkedBlock.cpp: (JSC::MarkedBlock::Handle::sweep): (JSC::MarkedBlock::Handle::isEmpty): Deleted. * heap/MarkedBlock.h: (JSC::MarkedBlock::marks const): (JSC::MarkedBlock::Handle::newlyAllocated const): * heap/MarkedBlockInlines.h: (JSC::MarkedBlock::Handle::isAllocated): (JSC::MarkedBlock::Handle::isEmpty): (JSC::MarkedBlock::Handle::emptyMode): (JSC::MarkedBlock::Handle::forEachMarkedCell): * heap/Subspace.cpp: (JSC::Subspace::didResizeBits): (JSC::Subspace::didRemoveBlock): (JSC::Subspace::didBeginSweepingToFreeList): * heap/Subspace.h: * heap/SubspaceInlines.h: (JSC::Subspace::forEachMarkedCell): * runtime/InferredStructure.cpp: (JSC::InferredStructure::InferredStructure): (JSC::InferredStructure::create): Deleted. (JSC::InferredStructure::destroy): Deleted. (JSC::InferredStructure::createStructure): Deleted. (JSC::InferredStructure::visitChildren): Deleted. (JSC::InferredStructure::finalizeUnconditionally): Deleted. (JSC::InferredStructure::finishCreation): Deleted. * runtime/InferredStructure.h: * runtime/InferredStructureWatchpoint.cpp: (JSC::InferredStructureWatchpoint::fireInternal): * runtime/InferredType.cpp: (JSC::InferredType::visitChildren): (JSC::InferredType::willStoreValueSlow): (JSC::InferredType::makeTopSlow): (JSC::InferredType::set): (JSC::InferredType::removeStructure): (JSC::InferredType::finalizeUnconditionally): * runtime/InferredType.h: * runtime/VM.cpp: (JSC::VM::VM): * runtime/VM.h: Source/WTF: This adds ConcurrentVector, which is like SegmentedVector, but wastes some space to allow resizing to proceed concurrently to access. It's not possible to resize concurrently to resizing, concurrent read/writes aren't protected from racing if they access the same element, and who knows what you'll get if you iterate up to size() while someone else append()s. The key insight is to stash all prior copies of the spine, so that nobody crashes trying to access a stale spine. I'm going to want to do the same thing for FastBitVector, by creating a segmented WordOwner class. That would require repeating the dance of having a spine that can resize while stashing old versions. So, the spine resizing logic is abstracted behind ConcurrentBuffer. You could use that as a kind of "concurrent vector" for immutable data. That's how ConcurrentVector uses it: it's an immutable array of segment pointers. * WTF.xcodeproj/project.pbxproj: * wtf/ConcurrentBuffer.h: Added. (WTF::ConcurrentBuffer::ConcurrentBuffer): (WTF::ConcurrentBuffer::~ConcurrentBuffer): (WTF::ConcurrentBuffer::growExact): (WTF::ConcurrentBuffer::grow): (WTF::ConcurrentBuffer::array const): (WTF::ConcurrentBuffer::operator[]): (WTF::ConcurrentBuffer::operator[] const): (WTF::ConcurrentBuffer::createArray): * wtf/ConcurrentVector.h: Added. (WTF::ConcurrentVectorIterator::~ConcurrentVectorIterator): (WTF::ConcurrentVectorIterator::operator* const): (WTF::ConcurrentVectorIterator::operator-> const): (WTF::ConcurrentVectorIterator::operator++): (WTF::ConcurrentVectorIterator::operator== const): (WTF::ConcurrentVectorIterator::operator!= const): (WTF::ConcurrentVectorIterator::operator=): (WTF::ConcurrentVectorIterator::ConcurrentVectorIterator): (WTF::ConcurrentVector::~ConcurrentVector): (WTF::ConcurrentVector::size const): (WTF::ConcurrentVector::isEmpty const): (WTF::ConcurrentVector::at): (WTF::ConcurrentVector::at const): (WTF::ConcurrentVector::operator[]): (WTF::ConcurrentVector::operator[] const): (WTF::ConcurrentVector::first): (WTF::ConcurrentVector::first const): (WTF::ConcurrentVector::last): (WTF::ConcurrentVector::last const): (WTF::ConcurrentVector::takeLast): (WTF::ConcurrentVector::append): (WTF::ConcurrentVector::alloc): (WTF::ConcurrentVector::removeLast): (WTF::ConcurrentVector::grow): (WTF::ConcurrentVector::begin): (WTF::ConcurrentVector::end): (WTF::ConcurrentVector::segmentExistsFor): (WTF::ConcurrentVector::segmentFor): (WTF::ConcurrentVector::subscriptFor): (WTF::ConcurrentVector::ensureSegmentsFor): (WTF::ConcurrentVector::ensureSegment): (WTF::ConcurrentVector::allocateSegment): Canonical link: https://commits.webkit.org/196644@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@225831 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-12-13 02:35:54 +00:00
for (size_t i = array ? array->size : 0; i < newSize; ++i)
new (newArray->data + i) T();
WTF::storeStoreFence();
m_array = newArray;
WTF::storeStoreFence();
m_allArrays.append(newArray);
}
void grow(size_t newSize)
{
size_t size = m_array ? m_array->size : 0;
if (newSize <= size)
return;
growExact(std::max(newSize, size * 2));
}
struct Array {
size_t size; // This is an immutable size.
T data[1];
};
Array* array() const { return m_array; }
T& operator[](size_t index) { return m_array->data[index]; }
const T& operator[](size_t index) const { return m_array->data[index]; }
private:
Array* createArray(size_t size)
{
Checked<size_t> objectSize = sizeof(T);
objectSize *= size;
objectSize += static_cast<size_t>(OBJECT_OFFSETOF(Array, data));
Rename Checked::unsafeGet() to Checked::value() https://bugs.webkit.org/show_bug.cgi?id=226514 Reviewed by Darin Adler. Rename Checked::unsafeGet() to Checked::value(). The "unsafeGet" naming is confusing as this function isn't really unsafe since it will crash if the value has overflowed. Also add an `operator T()` to implicitly convert a Checked to its underlying type without needing to call value(). Source/JavaScriptCore: * b3/B3Const32Value.cpp: (JSC::B3::Const32Value::checkAddConstant const): (JSC::B3::Const32Value::checkSubConstant const): (JSC::B3::Const32Value::checkMulConstant const): * b3/B3Const64Value.cpp: (JSC::B3::Const64Value::checkAddConstant const): (JSC::B3::Const64Value::checkSubConstant const): (JSC::B3::Const64Value::checkMulConstant const): * bytecompiler/BytecodeGenerator.h: (JSC::FinallyContext::numberOfBreaksOrContinues const): * dfg/DFGConstantFoldingPhase.cpp: (JSC::DFG::ConstantFoldingPhase::foldConstants): * dfg/DFGOperations.cpp: (JSC::DFG::JSC_DEFINE_JIT_OPERATION): * ftl/FTLLowerDFGToB3.cpp: (JSC::FTL::DFG::LowerDFGToB3::compileNewArrayWithSpread): (JSC::FTL::DFG::LowerDFGToB3::compileSpread): (JSC::FTL::DFG::LowerDFGToB3::compileCallOrConstructVarargsSpread): (JSC::FTL::DFG::LowerDFGToB3::compileForwardVarargsWithSpread): * ftl/FTLOperations.cpp: (JSC::FTL::JSC_DEFINE_JIT_OPERATION): * heap/Heap.cpp: (JSC::Heap::deprecatedReportExtraMemorySlowCase): (JSC::Heap::extraMemorySize): (JSC::Heap::updateAllocationLimits): (JSC::Heap::reportExtraMemoryVisited): * heap/SlotVisitor.cpp: (JSC::SlotVisitor::propagateExternalMemoryVisitedIfNecessary): * runtime/ArgList.cpp: (JSC::MarkedArgumentBuffer::slowEnsureCapacity): (JSC::MarkedArgumentBuffer::expandCapacity): * runtime/ArrayPrototype.cpp: (JSC::concatAppendOne): (JSC::JSC_DEFINE_HOST_FUNCTION): * runtime/CommonSlowPaths.cpp: (JSC::JSC_DEFINE_COMMON_SLOW_PATH): * runtime/DirectArguments.h: * runtime/HashMapImpl.h: (JSC::HashMapBuffer::allocationSize): (JSC::HashMapImpl::HashMapImpl): * runtime/HashMapImplInlines.h: (JSC::nextCapacity): (JSC::HashMapImpl<HashMapBucketType>::finishCreation): * runtime/JSBigInt.cpp: (JSC::JSBigInt::parseInt): * runtime/JSImmutableButterfly.h: (JSC::JSImmutableButterfly::tryCreate): * runtime/JSLexicalEnvironment.h: (JSC::JSLexicalEnvironment::offsetOfVariable): (JSC::JSLexicalEnvironment::allocationSizeForScopeSize): * runtime/JSObject.h: * runtime/JSPropertyNameEnumerator.cpp: (JSC::JSPropertyNameEnumerator::create): * runtime/JSString.h: * runtime/ScopedArguments.cpp: (JSC::ScopedArguments::createUninitialized): * runtime/StringPrototype.cpp: (JSC::jsSpliceSubstrings): (JSC::jsSpliceSubstringsWithSeparators): * runtime/StructureChain.cpp: (JSC::StructureChain::create): * runtime/VM.h: (JSC::ScratchBuffer::allocationSize): * runtime/WeakMapImpl.h: (JSC::WeakMapBuffer::allocationSize): * wasm/WasmAirIRGenerator.cpp: (JSC::Wasm::AirIRGenerator::AirIRGenerator): (JSC::Wasm::AirIRGenerator::emitCallPatchpoint): * wasm/WasmB3IRGenerator.cpp: (JSC::Wasm::B3IRGenerator::B3IRGenerator): * wasm/WasmInstance.cpp: * wasm/WasmInstance.h: (JSC::Wasm::Instance::allocationSize): * wasm/WasmLLIntGenerator.cpp: (JSC::Wasm::LLIntGenerator::push): (JSC::Wasm::LLIntGenerator::getDropKeepCount): (JSC::Wasm::LLIntGenerator::walkExpressionStack): (JSC::Wasm::LLIntGenerator::finalize): (JSC::Wasm::LLIntGenerator::callInformationForCaller): (JSC::Wasm::LLIntGenerator::addLoop): (JSC::Wasm::LLIntGenerator::addTopLevel): (JSC::Wasm::LLIntGenerator::addBlock): (JSC::Wasm::LLIntGenerator::addIf): (JSC::Wasm::LLIntGenerator::addElseToUnreachable): * wasm/WasmSignature.h: (JSC::Wasm::Signature::allocatedSize): * wasm/WasmStreamingParser.cpp: (JSC::Wasm::StreamingParser::addBytes): * wasm/WasmTable.cpp: (JSC::Wasm::Table::Table): (JSC::Wasm::Table::grow): (JSC::Wasm::FuncRefTable::FuncRefTable): * yarr/YarrInterpreter.cpp: (JSC::Yarr::Interpreter::DisjunctionContext::allocationSize): (JSC::Yarr::Interpreter::ParenthesesDisjunctionContext::allocationSize): (JSC::Yarr::Interpreter::allocParenthesesDisjunctionContext): (JSC::Yarr::ByteCompiler::atomCharacterClass): (JSC::Yarr::ByteCompiler::atomBackReference): (JSC::Yarr::ByteCompiler::atomParentheticalAssertionEnd): (JSC::Yarr::ByteCompiler::atomParenthesesSubpatternEnd): (JSC::Yarr::ByteCompiler::atomParenthesesOnceEnd): (JSC::Yarr::ByteCompiler::atomParenthesesTerminalEnd): (JSC::Yarr::ByteCompiler::emitDisjunction): * yarr/YarrInterpreter.h: (JSC::Yarr::ByteTerm::ByteTerm): (JSC::Yarr::ByteTerm::CheckInput): (JSC::Yarr::ByteTerm::UncheckInput): * yarr/YarrJIT.cpp: * yarr/YarrParser.h: (JSC::Yarr::Parser::consumeNumber): * yarr/YarrPattern.cpp: (JSC::Yarr::PatternTerm::dumpQuantifier): Source/WebCore: * bindings/js/SerializedScriptValue.cpp: (WebCore::CloneDeserializer::readTerminal): * dom/TextEncoderStreamEncoder.cpp: (WebCore::TextEncoderStreamEncoder::encode): * editing/markup.cpp: (WebCore::StyledMarkupAccumulator::takeResults): * html/FileInputType.cpp: (WebCore::FileInputType::saveFormControlState const): * html/ImageData.cpp: (WebCore::ImageData::create): (WebCore::ImageData::createUninitialized): * html/MediaElementSession.cpp: (WebCore::isElementRectMostlyInMainFrame): * html/canvas/WebGL2RenderingContext.cpp: (WebCore::WebGL2RenderingContext::sliceArrayBufferView): (WebCore::WebGL2RenderingContext::copyBufferSubData): (WebCore::WebGL2RenderingContext::getBufferSubData): (WebCore::WebGL2RenderingContext::validateClearBuffer): * html/canvas/WebGLBuffer.cpp: (WebCore::WebGLBuffer::associateBufferSubDataImpl): (WebCore::WebGLBuffer::associateCopyBufferSubData): * html/canvas/WebGLRenderingContextBase.cpp: (WebCore::clip2D): (WebCore::WebGLRenderingContextBase::validateDrawArrays): (WebCore::WebGLRenderingContextBase::validateDrawElements): (WebCore::WebGLRenderingContextBase::validateTexFuncData): (WebCore::WebGLRenderingContextBase::validateCompressedTexFuncData): (WebCore::WebGLRenderingContextBase::validateSimulatedVertexAttrib0): * html/canvas/WebGLRenderingContextBase.h: (WebCore::WebGLRenderingContextBase::validateTexImageSubRectangle): (WebCore::WebGLRenderingContextBase::checkedAddAndMultiply): * page/FrameView.h: (WebCore::FrameView::incrementVisuallyNonEmptyPixelCount): * page/History.cpp: (WebCore::History::stateObjectAdded): * platform/audio/AudioArray.h: (WebCore::AudioArray::resize): * platform/audio/cocoa/AudioFileReaderCocoa.cpp: (WebCore::tryCreateAudioBufferList): * platform/audio/cocoa/CARingBuffer.cpp: (WebCore::CARingBuffer::adoptStorage): (WebCore::CARingBuffer::initializeAfterAllocation): (WebCore::CARingBuffer::allocate): * platform/audio/cocoa/WebAudioBufferList.cpp: (WebCore::WebAudioBufferList::WebAudioBufferList): * platform/graphics/FormatConverter.h: (WebCore::FormatConverter::FormatConverter): * platform/graphics/GraphicsContextGL.cpp: (WebCore::GraphicsContextGL::computeImageSizeInBytes): * platform/graphics/ImageBackingStore.h: (WebCore::ImageBackingStore::setSize): (WebCore::ImageBackingStore::clear): * platform/graphics/ImageBufferBackend.cpp: (WebCore::ImageBufferBackend::calculateMemoryCost): * platform/graphics/ImageFrame.h: (WebCore::ImageFrame::frameBytes const): * platform/graphics/ImageSource.cpp: (WebCore::ImageSource::maximumSubsamplingLevel): * platform/graphics/PixelBuffer.cpp: (WebCore::PixelBuffer::tryCreateForDecoding): (WebCore::PixelBuffer::tryCreate): * platform/graphics/PixelBuffer.h: (WebCore::PixelBuffer::encode const): (WebCore::PixelBuffer::decode): * platform/graphics/avfoundation/objc/ImageDecoderAVFObjC.mm: (WebCore::ImageDecoderAVFObjC::frameBytesAtIndex const): * platform/graphics/avfoundation/objc/MediaSampleAVFObjC.mm: (WebCore::MediaSampleAVFObjC::setByteRangeOffset): (WebCore::MediaSampleAVFObjC::byteRangeForAttachment const): * platform/graphics/ca/GraphicsLayerCA.cpp: (WebCore::GraphicsLayerCA::updateBackdropFilters): * platform/graphics/ca/LayerPool.cpp: (WebCore::LayerPool::backingStoreBytesForSize): * platform/graphics/cg/GraphicsContextGLCG.cpp: (WebCore::GraphicsContextGLImageExtractor::extractImage): * platform/graphics/cg/ImageBufferCGBackend.cpp: (WebCore::ImageBufferCGBackend::calculateBytesPerRow): * platform/graphics/cg/ImageDecoderCG.cpp: (WebCore::ImageDecoderCG::frameBytesAtIndex const): * platform/graphics/cocoa/SourceBufferParser.cpp: (WebCore::SourceBufferParser::Segment::read const): * platform/graphics/filters/FEColorMatrix.cpp: (WebCore::effectApplyAccelerated): * platform/graphics/filters/FEGaussianBlur.cpp: (WebCore::FEGaussianBlur::platformApplySoftware): * platform/graphics/filters/FETurbulence.cpp: (WebCore::FETurbulence::platformApplySoftware): * platform/graphics/filters/FilterEffect.cpp: (WebCore::FilterEffect::unmultipliedResult): (WebCore::FilterEffect::premultipliedResult): (WebCore::copyPremultiplyingAlpha): (WebCore::copyUnpremultiplyingAlpha): * platform/graphics/gpu/cocoa/GPUBindGroupAllocatorMetal.mm: (WebCore::GPUBindGroupAllocator::allocateAndSetEncoders): (WebCore::GPUBindGroupAllocator::reallocate): * platform/graphics/gpu/cocoa/GPUCommandBufferMetal.mm: (WebCore::GPUCommandBuffer::copyBufferToBuffer): * platform/graphics/gpu/cocoa/GPURenderPassEncoderMetal.mm: (WebCore::GPURenderPassEncoder::drawIndexed): * platform/graphics/gstreamer/ImageDecoderGStreamer.cpp: (WebCore::ImageDecoderGStreamer::frameBytesAtIndex const): * platform/graphics/nicosia/NicosiaBuffer.cpp: (Nicosia::Buffer::Buffer): * platform/graphics/win/Direct2DUtilities.cpp: (WebCore::Direct2D::createDirect2DImageSurfaceWithData): * platform/graphics/win/ImageBufferDirect2DBackend.cpp: (WebCore::ImageBufferDirect2DBackend::compatibleBitmap): * platform/graphics/win/ImageDecoderDirect2D.cpp: (WebCore::ImageDecoderDirect2D::frameBytesAtIndex const): * platform/image-decoders/ScalableImageDecoder.cpp: (WebCore::ScalableImageDecoder::frameBytesAtIndex const): * platform/image-decoders/jpeg2000/JPEG2000ImageDecoder.cpp: (WebCore::sycc444ToRGB): (WebCore::sycc422ToRGB): (WebCore::sycc420ToRGB): * platform/ios/LegacyTileLayerPool.mm: (WebCore::LegacyTileLayerPool::bytesBackingLayerWithPixelSize): * platform/text/TextCodecUTF16.cpp: (WebCore::TextCodecUTF16::encode const): * platform/text/TextCodecUTF8.cpp: (WebCore::TextCodecUTF8::encodeUTF8): * rendering/RenderLayerCompositor.cpp: (WebCore::RenderLayerCompositor::requiresCompositingForCanvas const): * rendering/shapes/Shape.cpp: (WebCore::Shape::createRasterShape): * storage/StorageMap.cpp: (WebCore::StorageMap::setItem): * xml/XSLStyleSheetLibxslt.cpp: (WebCore::XSLStyleSheet::parseString): * xml/XSLTProcessorLibxslt.cpp: (WebCore::xsltParamArrayFromParameterMap): * xml/parser/CharacterReferenceParserInlines.h: (WebCore::consumeCharacterReference): Source/WebKit: * GPUProcess/graphics/RemoteRenderingBackend.cpp: (WebKit::RemoteRenderingBackend::nextDestinationImageBufferAfterApplyingDisplayLists): * NetworkProcess/WebStorage/LocalStorageDatabase.cpp: (WebKit::LocalStorageDatabase::setItem): * NetworkProcess/cache/CacheStorageEngineCache.cpp: (WebKit::CacheStorage::Cache::put): * Platform/IPC/ArgumentCoders.h: * Platform/IPC/cocoa/ConnectionCocoa.mm: (IPC::Connection::sendOutgoingMessage): (IPC::createMessageDecoder): * Platform/IPC/cocoa/MachMessage.cpp: (IPC::MachMessage::create): * Shared/ShareableBitmap.cpp: (WebKit::ShareableBitmap::Handle::encode const): (WebKit::ShareableBitmap::create): (WebKit::ShareableBitmap::createShareable): * Shared/ShareableBitmap.h: (WebKit::ShareableBitmap::bytesPerRow const): (WebKit::ShareableBitmap::sizeInBytes const): * Shared/ShareableResource.cpp: (WebKit::ShareableResource::create): * Shared/cg/ShareableBitmapCG.cpp: (WebKit::ShareableBitmap::calculateBytesPerRow): (WebKit::ShareableBitmap::createGraphicsContext): (WebKit::ShareableBitmap::createCGImage const): * Shared/mac/MediaFormatReader/MediaFormatReader.cpp: (WebKit::MediaFormatReader::copyTrackArray): * Shared/mac/MediaFormatReader/MediaSampleCursor.cpp: (WebKit::MediaSampleCursor::copySampleLocation const): * WebProcess/GPU/graphics/DisplayListWriterHandle.cpp: (WebKit::DisplayListWriterHandle::advance): * WebProcess/GPU/graphics/ImageBufferShareableBitmapBackend.cpp: (WebKit::ImageBufferShareableBitmapBackend::calculateBytesPerRow): * WebProcess/GPU/media/RemoteImageDecoderAVF.cpp: (WebKit::RemoteImageDecoderAVF::frameBytesAtIndex const): * WebProcess/Network/WebSocketChannel.cpp: (WebKit::WebSocketChannel::increaseBufferedAmount): * WebProcess/WebPage/ios/WebPageIOS.mm: (WebKit::WebPage::requestEvasionRectsAboveSelection): (WebKit::WebPage::updateSelectionWithDelta): Source/WTF: * wtf/CheckedArithmetic.h: (WTF::Checked::operator! const): (WTF::Checked::operator bool const): (WTF::Checked::operator T const): (WTF::Checked::value const): (WTF::Checked::operator==): (WTF::Checked::operator< const): (WTF::Checked::operator<= const): (WTF::Checked::operator> const): (WTF::Checked::operator>= const): * wtf/ConcurrentBuffer.h: * wtf/FastMalloc.cpp: (WTF::fastCalloc): (WTF::tryFastCalloc): * wtf/Gigacage.cpp: (Gigacage::tryMallocArray): * wtf/URLHelpers.cpp: (WTF::URLHelpers::userVisibleURL): * wtf/URLParser.cpp: (WTF::URLParser::parseIPv4Piece): * wtf/UniqueArray.h: * wtf/cocoa/NSURLExtras.mm: (WTF::dataWithUserTypedString): * wtf/glib/SocketConnection.cpp: (WTF::SocketConnection::readMessage): (WTF::SocketConnection::sendMessage): * wtf/text/CString.cpp: (WTF::CStringBuffer::createUninitialized): * wtf/text/StringBuffer.h: (WTF::StringBuffer::StringBuffer): * wtf/text/StringBuilderJSON.cpp: (WTF::StringBuilder::appendQuotedJSONString): * wtf/text/StringConcatenate.h: (WTF::tryMakeStringFromAdapters): * wtf/text/StringImpl.h: (WTF::StringImpl::allocationSize): * wtf/text/StringToIntegerConversion.h: (WTF::parseInteger): Tools: * TestWebKitAPI/Tests/WTF/CheckedArithmeticOperations.cpp: (TestWebKitAPI::CheckedArithmeticTester::run): (TestWebKitAPI::AllowMixedSignednessTest::run): (TestWebKitAPI::TEST): * TestWebKitAPI/Tests/WebCore/IntRectTests.cpp: (TestWebKitAPI::TEST): * TestWebKitAPI/Tests/WebCore/IntSizeTests.cpp: (TestWebKitAPI::TEST): Canonical link: https://commits.webkit.org/238371@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@278338 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-06-02 05:21:13 +00:00
Array* result = static_cast<Array*>(ConcurrentBufferMalloc::malloc(objectSize));
It should be possible to flag a cell for unconditional finalization https://bugs.webkit.org/show_bug.cgi?id=180636 Reviewed by Saam Barati. Source/JavaScriptCore: UnconditionalFinalizers were annoying - you had to allocate them and you had to manage a global linked list - but they had some nice properties: - You only did the hardest work (creating the UnconditionalFinalizer) on first GC where you survived and needed it. -> Just needing it wasn't enough. -> Just surviving wasn't enough. The new API based on IsoSubspaces meant that just surviving was enough to cause unconditional finalizer logic to be invoked. I think that's not great. InferredType got around this by making InferredStructure a cell, but this was a gross hack. For one, it meant that InferredStructure would survive during the GC in which its finalizer obviated the need for its existence. It's not really an idiom I want us to repeat because it sounds like the sort of thing that turns out to be subtly broken. We really need to have a way of indicating when you have entered into the state that requires your unconditional finalizer to be invoked. Basically, we want to be able to track the set of objects that need unconditional finalizers. Only the subset of that set that overlaps with the set of marked objects needs to be accurate. The easiest way to do this is a hierarchy of bitvectors: one to say which MarkedBlocks have objects that have unconditional finalizers, and another level to say which atoms within a MarkedBlock have unconditional finalizers. This change introduces IsoCellSet, which couples itself to the MarkedAllocator of some IsoSubspace to allow maintaining a set of objects (well, cells - you could do this with auxiliaries) that belong to that IsoSubspace. It'll have undefined behavior if you try to add/remove/contains an object that isn't in that IsoSubspace. For objects in that subspace, you can add/remove/contains and forEachMarkedCell. The cost of each IsoCellSet is at worst about 0.8% increase in size to every object in the subspace that the set is attached to. So, it makes sense to have a handful per subspace max. This change only needs one per subspace, but you could imagine more if we do this for WeakReferenceHarvester. To absolutely minimize the possibility that this incurs costs, the add/remove/contains functions can be used from any thread so long as forEachMarkedCell isn't running. This means that InferredType only needs to add itself to the set during visitChildren. Thus, it needs to both survive and need it for the hardest work to take place. The work of adding does involve a gnarly load chain that ends in a CAS: load block handle from block, load index, load segment, load bitvector, load bit -> if not set, then CAS. That's five dependent loads! However, it's perfect for running in parallel since the only write operations are to widely dispersed cache lines that contain the bits underlying the set. The best part is how forEachMarkedCell works. That skips blocks that don't have any objects that need unconditional finalizers, and only touches the memory of marked objects that have the unconditional finalizer bit set. It will walk those objects in roughly address order. I previously found that this speeds up walking over a lot of objects when I made similar changes for DOM GC (calling visitAdditionalChildren via forEachMarkedCell rather than by walking a HashSet). This change makes InferredStructure be a malloc object again, but now it's in an IsoHeap. My expectation for this change is that it's perf-neutral. Long-term, it gives us a path forward for eliminating UnconditionalFinalizer and WeakReferenceHarvester while using IsoSubspace in more places. * JavaScriptCore.xcodeproj/project.pbxproj: * Sources.txt: * heap/AtomIndices.h: Added. (JSC::AtomIndices::AtomIndices): * heap/Heap.cpp: (JSC::Heap::finalizeUnconditionalFinalizers): * heap/Heap.h: * heap/IsoCellSet.cpp: Added. (JSC::IsoCellSet::IsoCellSet): (JSC::IsoCellSet::~IsoCellSet): (JSC::IsoCellSet::addSlow): (JSC::IsoCellSet::didResizeBits): (JSC::IsoCellSet::didRemoveBlock): (JSC::IsoCellSet::sweepToFreeList): * heap/IsoCellSet.h: Added. * heap/IsoCellSetInlines.h: Added. (JSC::IsoCellSet::add): (JSC::IsoCellSet::remove): (JSC::IsoCellSet::contains const): (JSC::IsoCellSet::forEachMarkedCell): * heap/IsoSubspace.cpp: (JSC::IsoSubspace::didResizeBits): (JSC::IsoSubspace::didRemoveBlock): (JSC::IsoSubspace::didBeginSweepingToFreeList): * heap/IsoSubspace.h: * heap/MarkedAllocator.cpp: (JSC::MarkedAllocator::addBlock): (JSC::MarkedAllocator::removeBlock): * heap/MarkedAllocator.h: * heap/MarkedAllocatorInlines.h: * heap/MarkedBlock.cpp: (JSC::MarkedBlock::Handle::sweep): (JSC::MarkedBlock::Handle::isEmpty): Deleted. * heap/MarkedBlock.h: (JSC::MarkedBlock::marks const): (JSC::MarkedBlock::Handle::newlyAllocated const): * heap/MarkedBlockInlines.h: (JSC::MarkedBlock::Handle::isAllocated): (JSC::MarkedBlock::Handle::isEmpty): (JSC::MarkedBlock::Handle::emptyMode): (JSC::MarkedBlock::Handle::forEachMarkedCell): * heap/Subspace.cpp: (JSC::Subspace::didResizeBits): (JSC::Subspace::didRemoveBlock): (JSC::Subspace::didBeginSweepingToFreeList): * heap/Subspace.h: * heap/SubspaceInlines.h: (JSC::Subspace::forEachMarkedCell): * runtime/InferredStructure.cpp: (JSC::InferredStructure::InferredStructure): (JSC::InferredStructure::create): Deleted. (JSC::InferredStructure::destroy): Deleted. (JSC::InferredStructure::createStructure): Deleted. (JSC::InferredStructure::visitChildren): Deleted. (JSC::InferredStructure::finalizeUnconditionally): Deleted. (JSC::InferredStructure::finishCreation): Deleted. * runtime/InferredStructure.h: * runtime/InferredStructureWatchpoint.cpp: (JSC::InferredStructureWatchpoint::fireInternal): * runtime/InferredType.cpp: (JSC::InferredType::visitChildren): (JSC::InferredType::willStoreValueSlow): (JSC::InferredType::makeTopSlow): (JSC::InferredType::set): (JSC::InferredType::removeStructure): (JSC::InferredType::finalizeUnconditionally): * runtime/InferredType.h: * runtime/VM.cpp: (JSC::VM::VM): * runtime/VM.h: Source/WTF: This adds ConcurrentVector, which is like SegmentedVector, but wastes some space to allow resizing to proceed concurrently to access. It's not possible to resize concurrently to resizing, concurrent read/writes aren't protected from racing if they access the same element, and who knows what you'll get if you iterate up to size() while someone else append()s. The key insight is to stash all prior copies of the spine, so that nobody crashes trying to access a stale spine. I'm going to want to do the same thing for FastBitVector, by creating a segmented WordOwner class. That would require repeating the dance of having a spine that can resize while stashing old versions. So, the spine resizing logic is abstracted behind ConcurrentBuffer. You could use that as a kind of "concurrent vector" for immutable data. That's how ConcurrentVector uses it: it's an immutable array of segment pointers. * WTF.xcodeproj/project.pbxproj: * wtf/ConcurrentBuffer.h: Added. (WTF::ConcurrentBuffer::ConcurrentBuffer): (WTF::ConcurrentBuffer::~ConcurrentBuffer): (WTF::ConcurrentBuffer::growExact): (WTF::ConcurrentBuffer::grow): (WTF::ConcurrentBuffer::array const): (WTF::ConcurrentBuffer::operator[]): (WTF::ConcurrentBuffer::operator[] const): (WTF::ConcurrentBuffer::createArray): * wtf/ConcurrentVector.h: Added. (WTF::ConcurrentVectorIterator::~ConcurrentVectorIterator): (WTF::ConcurrentVectorIterator::operator* const): (WTF::ConcurrentVectorIterator::operator-> const): (WTF::ConcurrentVectorIterator::operator++): (WTF::ConcurrentVectorIterator::operator== const): (WTF::ConcurrentVectorIterator::operator!= const): (WTF::ConcurrentVectorIterator::operator=): (WTF::ConcurrentVectorIterator::ConcurrentVectorIterator): (WTF::ConcurrentVector::~ConcurrentVector): (WTF::ConcurrentVector::size const): (WTF::ConcurrentVector::isEmpty const): (WTF::ConcurrentVector::at): (WTF::ConcurrentVector::at const): (WTF::ConcurrentVector::operator[]): (WTF::ConcurrentVector::operator[] const): (WTF::ConcurrentVector::first): (WTF::ConcurrentVector::first const): (WTF::ConcurrentVector::last): (WTF::ConcurrentVector::last const): (WTF::ConcurrentVector::takeLast): (WTF::ConcurrentVector::append): (WTF::ConcurrentVector::alloc): (WTF::ConcurrentVector::removeLast): (WTF::ConcurrentVector::grow): (WTF::ConcurrentVector::begin): (WTF::ConcurrentVector::end): (WTF::ConcurrentVector::segmentExistsFor): (WTF::ConcurrentVector::segmentFor): (WTF::ConcurrentVector::subscriptFor): (WTF::ConcurrentVector::ensureSegmentsFor): (WTF::ConcurrentVector::ensureSegment): (WTF::ConcurrentVector::allocateSegment): Canonical link: https://commits.webkit.org/196644@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@225831 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-12-13 02:35:54 +00:00
result->size = size;
return result;
}
Array* m_array { nullptr };
Vector<Array*> m_allArrays;
};
} // namespace WTF