haikuwebkit/Source/WTF/wtf/CheckedBoolean.h

69 lines
2.0 KiB
C
Raw Permalink Normal View History

Implement a new allocator for backing stores https://bugs.webkit.org/show_bug.cgi?id=75181 Reviewed by Filip Pizlo. Source/JavaScriptCore: We want to move away from using fastMalloc for the backing stores for some of our objects (e.g. JSArray, JSObject, JSString, etc). These backing stores have a nice property in that they only have a single owner (i.e. a single pointer to them at any one time). One way that we can take advantage of this property is to implement a simple bump allocator/copying collector, which will run alongside our normal mark/sweep collector, that only needs to update the single owner pointer rather than having to redirect an arbitrary number of pointers in from-space to to-space. This plan can give us a number of benefits. We can beat fastMalloc in terms of both performance and memory usage, we can track how much memory we're using far more accurately than our rough estimation now through the use of reportExtraMemoryCost, and we can allocate arbitrary size objects (as opposed to being limited to size classes like we have been historically). This is also another step toward moving away from lazy destruction, which will improve our memory footprint. We start by creating said allocator and moving the ArrayStorage for JSArray to use it rather than fastMalloc. The design of the collector is as follows: Allocation: -The collector allocates 64KB chunks from the OS to use for object allocation. -Each chunk contains an offset, a flag indicating if the block has been pinned, and a payload, along with next and prev pointers so that they can be put in DoublyLinkedLists. -Any allocation greater than 64KB gets its own separate oversize block, which is managed separately from the rest. -If the allocator receives a request for more than the remaining amount in the current block, it grabs a fresh block. -Grabbing a fresh block means grabbing one off of the global free list (which is now shared between the mark/sweep allocator and the bump allocator) if there is one. If there isn't a new one we do one of two things: allocate a new block from the OS if we're not ready for a GC yet, or run a GC and then try again. If we still don't have enough space after the GC, we allocate a new block from the OS. Garbage collection: -At the start of garbage collection during conservative stack scanning, if we encounter what appears to be a pointer to a bump-allocated block of memory, we pin that block so that it will not be copied for this round of collection. -We also pin any oversize blocks that we encounter, which effectively doubles as a "mark bit" for that block. Any oversize blocks that aren't pinned at the end of copying are given back to the OS. -Marking threads are now also responsible for copying bump-allocated objects to newSpace -Each marking thread has a private 64KB block into which it copies bump-allocated objects that it encounters. -When that block fills up, the marking thread gives it back to the allocator and requests a new one. -When all marking has concluded, each thread gives back its copy block, even if it isn't full. -At the conclusion of copying (which is done by the end of the marking phase), we un-pin any pinned blocks and give any blocks left in from-space to the global free list. * CMakeLists.txt: * GNUmakefile.list.am: * JavaScriptCore.gypi: * JavaScriptCore.vcproj/JavaScriptCore/JavaScriptCore.vcproj: * JavaScriptCore.vcproj/WTF/WTF.vcproj: * JavaScriptCore.xcodeproj/project.pbxproj: * Target.pri: * heap/AllocationSpace.cpp: (JSC::AllocationSpace::allocateSlowCase): (JSC::AllocationSpace::allocateBlock): (JSC::AllocationSpace::freeBlocks): * heap/AllocationSpace.h: (JSC::AllocationSpace::waterMark): * heap/BumpBlock.h: Added. (JSC::BumpBlock::BumpBlock): * heap/BumpSpace.cpp: Added. (JSC::BumpSpace::tryAllocateSlowCase): * heap/BumpSpace.h: Added. (JSC::BumpSpace::isInCopyPhase): (JSC::BumpSpace::totalMemoryAllocated): (JSC::BumpSpace::totalMemoryUtilized): * heap/BumpSpaceInlineMethods.h: Added. (JSC::BumpSpace::BumpSpace): (JSC::BumpSpace::init): (JSC::BumpSpace::contains): (JSC::BumpSpace::pin): (JSC::BumpSpace::startedCopying): (JSC::BumpSpace::doneCopying): (JSC::BumpSpace::doneFillingBlock): (JSC::BumpSpace::recycleBlock): (JSC::BumpSpace::getFreshBlock): (JSC::BumpSpace::borrowBlock): (JSC::BumpSpace::addNewBlock): (JSC::BumpSpace::allocateNewBlock): (JSC::BumpSpace::fitsInBlock): (JSC::BumpSpace::fitsInCurrentBlock): (JSC::BumpSpace::tryAllocate): (JSC::BumpSpace::tryAllocateOversize): (JSC::BumpSpace::allocateFromBlock): (JSC::BumpSpace::tryReallocate): (JSC::BumpSpace::tryReallocateOversize): (JSC::BumpSpace::isOversize): (JSC::BumpSpace::isPinned): (JSC::BumpSpace::oversizeBlockFor): (JSC::BumpSpace::blockFor): * heap/ConservativeRoots.cpp: (JSC::ConservativeRoots::ConservativeRoots): (JSC::ConservativeRoots::genericAddPointer): (JSC::ConservativeRoots::add): * heap/ConservativeRoots.h: * heap/Heap.cpp: (JSC::Heap::Heap): (JSC::Heap::blockFreeingThreadMain): (JSC::Heap::reportExtraMemoryCostSlowCase): (JSC::Heap::getConservativeRegisterRoots): (JSC::Heap::markRoots): (JSC::Heap::collect): (JSC::Heap::releaseFreeBlocks): * heap/Heap.h: (JSC::Heap::waterMark): (JSC::Heap::highWaterMark): (JSC::Heap::setHighWaterMark): (JSC::Heap::tryAllocateStorage): (JSC::Heap::tryReallocateStorage): * heap/HeapBlock.h: Added. (JSC::HeapBlock::HeapBlock): * heap/MarkStack.cpp: (JSC::MarkStackThreadSharedData::MarkStackThreadSharedData): (JSC::SlotVisitor::drain): (JSC::SlotVisitor::drainFromShared): (JSC::SlotVisitor::startCopying): (JSC::SlotVisitor::allocateNewSpace): (JSC::SlotVisitor::copy): (JSC::SlotVisitor::copyAndAppend): (JSC::SlotVisitor::doneCopying): * heap/MarkStack.h: * heap/MarkedBlock.cpp: (JSC::MarkedBlock::recycle): (JSC::MarkedBlock::MarkedBlock): * heap/MarkedBlock.h: * heap/MarkedSpace.cpp: (JSC::MarkedSpace::MarkedSpace): * heap/MarkedSpace.h: (JSC::MarkedSpace::allocate): (JSC::MarkedSpace::forEachBlock): (JSC::MarkedSpace::SizeClass::resetAllocator): * heap/SlotVisitor.h: (JSC::SlotVisitor::SlotVisitor): * heap/TinyBloomFilter.h: (JSC::TinyBloomFilter::reset): * runtime/JSArray.cpp: (JSC::JSArray::JSArray): (JSC::JSArray::finishCreation): (JSC::JSArray::tryFinishCreationUninitialized): (JSC::JSArray::~JSArray): (JSC::JSArray::enterSparseMode): (JSC::JSArray::defineOwnNumericProperty): (JSC::JSArray::setLengthWritable): (JSC::JSArray::getOwnPropertySlotByIndex): (JSC::JSArray::getOwnPropertyDescriptor): (JSC::JSArray::putByIndexBeyondVectorLength): (JSC::JSArray::deletePropertyByIndex): (JSC::JSArray::getOwnPropertyNames): (JSC::JSArray::increaseVectorLength): (JSC::JSArray::unshiftCountSlowCase): (JSC::JSArray::setLength): (JSC::JSArray::pop): (JSC::JSArray::unshiftCount): (JSC::JSArray::visitChildren): (JSC::JSArray::sortNumeric): (JSC::JSArray::sort): (JSC::JSArray::compactForSorting): (JSC::JSArray::subclassData): (JSC::JSArray::setSubclassData): (JSC::JSArray::checkConsistency): * runtime/JSArray.h: (JSC::JSArray::inSparseMode): (JSC::JSArray::isLengthWritable): * wtf/CheckedBoolean.h: Added. (CheckedBoolean::CheckedBoolean): (CheckedBoolean::~CheckedBoolean): (CheckedBoolean::operator bool): * wtf/DoublyLinkedList.h: (WTF::::push): * wtf/StdLibExtras.h: (WTF::isPointerAligned): Source/JavaScriptGlue: Added forwarding header for new CheckedBoolean used in the bump allocator. * ForwardingHeaders/wtf/CheckedBoolean.h: Added. Source/WebCore: No new tests. Added forwarding header for new CheckedBoolean used in the bump allocator. * ForwardingHeaders/wtf/CheckedBoolean.h: Added. Canonical link: https://commits.webkit.org/93486@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@105442 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2012-01-19 21:49:56 +00:00
/*
* Copyright (C) 2011 Apple Inc. All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
* are met:
* 1. Redistributions of source code must retain the above copyright
* notice, this list of conditions and the following disclaimer.
* 2. Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
*
* THIS SOFTWARE IS PROVIDED BY APPLE INC. ``AS IS'' AND ANY
* EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
* PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR
* CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
* EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
* PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
* PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
* OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
* (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
* OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
Use pragma once in WTF https://bugs.webkit.org/show_bug.cgi?id=190527 Reviewed by Chris Dumez. Source/WTF: We also need to consistently include wtf headers from within wtf so we can build wtf without symbol redefinition errors from including the copy in Source and the copy in the build directory. * wtf/ASCIICType.h: * wtf/Assertions.cpp: * wtf/Assertions.h: * wtf/Atomics.h: * wtf/AutomaticThread.cpp: * wtf/AutomaticThread.h: * wtf/BackwardsGraph.h: * wtf/Bag.h: * wtf/BagToHashMap.h: * wtf/BitVector.cpp: * wtf/BitVector.h: * wtf/Bitmap.h: * wtf/BloomFilter.h: * wtf/Box.h: * wtf/BubbleSort.h: * wtf/BumpPointerAllocator.h: * wtf/ByteOrder.h: * wtf/CPUTime.cpp: * wtf/CallbackAggregator.h: * wtf/CheckedArithmetic.h: * wtf/CheckedBoolean.h: * wtf/ClockType.cpp: * wtf/ClockType.h: * wtf/CommaPrinter.h: * wtf/CompilationThread.cpp: * wtf/CompilationThread.h: * wtf/Compiler.h: * wtf/ConcurrentPtrHashSet.cpp: * wtf/ConcurrentVector.h: * wtf/Condition.h: * wtf/CountingLock.cpp: * wtf/CrossThreadTaskHandler.cpp: * wtf/CryptographicUtilities.cpp: * wtf/CryptographicUtilities.h: * wtf/CryptographicallyRandomNumber.cpp: * wtf/CryptographicallyRandomNumber.h: * wtf/CurrentTime.cpp: * wtf/DataLog.cpp: * wtf/DataLog.h: * wtf/DateMath.cpp: * wtf/DateMath.h: * wtf/DecimalNumber.cpp: * wtf/DecimalNumber.h: * wtf/Deque.h: * wtf/DisallowCType.h: * wtf/Dominators.h: * wtf/DoublyLinkedList.h: * wtf/FastBitVector.cpp: * wtf/FastMalloc.cpp: * wtf/FastMalloc.h: * wtf/FeatureDefines.h: * wtf/FilePrintStream.cpp: * wtf/FilePrintStream.h: * wtf/FlipBytes.h: * wtf/FunctionDispatcher.cpp: * wtf/FunctionDispatcher.h: * wtf/GetPtr.h: * wtf/Gigacage.cpp: * wtf/GlobalVersion.cpp: * wtf/GraphNodeWorklist.h: * wtf/GregorianDateTime.cpp: * wtf/GregorianDateTime.h: * wtf/HashFunctions.h: * wtf/HashMap.h: * wtf/HashMethod.h: * wtf/HashSet.h: * wtf/HashTable.cpp: * wtf/HashTraits.h: * wtf/Indenter.h: * wtf/IndexSparseSet.h: * wtf/InlineASM.h: * wtf/Insertion.h: * wtf/IteratorAdaptors.h: * wtf/IteratorRange.h: * wtf/JSONValues.cpp: * wtf/JSValueMalloc.cpp: * wtf/LEBDecoder.h: * wtf/Language.cpp: * wtf/ListDump.h: * wtf/Lock.cpp: * wtf/Lock.h: * wtf/LockAlgorithm.h: * wtf/LockedPrintStream.cpp: * wtf/Locker.h: * wtf/MD5.cpp: * wtf/MD5.h: * wtf/MainThread.cpp: * wtf/MainThread.h: * wtf/MallocPtr.h: * wtf/MathExtras.h: * wtf/MediaTime.cpp: * wtf/MediaTime.h: * wtf/MemoryPressureHandler.cpp: * wtf/MessageQueue.h: * wtf/MetaAllocator.cpp: * wtf/MetaAllocator.h: * wtf/MetaAllocatorHandle.h: * wtf/MonotonicTime.cpp: * wtf/MonotonicTime.h: * wtf/NakedPtr.h: * wtf/NoLock.h: * wtf/NoTailCalls.h: * wtf/Noncopyable.h: * wtf/NumberOfCores.cpp: * wtf/NumberOfCores.h: * wtf/OSAllocator.h: * wtf/OSAllocatorPosix.cpp: * wtf/OSRandomSource.cpp: * wtf/OSRandomSource.h: * wtf/ObjcRuntimeExtras.h: * wtf/OrderMaker.h: * wtf/PackedIntVector.h: * wtf/PageAllocation.h: * wtf/PageBlock.cpp: * wtf/PageBlock.h: * wtf/PageReservation.h: * wtf/ParallelHelperPool.cpp: * wtf/ParallelHelperPool.h: * wtf/ParallelJobs.h: * wtf/ParallelJobsLibdispatch.h: * wtf/ParallelVectorIterator.h: * wtf/ParkingLot.cpp: * wtf/ParkingLot.h: * wtf/Platform.h: * wtf/PointerComparison.h: * wtf/Poisoned.cpp: * wtf/PrintStream.cpp: * wtf/PrintStream.h: * wtf/ProcessID.h: * wtf/ProcessPrivilege.cpp: * wtf/RAMSize.cpp: * wtf/RAMSize.h: * wtf/RandomDevice.cpp: * wtf/RandomNumber.cpp: * wtf/RandomNumber.h: * wtf/RandomNumberSeed.h: * wtf/RangeSet.h: * wtf/RawPointer.h: * wtf/ReadWriteLock.cpp: * wtf/RedBlackTree.h: * wtf/Ref.h: * wtf/RefCountedArray.h: * wtf/RefCountedLeakCounter.cpp: * wtf/RefCountedLeakCounter.h: * wtf/RefCounter.h: * wtf/RefPtr.h: * wtf/RetainPtr.h: * wtf/RunLoop.cpp: * wtf/RunLoop.h: * wtf/RunLoopTimer.h: * wtf/RunLoopTimerCF.cpp: * wtf/SHA1.cpp: * wtf/SHA1.h: * wtf/SaturatedArithmetic.h: (saturatedSubtraction): * wtf/SchedulePair.h: * wtf/SchedulePairCF.cpp: * wtf/SchedulePairMac.mm: * wtf/ScopedLambda.h: * wtf/Seconds.cpp: * wtf/Seconds.h: * wtf/SegmentedVector.h: * wtf/SentinelLinkedList.h: * wtf/SharedTask.h: * wtf/SimpleStats.h: * wtf/SingleRootGraph.h: * wtf/SinglyLinkedList.h: * wtf/SixCharacterHash.cpp: * wtf/SixCharacterHash.h: * wtf/SmallPtrSet.h: * wtf/Spectrum.h: * wtf/StackBounds.cpp: * wtf/StackBounds.h: * wtf/StackStats.cpp: * wtf/StackStats.h: * wtf/StackTrace.cpp: * wtf/StdLibExtras.h: * wtf/StreamBuffer.h: * wtf/StringHashDumpContext.h: * wtf/StringPrintStream.cpp: * wtf/StringPrintStream.h: * wtf/ThreadGroup.cpp: * wtf/ThreadMessage.cpp: * wtf/ThreadSpecific.h: * wtf/Threading.cpp: * wtf/Threading.h: * wtf/ThreadingPrimitives.h: * wtf/ThreadingPthreads.cpp: * wtf/TimeWithDynamicClockType.cpp: * wtf/TimeWithDynamicClockType.h: * wtf/TimingScope.cpp: * wtf/TinyLRUCache.h: * wtf/TinyPtrSet.h: * wtf/TriState.h: * wtf/TypeCasts.h: * wtf/UUID.cpp: * wtf/UnionFind.h: * wtf/VMTags.h: * wtf/ValueCheck.h: * wtf/Vector.h: * wtf/VectorTraits.h: * wtf/WallTime.cpp: * wtf/WallTime.h: * wtf/WeakPtr.h: * wtf/WeakRandom.h: * wtf/WordLock.cpp: * wtf/WordLock.h: * wtf/WorkQueue.cpp: * wtf/WorkQueue.h: * wtf/WorkerPool.cpp: * wtf/cf/LanguageCF.cpp: * wtf/cf/RunLoopCF.cpp: * wtf/cocoa/Entitlements.mm: * wtf/cocoa/MachSendRight.cpp: * wtf/cocoa/MainThreadCocoa.mm: * wtf/cocoa/MemoryFootprintCocoa.cpp: * wtf/cocoa/WorkQueueCocoa.cpp: * wtf/dtoa.cpp: * wtf/dtoa.h: * wtf/ios/WebCoreThread.cpp: * wtf/ios/WebCoreThread.h: * wtf/mac/AppKitCompatibilityDeclarations.h: * wtf/mac/DeprecatedSymbolsUsedBySafari.mm: * wtf/mbmalloc.cpp: * wtf/persistence/PersistentCoders.cpp: * wtf/persistence/PersistentDecoder.cpp: * wtf/persistence/PersistentEncoder.cpp: * wtf/spi/cf/CFBundleSPI.h: * wtf/spi/darwin/CommonCryptoSPI.h: * wtf/text/ASCIIFastPath.h: * wtf/text/ASCIILiteral.cpp: * wtf/text/AtomicString.cpp: * wtf/text/AtomicString.h: * wtf/text/AtomicStringHash.h: * wtf/text/AtomicStringImpl.cpp: * wtf/text/AtomicStringImpl.h: * wtf/text/AtomicStringTable.cpp: * wtf/text/AtomicStringTable.h: * wtf/text/Base64.cpp: * wtf/text/CString.cpp: * wtf/text/CString.h: * wtf/text/ConversionMode.h: * wtf/text/ExternalStringImpl.cpp: * wtf/text/IntegerToStringConversion.h: * wtf/text/LChar.h: * wtf/text/LineEnding.cpp: * wtf/text/StringBuffer.h: * wtf/text/StringBuilder.cpp: * wtf/text/StringBuilder.h: * wtf/text/StringBuilderJSON.cpp: * wtf/text/StringCommon.h: * wtf/text/StringConcatenate.h: * wtf/text/StringHash.h: * wtf/text/StringImpl.cpp: * wtf/text/StringImpl.h: * wtf/text/StringOperators.h: * wtf/text/StringView.cpp: * wtf/text/StringView.h: * wtf/text/SymbolImpl.cpp: * wtf/text/SymbolRegistry.cpp: * wtf/text/SymbolRegistry.h: * wtf/text/TextBreakIterator.cpp: * wtf/text/TextBreakIterator.h: * wtf/text/TextBreakIteratorInternalICU.h: * wtf/text/TextPosition.h: * wtf/text/TextStream.cpp: * wtf/text/UniquedStringImpl.h: * wtf/text/WTFString.cpp: * wtf/text/WTFString.h: * wtf/text/cocoa/StringCocoa.mm: * wtf/text/cocoa/StringViewCocoa.mm: * wtf/text/cocoa/TextBreakIteratorInternalICUCocoa.cpp: * wtf/text/icu/UTextProvider.cpp: * wtf/text/icu/UTextProvider.h: * wtf/text/icu/UTextProviderLatin1.cpp: * wtf/text/icu/UTextProviderLatin1.h: * wtf/text/icu/UTextProviderUTF16.cpp: * wtf/text/icu/UTextProviderUTF16.h: * wtf/threads/BinarySemaphore.cpp: * wtf/threads/BinarySemaphore.h: * wtf/threads/Signals.cpp: * wtf/unicode/CharacterNames.h: * wtf/unicode/Collator.h: * wtf/unicode/CollatorDefault.cpp: * wtf/unicode/UTF8.cpp: * wtf/unicode/UTF8.h: Tools: Put WorkQueue in namespace DRT so it does not conflict with WTF::WorkQueue. * DumpRenderTree/TestRunner.cpp: (TestRunner::queueLoadHTMLString): (TestRunner::queueLoadAlternateHTMLString): (TestRunner::queueBackNavigation): (TestRunner::queueForwardNavigation): (TestRunner::queueLoadingScript): (TestRunner::queueNonLoadingScript): (TestRunner::queueReload): * DumpRenderTree/WorkQueue.cpp: (WorkQueue::singleton): Deleted. (WorkQueue::WorkQueue): Deleted. (WorkQueue::queue): Deleted. (WorkQueue::dequeue): Deleted. (WorkQueue::count): Deleted. (WorkQueue::clear): Deleted. (WorkQueue::processWork): Deleted. * DumpRenderTree/WorkQueue.h: (WorkQueue::setFrozen): Deleted. * DumpRenderTree/WorkQueueItem.h: * DumpRenderTree/mac/DumpRenderTree.mm: (runTest): * DumpRenderTree/mac/FrameLoadDelegate.mm: (-[FrameLoadDelegate processWork:]): (-[FrameLoadDelegate webView:locationChangeDone:forDataSource:]): * DumpRenderTree/mac/TestRunnerMac.mm: (TestRunner::notifyDone): (TestRunner::forceImmediateCompletion): (TestRunner::queueLoad): * DumpRenderTree/win/DumpRenderTree.cpp: (runTest): * DumpRenderTree/win/FrameLoadDelegate.cpp: (FrameLoadDelegate::processWork): (FrameLoadDelegate::locationChangeDone): * DumpRenderTree/win/TestRunnerWin.cpp: (TestRunner::notifyDone): (TestRunner::forceImmediateCompletion): (TestRunner::queueLoad): Canonical link: https://commits.webkit.org/205473@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@237099 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2018-10-15 14:24:49 +00:00
#pragma once
Implement a new allocator for backing stores https://bugs.webkit.org/show_bug.cgi?id=75181 Reviewed by Filip Pizlo. Source/JavaScriptCore: We want to move away from using fastMalloc for the backing stores for some of our objects (e.g. JSArray, JSObject, JSString, etc). These backing stores have a nice property in that they only have a single owner (i.e. a single pointer to them at any one time). One way that we can take advantage of this property is to implement a simple bump allocator/copying collector, which will run alongside our normal mark/sweep collector, that only needs to update the single owner pointer rather than having to redirect an arbitrary number of pointers in from-space to to-space. This plan can give us a number of benefits. We can beat fastMalloc in terms of both performance and memory usage, we can track how much memory we're using far more accurately than our rough estimation now through the use of reportExtraMemoryCost, and we can allocate arbitrary size objects (as opposed to being limited to size classes like we have been historically). This is also another step toward moving away from lazy destruction, which will improve our memory footprint. We start by creating said allocator and moving the ArrayStorage for JSArray to use it rather than fastMalloc. The design of the collector is as follows: Allocation: -The collector allocates 64KB chunks from the OS to use for object allocation. -Each chunk contains an offset, a flag indicating if the block has been pinned, and a payload, along with next and prev pointers so that they can be put in DoublyLinkedLists. -Any allocation greater than 64KB gets its own separate oversize block, which is managed separately from the rest. -If the allocator receives a request for more than the remaining amount in the current block, it grabs a fresh block. -Grabbing a fresh block means grabbing one off of the global free list (which is now shared between the mark/sweep allocator and the bump allocator) if there is one. If there isn't a new one we do one of two things: allocate a new block from the OS if we're not ready for a GC yet, or run a GC and then try again. If we still don't have enough space after the GC, we allocate a new block from the OS. Garbage collection: -At the start of garbage collection during conservative stack scanning, if we encounter what appears to be a pointer to a bump-allocated block of memory, we pin that block so that it will not be copied for this round of collection. -We also pin any oversize blocks that we encounter, which effectively doubles as a "mark bit" for that block. Any oversize blocks that aren't pinned at the end of copying are given back to the OS. -Marking threads are now also responsible for copying bump-allocated objects to newSpace -Each marking thread has a private 64KB block into which it copies bump-allocated objects that it encounters. -When that block fills up, the marking thread gives it back to the allocator and requests a new one. -When all marking has concluded, each thread gives back its copy block, even if it isn't full. -At the conclusion of copying (which is done by the end of the marking phase), we un-pin any pinned blocks and give any blocks left in from-space to the global free list. * CMakeLists.txt: * GNUmakefile.list.am: * JavaScriptCore.gypi: * JavaScriptCore.vcproj/JavaScriptCore/JavaScriptCore.vcproj: * JavaScriptCore.vcproj/WTF/WTF.vcproj: * JavaScriptCore.xcodeproj/project.pbxproj: * Target.pri: * heap/AllocationSpace.cpp: (JSC::AllocationSpace::allocateSlowCase): (JSC::AllocationSpace::allocateBlock): (JSC::AllocationSpace::freeBlocks): * heap/AllocationSpace.h: (JSC::AllocationSpace::waterMark): * heap/BumpBlock.h: Added. (JSC::BumpBlock::BumpBlock): * heap/BumpSpace.cpp: Added. (JSC::BumpSpace::tryAllocateSlowCase): * heap/BumpSpace.h: Added. (JSC::BumpSpace::isInCopyPhase): (JSC::BumpSpace::totalMemoryAllocated): (JSC::BumpSpace::totalMemoryUtilized): * heap/BumpSpaceInlineMethods.h: Added. (JSC::BumpSpace::BumpSpace): (JSC::BumpSpace::init): (JSC::BumpSpace::contains): (JSC::BumpSpace::pin): (JSC::BumpSpace::startedCopying): (JSC::BumpSpace::doneCopying): (JSC::BumpSpace::doneFillingBlock): (JSC::BumpSpace::recycleBlock): (JSC::BumpSpace::getFreshBlock): (JSC::BumpSpace::borrowBlock): (JSC::BumpSpace::addNewBlock): (JSC::BumpSpace::allocateNewBlock): (JSC::BumpSpace::fitsInBlock): (JSC::BumpSpace::fitsInCurrentBlock): (JSC::BumpSpace::tryAllocate): (JSC::BumpSpace::tryAllocateOversize): (JSC::BumpSpace::allocateFromBlock): (JSC::BumpSpace::tryReallocate): (JSC::BumpSpace::tryReallocateOversize): (JSC::BumpSpace::isOversize): (JSC::BumpSpace::isPinned): (JSC::BumpSpace::oversizeBlockFor): (JSC::BumpSpace::blockFor): * heap/ConservativeRoots.cpp: (JSC::ConservativeRoots::ConservativeRoots): (JSC::ConservativeRoots::genericAddPointer): (JSC::ConservativeRoots::add): * heap/ConservativeRoots.h: * heap/Heap.cpp: (JSC::Heap::Heap): (JSC::Heap::blockFreeingThreadMain): (JSC::Heap::reportExtraMemoryCostSlowCase): (JSC::Heap::getConservativeRegisterRoots): (JSC::Heap::markRoots): (JSC::Heap::collect): (JSC::Heap::releaseFreeBlocks): * heap/Heap.h: (JSC::Heap::waterMark): (JSC::Heap::highWaterMark): (JSC::Heap::setHighWaterMark): (JSC::Heap::tryAllocateStorage): (JSC::Heap::tryReallocateStorage): * heap/HeapBlock.h: Added. (JSC::HeapBlock::HeapBlock): * heap/MarkStack.cpp: (JSC::MarkStackThreadSharedData::MarkStackThreadSharedData): (JSC::SlotVisitor::drain): (JSC::SlotVisitor::drainFromShared): (JSC::SlotVisitor::startCopying): (JSC::SlotVisitor::allocateNewSpace): (JSC::SlotVisitor::copy): (JSC::SlotVisitor::copyAndAppend): (JSC::SlotVisitor::doneCopying): * heap/MarkStack.h: * heap/MarkedBlock.cpp: (JSC::MarkedBlock::recycle): (JSC::MarkedBlock::MarkedBlock): * heap/MarkedBlock.h: * heap/MarkedSpace.cpp: (JSC::MarkedSpace::MarkedSpace): * heap/MarkedSpace.h: (JSC::MarkedSpace::allocate): (JSC::MarkedSpace::forEachBlock): (JSC::MarkedSpace::SizeClass::resetAllocator): * heap/SlotVisitor.h: (JSC::SlotVisitor::SlotVisitor): * heap/TinyBloomFilter.h: (JSC::TinyBloomFilter::reset): * runtime/JSArray.cpp: (JSC::JSArray::JSArray): (JSC::JSArray::finishCreation): (JSC::JSArray::tryFinishCreationUninitialized): (JSC::JSArray::~JSArray): (JSC::JSArray::enterSparseMode): (JSC::JSArray::defineOwnNumericProperty): (JSC::JSArray::setLengthWritable): (JSC::JSArray::getOwnPropertySlotByIndex): (JSC::JSArray::getOwnPropertyDescriptor): (JSC::JSArray::putByIndexBeyondVectorLength): (JSC::JSArray::deletePropertyByIndex): (JSC::JSArray::getOwnPropertyNames): (JSC::JSArray::increaseVectorLength): (JSC::JSArray::unshiftCountSlowCase): (JSC::JSArray::setLength): (JSC::JSArray::pop): (JSC::JSArray::unshiftCount): (JSC::JSArray::visitChildren): (JSC::JSArray::sortNumeric): (JSC::JSArray::sort): (JSC::JSArray::compactForSorting): (JSC::JSArray::subclassData): (JSC::JSArray::setSubclassData): (JSC::JSArray::checkConsistency): * runtime/JSArray.h: (JSC::JSArray::inSparseMode): (JSC::JSArray::isLengthWritable): * wtf/CheckedBoolean.h: Added. (CheckedBoolean::CheckedBoolean): (CheckedBoolean::~CheckedBoolean): (CheckedBoolean::operator bool): * wtf/DoublyLinkedList.h: (WTF::::push): * wtf/StdLibExtras.h: (WTF::isPointerAligned): Source/JavaScriptGlue: Added forwarding header for new CheckedBoolean used in the bump allocator. * ForwardingHeaders/wtf/CheckedBoolean.h: Added. Source/WebCore: No new tests. Added forwarding header for new CheckedBoolean used in the bump allocator. * ForwardingHeaders/wtf/CheckedBoolean.h: Added. Canonical link: https://commits.webkit.org/93486@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@105442 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2012-01-19 21:49:56 +00:00
#include <wtf/Assertions.h>
class CheckedBoolean {
[WTF][JSC] Make JSC and WTF aggressively-fast-malloced https://bugs.webkit.org/show_bug.cgi?id=200611 Reviewed by Saam Barati. Source/JavaScriptCore: This patch aggressively puts many classes into FastMalloc. In JSC side, we grep `std::make_unique` etc. to find potentially system-malloc-allocated classes. After this patch, all the JSC related allocations in JetStream2 cli is done from bmalloc. In the future, it would be nice that we add `WTF::makeUnique<T>` helper function and throw a compile error if `T` is not FastMalloc annotated[1]. Putting WebKit classes in FastMalloc has many benefits. 1. Simply, it is fast. 2. vmmap can tell the amount of memory used for WebKit. 3. bmalloc can isolate WebKit memory allocation from the rest of the world. This is useful since we can know more about what component is corrupting the memory from the memory corruption crash. [1]: https://bugs.webkit.org/show_bug.cgi?id=200620 * API/ObjCCallbackFunction.mm: * assembler/AbstractMacroAssembler.h: * b3/B3PhiChildren.h: * b3/air/AirAllocateRegistersAndStackAndGenerateCode.h: * b3/air/AirDisassembler.h: * bytecode/AccessCaseSnippetParams.h: * bytecode/CallVariant.h: * bytecode/DeferredSourceDump.h: * bytecode/ExecutionCounter.h: * bytecode/GetByIdStatus.h: * bytecode/GetByIdVariant.h: * bytecode/InByIdStatus.h: * bytecode/InByIdVariant.h: * bytecode/InstanceOfStatus.h: * bytecode/InstanceOfVariant.h: * bytecode/PutByIdStatus.h: * bytecode/PutByIdVariant.h: * bytecode/ValueProfile.h: * dfg/DFGAbstractInterpreter.h: * dfg/DFGByteCodeParser.cpp: (JSC::DFG::ByteCodeParser::newVariableAccessData): * dfg/DFGFlowIndexing.h: * dfg/DFGFlowMap.h: * dfg/DFGLiveCatchVariablePreservationPhase.cpp: (JSC::DFG::LiveCatchVariablePreservationPhase::newVariableAccessData): * dfg/DFGMaximalFlushInsertionPhase.cpp: (JSC::DFG::MaximalFlushInsertionPhase::newVariableAccessData): * dfg/DFGOSRExit.h: * dfg/DFGSpeculativeJIT.h: * dfg/DFGVariableAccessData.h: * disassembler/ARM64/A64DOpcode.h: * inspector/remote/socket/RemoteInspectorMessageParser.h: * inspector/remote/socket/RemoteInspectorSocket.h: * inspector/remote/socket/RemoteInspectorSocketEndpoint.h: * jit/PCToCodeOriginMap.h: * runtime/BasicBlockLocation.h: * runtime/DoublePredictionFuzzerAgent.h: * runtime/JSRunLoopTimer.h: * runtime/PromiseDeferredTimer.h: (JSC::PromiseDeferredTimer::create): PromiseDeferredTimer should be allocated as `Ref<>` instead of `std::unique_ptr` since it is inheriting ThreadSafeRefCounted<>. Holding such a class with std::unique_ptr could lead to potentially dangerous operations (like, someone holds it with Ref<> while it is deleted by std::unique_ptr<>). * runtime/RandomizingFuzzerAgent.h: * runtime/SymbolTable.h: * runtime/VM.cpp: (JSC::VM::VM): * runtime/VM.h: * tools/JSDollarVM.cpp: * tools/SigillCrashAnalyzer.cpp: * wasm/WasmFormat.h: * wasm/WasmMemory.cpp: * wasm/WasmSignature.h: * yarr/YarrJIT.h: Source/WebCore: Changed the accessor since we changed std::unique_ptr to Ref for this field. No behavior change. * bindings/js/WorkerScriptController.cpp: (WebCore::WorkerScriptController::addTimerSetNotification): (WebCore::WorkerScriptController::removeTimerSetNotification): Source/WTF: WTF has many data structures, in particular, containers. And these containers can be allocated like `std::make_unique<Container>()`. Without WTF_MAKE_FAST_ALLOCATED, this container itself is allocated from the system malloc. This patch attaches WTF_MAKE_FAST_ALLOCATED more aggressively not to allocate them from the system malloc. And we add some `final` to containers and classes that would be never inherited. * wtf/Assertions.cpp: * wtf/Atomics.h: * wtf/AutodrainedPool.h: * wtf/Bag.h: (WTF::Bag::Bag): Deleted. (WTF::Bag::~Bag): Deleted. (WTF::Bag::clear): Deleted. (WTF::Bag::add): Deleted. (WTF::Bag::iterator::iterator): Deleted. (WTF::Bag::iterator::operator! const): Deleted. (WTF::Bag::iterator::operator* const): Deleted. (WTF::Bag::iterator::operator++): Deleted. (WTF::Bag::iterator::operator== const): Deleted. (WTF::Bag::iterator::operator!= const): Deleted. (WTF::Bag::begin): Deleted. (WTF::Bag::begin const): Deleted. (WTF::Bag::end const): Deleted. (WTF::Bag::isEmpty const): Deleted. (WTF::Bag::unwrappedHead const): Deleted. * wtf/BitVector.h: (WTF::BitVector::BitVector): Deleted. (WTF::BitVector::~BitVector): Deleted. (WTF::BitVector::operator=): Deleted. (WTF::BitVector::size const): Deleted. (WTF::BitVector::ensureSize): Deleted. (WTF::BitVector::quickGet const): Deleted. (WTF::BitVector::quickSet): Deleted. (WTF::BitVector::quickClear): Deleted. (WTF::BitVector::get const): Deleted. (WTF::BitVector::contains const): Deleted. (WTF::BitVector::set): Deleted. (WTF::BitVector::add): Deleted. (WTF::BitVector::ensureSizeAndSet): Deleted. (WTF::BitVector::clear): Deleted. (WTF::BitVector::remove): Deleted. (WTF::BitVector::merge): Deleted. (WTF::BitVector::filter): Deleted. (WTF::BitVector::exclude): Deleted. (WTF::BitVector::bitCount const): Deleted. (WTF::BitVector::isEmpty const): Deleted. (WTF::BitVector::findBit const): Deleted. (WTF::BitVector::isEmptyValue const): Deleted. (WTF::BitVector::isDeletedValue const): Deleted. (WTF::BitVector::isEmptyOrDeletedValue const): Deleted. (WTF::BitVector::operator== const): Deleted. (WTF::BitVector::hash const): Deleted. (WTF::BitVector::iterator::iterator): Deleted. (WTF::BitVector::iterator::operator* const): Deleted. (WTF::BitVector::iterator::operator++): Deleted. (WTF::BitVector::iterator::isAtEnd const): Deleted. (WTF::BitVector::iterator::operator== const): Deleted. (WTF::BitVector::iterator::operator!= const): Deleted. (WTF::BitVector::begin const): Deleted. (WTF::BitVector::end const): Deleted. (WTF::BitVector::bitsInPointer): Deleted. (WTF::BitVector::maxInlineBits): Deleted. (WTF::BitVector::byteCount): Deleted. (WTF::BitVector::makeInlineBits): Deleted. (WTF::BitVector::cleanseInlineBits): Deleted. (WTF::BitVector::bitCount): Deleted. (WTF::BitVector::findBitFast const): Deleted. (WTF::BitVector::findBitSimple const): Deleted. (WTF::BitVector::OutOfLineBits::numBits const): Deleted. (WTF::BitVector::OutOfLineBits::numWords const): Deleted. (WTF::BitVector::OutOfLineBits::bits): Deleted. (WTF::BitVector::OutOfLineBits::bits const): Deleted. (WTF::BitVector::OutOfLineBits::OutOfLineBits): Deleted. (WTF::BitVector::isInline const): Deleted. (WTF::BitVector::outOfLineBits const): Deleted. (WTF::BitVector::outOfLineBits): Deleted. (WTF::BitVector::bits): Deleted. (WTF::BitVector::bits const): Deleted. * wtf/Bitmap.h: (WTF::Bitmap::size): Deleted. (WTF::Bitmap::iterator::iterator): Deleted. (WTF::Bitmap::iterator::operator* const): Deleted. (WTF::Bitmap::iterator::operator++): Deleted. (WTF::Bitmap::iterator::operator== const): Deleted. (WTF::Bitmap::iterator::operator!= const): Deleted. (WTF::Bitmap::begin const): Deleted. (WTF::Bitmap::end const): Deleted. * wtf/Box.h: * wtf/BumpPointerAllocator.h: * wtf/CPUTime.h: * wtf/CheckedBoolean.h: * wtf/CommaPrinter.h: (WTF::CommaPrinter::CommaPrinter): Deleted. (WTF::CommaPrinter::dump const): Deleted. (WTF::CommaPrinter::didPrint const): Deleted. * wtf/CompactPointerTuple.h: (WTF::CompactPointerTuple::encodeType): Deleted. (WTF::CompactPointerTuple::decodeType): Deleted. (WTF::CompactPointerTuple::CompactPointerTuple): Deleted. (WTF::CompactPointerTuple::pointer const): Deleted. (WTF::CompactPointerTuple::setPointer): Deleted. (WTF::CompactPointerTuple::type const): Deleted. (WTF::CompactPointerTuple::setType): Deleted. * wtf/CompilationThread.h: (WTF::CompilationScope::CompilationScope): Deleted. (WTF::CompilationScope::~CompilationScope): Deleted. (WTF::CompilationScope::leaveEarly): Deleted. * wtf/CompletionHandler.h: (WTF::CompletionHandler<Out): (WTF::Detail::CallableWrapper<CompletionHandler<Out): (WTF::CompletionHandlerCallingScope::CompletionHandlerCallingScope): Deleted. (WTF::CompletionHandlerCallingScope::~CompletionHandlerCallingScope): Deleted. (WTF::CompletionHandlerCallingScope::CompletionHandler<void): Deleted. * wtf/ConcurrentBuffer.h: (WTF::ConcurrentBuffer::ConcurrentBuffer): Deleted. (WTF::ConcurrentBuffer::~ConcurrentBuffer): Deleted. (WTF::ConcurrentBuffer::growExact): Deleted. (WTF::ConcurrentBuffer::grow): Deleted. (WTF::ConcurrentBuffer::array const): Deleted. (WTF::ConcurrentBuffer::operator[]): Deleted. (WTF::ConcurrentBuffer::operator[] const): Deleted. (WTF::ConcurrentBuffer::createArray): Deleted. * wtf/ConcurrentPtrHashSet.h: (WTF::ConcurrentPtrHashSet::contains): Deleted. (WTF::ConcurrentPtrHashSet::add): Deleted. (WTF::ConcurrentPtrHashSet::size const): Deleted. (WTF::ConcurrentPtrHashSet::Table::maxLoad const): Deleted. (WTF::ConcurrentPtrHashSet::hash): Deleted. (WTF::ConcurrentPtrHashSet::cast): Deleted. (WTF::ConcurrentPtrHashSet::containsImpl const): Deleted. (WTF::ConcurrentPtrHashSet::addImpl): Deleted. * wtf/ConcurrentVector.h: (WTF::ConcurrentVector::~ConcurrentVector): Deleted. (WTF::ConcurrentVector::size const): Deleted. (WTF::ConcurrentVector::isEmpty const): Deleted. (WTF::ConcurrentVector::at): Deleted. (WTF::ConcurrentVector::at const): Deleted. (WTF::ConcurrentVector::operator[]): Deleted. (WTF::ConcurrentVector::operator[] const): Deleted. (WTF::ConcurrentVector::first): Deleted. (WTF::ConcurrentVector::first const): Deleted. (WTF::ConcurrentVector::last): Deleted. (WTF::ConcurrentVector::last const): Deleted. (WTF::ConcurrentVector::takeLast): Deleted. (WTF::ConcurrentVector::append): Deleted. (WTF::ConcurrentVector::alloc): Deleted. (WTF::ConcurrentVector::removeLast): Deleted. (WTF::ConcurrentVector::grow): Deleted. (WTF::ConcurrentVector::begin): Deleted. (WTF::ConcurrentVector::end): Deleted. (WTF::ConcurrentVector::segmentExistsFor): Deleted. (WTF::ConcurrentVector::segmentFor): Deleted. (WTF::ConcurrentVector::subscriptFor): Deleted. (WTF::ConcurrentVector::ensureSegmentsFor): Deleted. (WTF::ConcurrentVector::ensureSegment): Deleted. (WTF::ConcurrentVector::allocateSegment): Deleted. * wtf/Condition.h: (WTF::Condition::waitUntil): Deleted. (WTF::Condition::waitFor): Deleted. (WTF::Condition::wait): Deleted. (WTF::Condition::notifyOne): Deleted. (WTF::Condition::notifyAll): Deleted. * wtf/CountingLock.h: (WTF::CountingLock::LockHooks::lockHook): Deleted. (WTF::CountingLock::LockHooks::unlockHook): Deleted. (WTF::CountingLock::LockHooks::parkHook): Deleted. (WTF::CountingLock::LockHooks::handoffHook): Deleted. (WTF::CountingLock::tryLock): Deleted. (WTF::CountingLock::lock): Deleted. (WTF::CountingLock::unlock): Deleted. (WTF::CountingLock::isHeld const): Deleted. (WTF::CountingLock::isLocked const): Deleted. (WTF::CountingLock::Count::operator bool const): Deleted. (WTF::CountingLock::Count::operator== const): Deleted. (WTF::CountingLock::Count::operator!= const): Deleted. (WTF::CountingLock::tryOptimisticRead): Deleted. (WTF::CountingLock::validate): Deleted. (WTF::CountingLock::doOptimizedRead): Deleted. (WTF::CountingLock::tryOptimisticFencelessRead): Deleted. (WTF::CountingLock::fencelessValidate): Deleted. (WTF::CountingLock::doOptimizedFencelessRead): Deleted. (WTF::CountingLock::getCount): Deleted. * wtf/CrossThreadQueue.h: * wtf/CrossThreadTask.h: * wtf/CryptographicallyRandomNumber.cpp: * wtf/DataMutex.h: * wtf/DateMath.h: * wtf/Deque.h: (WTF::Deque::size const): Deleted. (WTF::Deque::isEmpty const): Deleted. (WTF::Deque::begin): Deleted. (WTF::Deque::end): Deleted. (WTF::Deque::begin const): Deleted. (WTF::Deque::end const): Deleted. (WTF::Deque::rbegin): Deleted. (WTF::Deque::rend): Deleted. (WTF::Deque::rbegin const): Deleted. (WTF::Deque::rend const): Deleted. (WTF::Deque::first): Deleted. (WTF::Deque::first const): Deleted. (WTF::Deque::last): Deleted. (WTF::Deque::last const): Deleted. (WTF::Deque::append): Deleted. * wtf/Dominators.h: * wtf/DoublyLinkedList.h: * wtf/Expected.h: * wtf/FastBitVector.h: * wtf/FileMetadata.h: * wtf/FileSystem.h: * wtf/GraphNodeWorklist.h: * wtf/GregorianDateTime.h: (WTF::GregorianDateTime::GregorianDateTime): Deleted. (WTF::GregorianDateTime::year const): Deleted. (WTF::GregorianDateTime::month const): Deleted. (WTF::GregorianDateTime::yearDay const): Deleted. (WTF::GregorianDateTime::monthDay const): Deleted. (WTF::GregorianDateTime::weekDay const): Deleted. (WTF::GregorianDateTime::hour const): Deleted. (WTF::GregorianDateTime::minute const): Deleted. (WTF::GregorianDateTime::second const): Deleted. (WTF::GregorianDateTime::utcOffset const): Deleted. (WTF::GregorianDateTime::isDST const): Deleted. (WTF::GregorianDateTime::setYear): Deleted. (WTF::GregorianDateTime::setMonth): Deleted. (WTF::GregorianDateTime::setYearDay): Deleted. (WTF::GregorianDateTime::setMonthDay): Deleted. (WTF::GregorianDateTime::setWeekDay): Deleted. (WTF::GregorianDateTime::setHour): Deleted. (WTF::GregorianDateTime::setMinute): Deleted. (WTF::GregorianDateTime::setSecond): Deleted. (WTF::GregorianDateTime::setUtcOffset): Deleted. (WTF::GregorianDateTime::setIsDST): Deleted. (WTF::GregorianDateTime::operator tm const): Deleted. (WTF::GregorianDateTime::copyFrom): Deleted. * wtf/HashTable.h: * wtf/Hasher.h: * wtf/HexNumber.h: * wtf/Indenter.h: * wtf/IndexMap.h: * wtf/IndexSet.h: * wtf/IndexSparseSet.h: * wtf/IndexedContainerIterator.h: * wtf/Insertion.h: * wtf/IteratorAdaptors.h: * wtf/IteratorRange.h: * wtf/KeyValuePair.h: * wtf/ListHashSet.h: (WTF::ListHashSet::begin): Deleted. (WTF::ListHashSet::end): Deleted. (WTF::ListHashSet::begin const): Deleted. (WTF::ListHashSet::end const): Deleted. (WTF::ListHashSet::random): Deleted. (WTF::ListHashSet::random const): Deleted. (WTF::ListHashSet::rbegin): Deleted. (WTF::ListHashSet::rend): Deleted. (WTF::ListHashSet::rbegin const): Deleted. (WTF::ListHashSet::rend const): Deleted. * wtf/Liveness.h: * wtf/LocklessBag.h: (WTF::LocklessBag::LocklessBag): Deleted. (WTF::LocklessBag::add): Deleted. (WTF::LocklessBag::iterate): Deleted. (WTF::LocklessBag::consumeAll): Deleted. (WTF::LocklessBag::consumeAllWithNode): Deleted. (WTF::LocklessBag::~LocklessBag): Deleted. * wtf/LoggingHashID.h: * wtf/MD5.h: * wtf/MachSendRight.h: * wtf/MainThreadData.h: * wtf/Markable.h: * wtf/MediaTime.h: * wtf/MemoryPressureHandler.h: * wtf/MessageQueue.h: (WTF::MessageQueue::MessageQueue): Deleted. * wtf/MetaAllocator.h: * wtf/MonotonicTime.h: (WTF::MonotonicTime::MonotonicTime): Deleted. (WTF::MonotonicTime::fromRawSeconds): Deleted. (WTF::MonotonicTime::infinity): Deleted. (WTF::MonotonicTime::nan): Deleted. (WTF::MonotonicTime::secondsSinceEpoch const): Deleted. (WTF::MonotonicTime::approximateMonotonicTime const): Deleted. (WTF::MonotonicTime::operator bool const): Deleted. (WTF::MonotonicTime::operator+ const): Deleted. (WTF::MonotonicTime::operator- const): Deleted. (WTF::MonotonicTime::operator% const): Deleted. (WTF::MonotonicTime::operator+=): Deleted. (WTF::MonotonicTime::operator-=): Deleted. (WTF::MonotonicTime::operator== const): Deleted. (WTF::MonotonicTime::operator!= const): Deleted. (WTF::MonotonicTime::operator< const): Deleted. (WTF::MonotonicTime::operator> const): Deleted. (WTF::MonotonicTime::operator<= const): Deleted. (WTF::MonotonicTime::operator>= const): Deleted. (WTF::MonotonicTime::isolatedCopy const): Deleted. (WTF::MonotonicTime::encode const): Deleted. (WTF::MonotonicTime::decode): Deleted. * wtf/NaturalLoops.h: * wtf/NoLock.h: * wtf/OSAllocator.h: * wtf/OptionSet.h: * wtf/Optional.h: * wtf/OrderMaker.h: * wtf/Packed.h: (WTF::alignof): * wtf/PackedIntVector.h: (WTF::PackedIntVector::PackedIntVector): Deleted. (WTF::PackedIntVector::operator=): Deleted. (WTF::PackedIntVector::size const): Deleted. (WTF::PackedIntVector::ensureSize): Deleted. (WTF::PackedIntVector::resize): Deleted. (WTF::PackedIntVector::clearAll): Deleted. (WTF::PackedIntVector::get const): Deleted. (WTF::PackedIntVector::set): Deleted. (WTF::PackedIntVector::mask): Deleted. * wtf/PageBlock.h: * wtf/ParallelJobsOpenMP.h: * wtf/ParkingLot.h: * wtf/PriorityQueue.h: (WTF::PriorityQueue::size const): Deleted. (WTF::PriorityQueue::isEmpty const): Deleted. (WTF::PriorityQueue::enqueue): Deleted. (WTF::PriorityQueue::peek const): Deleted. (WTF::PriorityQueue::dequeue): Deleted. (WTF::PriorityQueue::decreaseKey): Deleted. (WTF::PriorityQueue::increaseKey): Deleted. (WTF::PriorityQueue::begin const): Deleted. (WTF::PriorityQueue::end const): Deleted. (WTF::PriorityQueue::isValidHeap const): Deleted. (WTF::PriorityQueue::parentOf): Deleted. (WTF::PriorityQueue::leftChildOf): Deleted. (WTF::PriorityQueue::rightChildOf): Deleted. (WTF::PriorityQueue::siftUp): Deleted. (WTF::PriorityQueue::siftDown): Deleted. * wtf/RandomDevice.h: * wtf/Range.h: * wtf/RangeSet.h: (WTF::RangeSet::RangeSet): Deleted. (WTF::RangeSet::~RangeSet): Deleted. (WTF::RangeSet::add): Deleted. (WTF::RangeSet::contains const): Deleted. (WTF::RangeSet::overlaps const): Deleted. (WTF::RangeSet::clear): Deleted. (WTF::RangeSet::dump const): Deleted. (WTF::RangeSet::dumpRaw const): Deleted. (WTF::RangeSet::begin const): Deleted. (WTF::RangeSet::end const): Deleted. (WTF::RangeSet::addAll): Deleted. (WTF::RangeSet::compact): Deleted. (WTF::RangeSet::overlapsNonEmpty): Deleted. (WTF::RangeSet::subsumesNonEmpty): Deleted. (WTF::RangeSet::findRange const): Deleted. * wtf/RecursableLambda.h: * wtf/RedBlackTree.h: (WTF::RedBlackTree::Node::successor const): Deleted. (WTF::RedBlackTree::Node::predecessor const): Deleted. (WTF::RedBlackTree::Node::successor): Deleted. (WTF::RedBlackTree::Node::predecessor): Deleted. (WTF::RedBlackTree::Node::reset): Deleted. (WTF::RedBlackTree::Node::parent const): Deleted. (WTF::RedBlackTree::Node::setParent): Deleted. (WTF::RedBlackTree::Node::left const): Deleted. (WTF::RedBlackTree::Node::setLeft): Deleted. (WTF::RedBlackTree::Node::right const): Deleted. (WTF::RedBlackTree::Node::setRight): Deleted. (WTF::RedBlackTree::Node::color const): Deleted. (WTF::RedBlackTree::Node::setColor): Deleted. (WTF::RedBlackTree::RedBlackTree): Deleted. (WTF::RedBlackTree::insert): Deleted. (WTF::RedBlackTree::remove): Deleted. (WTF::RedBlackTree::findExact const): Deleted. (WTF::RedBlackTree::findLeastGreaterThanOrEqual const): Deleted. (WTF::RedBlackTree::findGreatestLessThanOrEqual const): Deleted. (WTF::RedBlackTree::first const): Deleted. (WTF::RedBlackTree::last const): Deleted. (WTF::RedBlackTree::size): Deleted. (WTF::RedBlackTree::isEmpty): Deleted. (WTF::RedBlackTree::treeMinimum): Deleted. (WTF::RedBlackTree::treeMaximum): Deleted. (WTF::RedBlackTree::treeInsert): Deleted. (WTF::RedBlackTree::leftRotate): Deleted. (WTF::RedBlackTree::rightRotate): Deleted. (WTF::RedBlackTree::removeFixup): Deleted. * wtf/ResourceUsage.h: * wtf/RunLoop.cpp: * wtf/RunLoopTimer.h: * wtf/SHA1.h: * wtf/Seconds.h: (WTF::Seconds::Seconds): Deleted. (WTF::Seconds::value const): Deleted. (WTF::Seconds::minutes const): Deleted. (WTF::Seconds::seconds const): Deleted. (WTF::Seconds::milliseconds const): Deleted. (WTF::Seconds::microseconds const): Deleted. (WTF::Seconds::nanoseconds const): Deleted. (WTF::Seconds::minutesAs const): Deleted. (WTF::Seconds::secondsAs const): Deleted. (WTF::Seconds::millisecondsAs const): Deleted. (WTF::Seconds::microsecondsAs const): Deleted. (WTF::Seconds::nanosecondsAs const): Deleted. (WTF::Seconds::fromMinutes): Deleted. (WTF::Seconds::fromHours): Deleted. (WTF::Seconds::fromMilliseconds): Deleted. (WTF::Seconds::fromMicroseconds): Deleted. (WTF::Seconds::fromNanoseconds): Deleted. (WTF::Seconds::infinity): Deleted. (WTF::Seconds::nan): Deleted. (WTF::Seconds::operator bool const): Deleted. (WTF::Seconds::operator+ const): Deleted. (WTF::Seconds::operator- const): Deleted. (WTF::Seconds::operator* const): Deleted. (WTF::Seconds::operator/ const): Deleted. (WTF::Seconds::operator% const): Deleted. (WTF::Seconds::operator+=): Deleted. (WTF::Seconds::operator-=): Deleted. (WTF::Seconds::operator*=): Deleted. (WTF::Seconds::operator/=): Deleted. (WTF::Seconds::operator%=): Deleted. (WTF::Seconds::operator== const): Deleted. (WTF::Seconds::operator!= const): Deleted. (WTF::Seconds::operator< const): Deleted. (WTF::Seconds::operator> const): Deleted. (WTF::Seconds::operator<= const): Deleted. (WTF::Seconds::operator>= const): Deleted. (WTF::Seconds::isolatedCopy const): Deleted. (WTF::Seconds::encode const): Deleted. (WTF::Seconds::decode): Deleted. * wtf/SegmentedVector.h: (WTF::SegmentedVector::~SegmentedVector): Deleted. (WTF::SegmentedVector::size const): Deleted. (WTF::SegmentedVector::isEmpty const): Deleted. (WTF::SegmentedVector::at): Deleted. (WTF::SegmentedVector::at const): Deleted. (WTF::SegmentedVector::operator[]): Deleted. (WTF::SegmentedVector::operator[] const): Deleted. (WTF::SegmentedVector::first): Deleted. (WTF::SegmentedVector::first const): Deleted. (WTF::SegmentedVector::last): Deleted. (WTF::SegmentedVector::last const): Deleted. (WTF::SegmentedVector::takeLast): Deleted. (WTF::SegmentedVector::append): Deleted. (WTF::SegmentedVector::alloc): Deleted. (WTF::SegmentedVector::removeLast): Deleted. (WTF::SegmentedVector::grow): Deleted. (WTF::SegmentedVector::clear): Deleted. (WTF::SegmentedVector::begin): Deleted. (WTF::SegmentedVector::end): Deleted. (WTF::SegmentedVector::shrinkToFit): Deleted. (WTF::SegmentedVector::deleteAllSegments): Deleted. (WTF::SegmentedVector::segmentExistsFor): Deleted. (WTF::SegmentedVector::segmentFor): Deleted. (WTF::SegmentedVector::subscriptFor): Deleted. (WTF::SegmentedVector::ensureSegmentsFor): Deleted. (WTF::SegmentedVector::ensureSegment): Deleted. (WTF::SegmentedVector::allocateSegment): Deleted. * wtf/SetForScope.h: * wtf/SingleRootGraph.h: * wtf/SinglyLinkedList.h: * wtf/SmallPtrSet.h: * wtf/SpanningTree.h: * wtf/Spectrum.h: * wtf/StackBounds.h: * wtf/StackShot.h: * wtf/StackShotProfiler.h: * wtf/StackStats.h: * wtf/StackTrace.h: * wtf/StreamBuffer.h: * wtf/SynchronizedFixedQueue.h: (WTF::SynchronizedFixedQueue::create): Deleted. (WTF::SynchronizedFixedQueue::open): Deleted. (WTF::SynchronizedFixedQueue::close): Deleted. (WTF::SynchronizedFixedQueue::isOpen): Deleted. (WTF::SynchronizedFixedQueue::enqueue): Deleted. (WTF::SynchronizedFixedQueue::dequeue): Deleted. (WTF::SynchronizedFixedQueue::SynchronizedFixedQueue): Deleted. * wtf/SystemTracing.h: * wtf/ThreadGroup.h: (WTF::ThreadGroup::create): Deleted. (WTF::ThreadGroup::threads const): Deleted. (WTF::ThreadGroup::getLock): Deleted. (WTF::ThreadGroup::weakFromThis): Deleted. * wtf/ThreadSpecific.h: * wtf/ThreadingPrimitives.h: (WTF::Mutex::impl): Deleted. * wtf/TimeWithDynamicClockType.h: (WTF::TimeWithDynamicClockType::TimeWithDynamicClockType): Deleted. (WTF::TimeWithDynamicClockType::fromRawSeconds): Deleted. (WTF::TimeWithDynamicClockType::secondsSinceEpoch const): Deleted. (WTF::TimeWithDynamicClockType::clockType const): Deleted. (WTF::TimeWithDynamicClockType::withSameClockAndRawSeconds const): Deleted. (WTF::TimeWithDynamicClockType::operator bool const): Deleted. (WTF::TimeWithDynamicClockType::operator+ const): Deleted. (WTF::TimeWithDynamicClockType::operator- const): Deleted. (WTF::TimeWithDynamicClockType::operator+=): Deleted. (WTF::TimeWithDynamicClockType::operator-=): Deleted. (WTF::TimeWithDynamicClockType::operator== const): Deleted. (WTF::TimeWithDynamicClockType::operator!= const): Deleted. * wtf/TimingScope.h: * wtf/TinyLRUCache.h: * wtf/TinyPtrSet.h: * wtf/URLParser.cpp: * wtf/URLParser.h: * wtf/Unexpected.h: * wtf/Variant.h: * wtf/WTFSemaphore.h: (WTF::Semaphore::Semaphore): Deleted. (WTF::Semaphore::signal): Deleted. (WTF::Semaphore::waitUntil): Deleted. (WTF::Semaphore::waitFor): Deleted. (WTF::Semaphore::wait): Deleted. * wtf/WallTime.h: (WTF::WallTime::WallTime): Deleted. (WTF::WallTime::fromRawSeconds): Deleted. (WTF::WallTime::infinity): Deleted. (WTF::WallTime::nan): Deleted. (WTF::WallTime::secondsSinceEpoch const): Deleted. (WTF::WallTime::approximateWallTime const): Deleted. (WTF::WallTime::operator bool const): Deleted. (WTF::WallTime::operator+ const): Deleted. (WTF::WallTime::operator- const): Deleted. (WTF::WallTime::operator+=): Deleted. (WTF::WallTime::operator-=): Deleted. (WTF::WallTime::operator== const): Deleted. (WTF::WallTime::operator!= const): Deleted. (WTF::WallTime::operator< const): Deleted. (WTF::WallTime::operator> const): Deleted. (WTF::WallTime::operator<= const): Deleted. (WTF::WallTime::operator>= const): Deleted. (WTF::WallTime::isolatedCopy const): Deleted. * wtf/WeakHashSet.h: (WTF::WeakHashSet::WeakHashSetConstIterator::WeakHashSetConstIterator): Deleted. (WTF::WeakHashSet::WeakHashSetConstIterator::get const): Deleted. (WTF::WeakHashSet::WeakHashSetConstIterator::operator* const): Deleted. (WTF::WeakHashSet::WeakHashSetConstIterator::operator-> const): Deleted. (WTF::WeakHashSet::WeakHashSetConstIterator::operator++): Deleted. (WTF::WeakHashSet::WeakHashSetConstIterator::skipEmptyBuckets): Deleted. (WTF::WeakHashSet::WeakHashSetConstIterator::operator== const): Deleted. (WTF::WeakHashSet::WeakHashSetConstIterator::operator!= const): Deleted. (WTF::WeakHashSet::WeakHashSet): Deleted. (WTF::WeakHashSet::begin const): Deleted. (WTF::WeakHashSet::end const): Deleted. (WTF::WeakHashSet::add): Deleted. (WTF::WeakHashSet::remove): Deleted. (WTF::WeakHashSet::contains const): Deleted. (WTF::WeakHashSet::capacity const): Deleted. (WTF::WeakHashSet::computesEmpty const): Deleted. (WTF::WeakHashSet::hasNullReferences const): Deleted. (WTF::WeakHashSet::computeSize const): Deleted. (WTF::WeakHashSet::checkConsistency const): Deleted. * wtf/WeakRandom.h: (WTF::WeakRandom::WeakRandom): Deleted. (WTF::WeakRandom::setSeed): Deleted. (WTF::WeakRandom::seed const): Deleted. (WTF::WeakRandom::get): Deleted. (WTF::WeakRandom::getUint32): Deleted. (WTF::WeakRandom::lowOffset): Deleted. (WTF::WeakRandom::highOffset): Deleted. (WTF::WeakRandom::nextState): Deleted. (WTF::WeakRandom::generate): Deleted. (WTF::WeakRandom::advance): Deleted. * wtf/WordLock.h: (WTF::WordLock::lock): Deleted. (WTF::WordLock::unlock): Deleted. (WTF::WordLock::isHeld const): Deleted. (WTF::WordLock::isLocked const): Deleted. (WTF::WordLock::isFullyReset const): Deleted. * wtf/generic/MainThreadGeneric.cpp: * wtf/glib/GMutexLocker.h: * wtf/linux/CurrentProcessMemoryStatus.h: * wtf/posix/ThreadingPOSIX.cpp: (WTF::Semaphore::Semaphore): Deleted. (WTF::Semaphore::~Semaphore): Deleted. (WTF::Semaphore::wait): Deleted. (WTF::Semaphore::post): Deleted. * wtf/text/ASCIILiteral.h: (WTF::ASCIILiteral::operator const char* const): Deleted. (WTF::ASCIILiteral::fromLiteralUnsafe): Deleted. (WTF::ASCIILiteral::null): Deleted. (WTF::ASCIILiteral::characters const): Deleted. (WTF::ASCIILiteral::ASCIILiteral): Deleted. * wtf/text/AtomString.h: (WTF::AtomString::operator=): Deleted. (WTF::AtomString::isHashTableDeletedValue const): Deleted. (WTF::AtomString::existingHash const): Deleted. (WTF::AtomString::operator const String& const): Deleted. (WTF::AtomString::string const): Deleted. (WTF::AtomString::impl const): Deleted. (WTF::AtomString::is8Bit const): Deleted. (WTF::AtomString::characters8 const): Deleted. (WTF::AtomString::characters16 const): Deleted. (WTF::AtomString::length const): Deleted. (WTF::AtomString::operator[] const): Deleted. (WTF::AtomString::contains const): Deleted. (WTF::AtomString::containsIgnoringASCIICase const): Deleted. (WTF::AtomString::find const): Deleted. (WTF::AtomString::findIgnoringASCIICase const): Deleted. (WTF::AtomString::startsWith const): Deleted. (WTF::AtomString::startsWithIgnoringASCIICase const): Deleted. (WTF::AtomString::endsWith const): Deleted. (WTF::AtomString::endsWithIgnoringASCIICase const): Deleted. (WTF::AtomString::toInt const): Deleted. (WTF::AtomString::toDouble const): Deleted. (WTF::AtomString::toFloat const): Deleted. (WTF::AtomString::percentage const): Deleted. (WTF::AtomString::isNull const): Deleted. (WTF::AtomString::isEmpty const): Deleted. (WTF::AtomString::operator NSString * const): Deleted. * wtf/text/AtomStringImpl.h: (WTF::AtomStringImpl::lookUp): Deleted. (WTF::AtomStringImpl::add): Deleted. (WTF::AtomStringImpl::addWithStringTableProvider): Deleted. * wtf/text/CString.h: (WTF::CStringBuffer::data): Deleted. (WTF::CStringBuffer::length const): Deleted. (WTF::CStringBuffer::CStringBuffer): Deleted. (WTF::CStringBuffer::mutableData): Deleted. (WTF::CString::CString): Deleted. (WTF::CString::data const): Deleted. (WTF::CString::length const): Deleted. (WTF::CString::isNull const): Deleted. (WTF::CString::buffer const): Deleted. (WTF::CString::isHashTableDeletedValue const): Deleted. * wtf/text/ExternalStringImpl.h: (WTF::ExternalStringImpl::freeExternalBuffer): Deleted. * wtf/text/LineBreakIteratorPoolICU.h: * wtf/text/NullTextBreakIterator.h: * wtf/text/OrdinalNumber.h: * wtf/text/StringBuffer.h: * wtf/text/StringBuilder.h: * wtf/text/StringConcatenateNumbers.h: * wtf/text/StringHasher.h: * wtf/text/StringImpl.h: * wtf/text/StringView.cpp: * wtf/text/StringView.h: (WTF::StringView::left const): Deleted. (WTF::StringView::right const): Deleted. (WTF::StringView::underlyingStringIsValid const): Deleted. (WTF::StringView::setUnderlyingString): Deleted. * wtf/text/SymbolImpl.h: (WTF::SymbolImpl::StaticSymbolImpl::StaticSymbolImpl): Deleted. (WTF::SymbolImpl::StaticSymbolImpl::operator SymbolImpl&): Deleted. (WTF::PrivateSymbolImpl::PrivateSymbolImpl): Deleted. (WTF::RegisteredSymbolImpl::symbolRegistry const): Deleted. (WTF::RegisteredSymbolImpl::clearSymbolRegistry): Deleted. (WTF::RegisteredSymbolImpl::RegisteredSymbolImpl): Deleted. * wtf/text/SymbolRegistry.h: * wtf/text/TextBreakIterator.h: * wtf/text/TextPosition.h: * wtf/text/TextStream.h: * wtf/text/WTFString.h: (WTF::String::swap): Deleted. (WTF::String::adopt): Deleted. (WTF::String::isNull const): Deleted. (WTF::String::isEmpty const): Deleted. (WTF::String::impl const): Deleted. (WTF::String::releaseImpl): Deleted. (WTF::String::length const): Deleted. (WTF::String::characters8 const): Deleted. (WTF::String::characters16 const): Deleted. (WTF::String::is8Bit const): Deleted. (WTF::String::sizeInBytes const): Deleted. (WTF::String::operator[] const): Deleted. (WTF::String::find const): Deleted. (WTF::String::findIgnoringASCIICase const): Deleted. (WTF::String::reverseFind const): Deleted. (WTF::String::contains const): Deleted. (WTF::String::containsIgnoringASCIICase const): Deleted. (WTF::String::startsWith const): Deleted. (WTF::String::startsWithIgnoringASCIICase const): Deleted. (WTF::String::hasInfixStartingAt const): Deleted. (WTF::String::endsWith const): Deleted. (WTF::String::endsWithIgnoringASCIICase const): Deleted. (WTF::String::hasInfixEndingAt const): Deleted. (WTF::String::append): Deleted. (WTF::String::left const): Deleted. (WTF::String::right const): Deleted. (WTF::String::createUninitialized): Deleted. (WTF::String::fromUTF8WithLatin1Fallback): Deleted. (WTF::String::isAllASCII const): Deleted. (WTF::String::isAllLatin1 const): Deleted. (WTF::String::isSpecialCharacter const): Deleted. (WTF::String::isHashTableDeletedValue const): Deleted. (WTF::String::hash const): Deleted. (WTF::String::existingHash const): Deleted. * wtf/text/cf/TextBreakIteratorCF.h: * wtf/text/icu/TextBreakIteratorICU.h: * wtf/text/icu/UTextProviderLatin1.h: * wtf/threads/BinarySemaphore.h: (WTF::BinarySemaphore::waitFor): Deleted. (WTF::BinarySemaphore::wait): Deleted. * wtf/unicode/Collator.h: * wtf/win/GDIObject.h: * wtf/win/PathWalker.h: * wtf/win/Win32Handle.h: Canonical link: https://commits.webkit.org/214396@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@248546 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2019-08-12 20:57:15 +00:00
WTF_MAKE_FAST_ALLOCATED;
Implement a new allocator for backing stores https://bugs.webkit.org/show_bug.cgi?id=75181 Reviewed by Filip Pizlo. Source/JavaScriptCore: We want to move away from using fastMalloc for the backing stores for some of our objects (e.g. JSArray, JSObject, JSString, etc). These backing stores have a nice property in that they only have a single owner (i.e. a single pointer to them at any one time). One way that we can take advantage of this property is to implement a simple bump allocator/copying collector, which will run alongside our normal mark/sweep collector, that only needs to update the single owner pointer rather than having to redirect an arbitrary number of pointers in from-space to to-space. This plan can give us a number of benefits. We can beat fastMalloc in terms of both performance and memory usage, we can track how much memory we're using far more accurately than our rough estimation now through the use of reportExtraMemoryCost, and we can allocate arbitrary size objects (as opposed to being limited to size classes like we have been historically). This is also another step toward moving away from lazy destruction, which will improve our memory footprint. We start by creating said allocator and moving the ArrayStorage for JSArray to use it rather than fastMalloc. The design of the collector is as follows: Allocation: -The collector allocates 64KB chunks from the OS to use for object allocation. -Each chunk contains an offset, a flag indicating if the block has been pinned, and a payload, along with next and prev pointers so that they can be put in DoublyLinkedLists. -Any allocation greater than 64KB gets its own separate oversize block, which is managed separately from the rest. -If the allocator receives a request for more than the remaining amount in the current block, it grabs a fresh block. -Grabbing a fresh block means grabbing one off of the global free list (which is now shared between the mark/sweep allocator and the bump allocator) if there is one. If there isn't a new one we do one of two things: allocate a new block from the OS if we're not ready for a GC yet, or run a GC and then try again. If we still don't have enough space after the GC, we allocate a new block from the OS. Garbage collection: -At the start of garbage collection during conservative stack scanning, if we encounter what appears to be a pointer to a bump-allocated block of memory, we pin that block so that it will not be copied for this round of collection. -We also pin any oversize blocks that we encounter, which effectively doubles as a "mark bit" for that block. Any oversize blocks that aren't pinned at the end of copying are given back to the OS. -Marking threads are now also responsible for copying bump-allocated objects to newSpace -Each marking thread has a private 64KB block into which it copies bump-allocated objects that it encounters. -When that block fills up, the marking thread gives it back to the allocator and requests a new one. -When all marking has concluded, each thread gives back its copy block, even if it isn't full. -At the conclusion of copying (which is done by the end of the marking phase), we un-pin any pinned blocks and give any blocks left in from-space to the global free list. * CMakeLists.txt: * GNUmakefile.list.am: * JavaScriptCore.gypi: * JavaScriptCore.vcproj/JavaScriptCore/JavaScriptCore.vcproj: * JavaScriptCore.vcproj/WTF/WTF.vcproj: * JavaScriptCore.xcodeproj/project.pbxproj: * Target.pri: * heap/AllocationSpace.cpp: (JSC::AllocationSpace::allocateSlowCase): (JSC::AllocationSpace::allocateBlock): (JSC::AllocationSpace::freeBlocks): * heap/AllocationSpace.h: (JSC::AllocationSpace::waterMark): * heap/BumpBlock.h: Added. (JSC::BumpBlock::BumpBlock): * heap/BumpSpace.cpp: Added. (JSC::BumpSpace::tryAllocateSlowCase): * heap/BumpSpace.h: Added. (JSC::BumpSpace::isInCopyPhase): (JSC::BumpSpace::totalMemoryAllocated): (JSC::BumpSpace::totalMemoryUtilized): * heap/BumpSpaceInlineMethods.h: Added. (JSC::BumpSpace::BumpSpace): (JSC::BumpSpace::init): (JSC::BumpSpace::contains): (JSC::BumpSpace::pin): (JSC::BumpSpace::startedCopying): (JSC::BumpSpace::doneCopying): (JSC::BumpSpace::doneFillingBlock): (JSC::BumpSpace::recycleBlock): (JSC::BumpSpace::getFreshBlock): (JSC::BumpSpace::borrowBlock): (JSC::BumpSpace::addNewBlock): (JSC::BumpSpace::allocateNewBlock): (JSC::BumpSpace::fitsInBlock): (JSC::BumpSpace::fitsInCurrentBlock): (JSC::BumpSpace::tryAllocate): (JSC::BumpSpace::tryAllocateOversize): (JSC::BumpSpace::allocateFromBlock): (JSC::BumpSpace::tryReallocate): (JSC::BumpSpace::tryReallocateOversize): (JSC::BumpSpace::isOversize): (JSC::BumpSpace::isPinned): (JSC::BumpSpace::oversizeBlockFor): (JSC::BumpSpace::blockFor): * heap/ConservativeRoots.cpp: (JSC::ConservativeRoots::ConservativeRoots): (JSC::ConservativeRoots::genericAddPointer): (JSC::ConservativeRoots::add): * heap/ConservativeRoots.h: * heap/Heap.cpp: (JSC::Heap::Heap): (JSC::Heap::blockFreeingThreadMain): (JSC::Heap::reportExtraMemoryCostSlowCase): (JSC::Heap::getConservativeRegisterRoots): (JSC::Heap::markRoots): (JSC::Heap::collect): (JSC::Heap::releaseFreeBlocks): * heap/Heap.h: (JSC::Heap::waterMark): (JSC::Heap::highWaterMark): (JSC::Heap::setHighWaterMark): (JSC::Heap::tryAllocateStorage): (JSC::Heap::tryReallocateStorage): * heap/HeapBlock.h: Added. (JSC::HeapBlock::HeapBlock): * heap/MarkStack.cpp: (JSC::MarkStackThreadSharedData::MarkStackThreadSharedData): (JSC::SlotVisitor::drain): (JSC::SlotVisitor::drainFromShared): (JSC::SlotVisitor::startCopying): (JSC::SlotVisitor::allocateNewSpace): (JSC::SlotVisitor::copy): (JSC::SlotVisitor::copyAndAppend): (JSC::SlotVisitor::doneCopying): * heap/MarkStack.h: * heap/MarkedBlock.cpp: (JSC::MarkedBlock::recycle): (JSC::MarkedBlock::MarkedBlock): * heap/MarkedBlock.h: * heap/MarkedSpace.cpp: (JSC::MarkedSpace::MarkedSpace): * heap/MarkedSpace.h: (JSC::MarkedSpace::allocate): (JSC::MarkedSpace::forEachBlock): (JSC::MarkedSpace::SizeClass::resetAllocator): * heap/SlotVisitor.h: (JSC::SlotVisitor::SlotVisitor): * heap/TinyBloomFilter.h: (JSC::TinyBloomFilter::reset): * runtime/JSArray.cpp: (JSC::JSArray::JSArray): (JSC::JSArray::finishCreation): (JSC::JSArray::tryFinishCreationUninitialized): (JSC::JSArray::~JSArray): (JSC::JSArray::enterSparseMode): (JSC::JSArray::defineOwnNumericProperty): (JSC::JSArray::setLengthWritable): (JSC::JSArray::getOwnPropertySlotByIndex): (JSC::JSArray::getOwnPropertyDescriptor): (JSC::JSArray::putByIndexBeyondVectorLength): (JSC::JSArray::deletePropertyByIndex): (JSC::JSArray::getOwnPropertyNames): (JSC::JSArray::increaseVectorLength): (JSC::JSArray::unshiftCountSlowCase): (JSC::JSArray::setLength): (JSC::JSArray::pop): (JSC::JSArray::unshiftCount): (JSC::JSArray::visitChildren): (JSC::JSArray::sortNumeric): (JSC::JSArray::sort): (JSC::JSArray::compactForSorting): (JSC::JSArray::subclassData): (JSC::JSArray::setSubclassData): (JSC::JSArray::checkConsistency): * runtime/JSArray.h: (JSC::JSArray::inSparseMode): (JSC::JSArray::isLengthWritable): * wtf/CheckedBoolean.h: Added. (CheckedBoolean::CheckedBoolean): (CheckedBoolean::~CheckedBoolean): (CheckedBoolean::operator bool): * wtf/DoublyLinkedList.h: (WTF::::push): * wtf/StdLibExtras.h: (WTF::isPointerAligned): Source/JavaScriptGlue: Added forwarding header for new CheckedBoolean used in the bump allocator. * ForwardingHeaders/wtf/CheckedBoolean.h: Added. Source/WebCore: No new tests. Added forwarding header for new CheckedBoolean used in the bump allocator. * ForwardingHeaders/wtf/CheckedBoolean.h: Added. Canonical link: https://commits.webkit.org/93486@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@105442 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2012-01-19 21:49:56 +00:00
public:
PerformanceTests: Convert ASSERT_DISABLED to ASSERT_ENABLED, and fix some tests of NDEBUG that should actually test for ASSERT_ENABLED. https://bugs.webkit.org/show_bug.cgi?id=205776 Reviewed by Saam Barati. * JetStream2/wasm/HashSet.cpp: * StitchMarker/wtf/Assertions.h: * StitchMarker/wtf/DateMath.cpp: (WTF::initializeDates): * StitchMarker/wtf/HashTable.h: * StitchMarker/wtf/Hasher.h: (WTF::StringHasher::addCharacters): * StitchMarker/wtf/NeverDestroyed.h: (WTF::LazyNeverDestroyed::construct): * StitchMarker/wtf/StackBounds.h: (WTF::StackBounds::checkConsistency const): * StitchMarker/wtf/ValueCheck.h: * StitchMarker/wtf/Vector.h: (WTF::minCapacity>::checkConsistency): * StitchMarker/wtf/text/AtomicStringImpl.cpp: * StitchMarker/wtf/text/AtomicStringImpl.h: * StitchMarker/wtf/text/StringCommon.h: (WTF::hasPrefixWithLettersIgnoringASCIICaseCommon): * StitchMarker/wtf/text/StringImpl.h: * StitchMarker/wtf/text/SymbolImpl.h: * StitchMarker/wtf/text/UniquedStringImpl.h: Source/JavaScriptCore: Convert ASSERT_DISABLED to ASSERT_ENABLED, and fix some tests of NDEBUG that should actually test for ASSERT_ENABLED. https://bugs.webkit.org/show_bug.cgi?id=205776 Reviewed by Saam Barati. * API/tests/testapi.c: * assembler/ARM64Assembler.h: (JSC::ARM64Assembler::replaceWithLoad): (JSC::ARM64Assembler::replaceWithAddressComputation): * assembler/AssemblerBuffer.h: (JSC::AssemblerBuffer::LocalWriter::LocalWriter): * assembler/LinkBuffer.cpp: (JSC::LinkBuffer::copyCompactAndLinkCode): * assembler/ProbeStack.cpp: (JSC::Probe::Stack::Stack): * assembler/ProbeStack.h: * b3/B3FoldPathConstants.cpp: * b3/B3LowerToAir.cpp: * b3/B3MemoryValue.cpp: (JSC::B3::MemoryValue::MemoryValue): * b3/B3Opcode.cpp: * b3/B3Type.h: * b3/B3TypeMap.h: * b3/B3Width.h: * b3/air/AirAllocateRegistersAndStackAndGenerateCode.cpp: (JSC::B3::Air::GenerateAndAllocateRegisters::prepareForGeneration): (JSC::B3::Air::GenerateAndAllocateRegisters::generate): * b3/air/AirAllocateRegistersAndStackAndGenerateCode.h: * b3/air/AirAllocateRegistersByGraphColoring.cpp: * b3/air/AirArg.cpp: * b3/air/AirArg.h: * b3/air/AirCode.h: * b3/air/AirEmitShuffle.cpp: (JSC::B3::Air::emitShuffle): * builtins/BuiltinExecutables.cpp: (JSC::BuiltinExecutables::createExecutable): * bytecode/AccessCase.cpp: * bytecode/AccessCase.h: * bytecode/CallVariant.cpp: (JSC::variantListWithVariant): * bytecode/CodeBlock.cpp: (JSC::CodeBlock::ensureCatchLivenessIsComputedForBytecodeIndex): * bytecode/CodeBlockHash.cpp: (JSC::CodeBlockHash::dump const): * bytecode/StructureStubInfo.cpp: * bytecode/StructureStubInfo.h: * bytecompiler/NodesCodegen.cpp: (JSC::FunctionCallResolveNode::emitBytecode): * bytecompiler/RegisterID.h: (JSC::RegisterID::RegisterID): (JSC::RegisterID::setIndex): * debugger/Debugger.cpp: (JSC::Debugger::removeBreakpoint): * debugger/DebuggerEvalEnabler.h: (JSC::DebuggerEvalEnabler::DebuggerEvalEnabler): (JSC::DebuggerEvalEnabler::~DebuggerEvalEnabler): * dfg/DFGAbstractInterpreterInlines.h: (JSC::DFG::AbstractInterpreter<AbstractStateType>::observeTransitions): * dfg/DFGAbstractValue.cpp: * dfg/DFGAbstractValue.h: (JSC::DFG::AbstractValue::merge): (JSC::DFG::AbstractValue::checkConsistency const): (JSC::DFG::AbstractValue::assertIsRegistered const): * dfg/DFGArithMode.h: (JSC::DFG::doesOverflow): * dfg/DFGBasicBlock.cpp: (JSC::DFG::BasicBlock::BasicBlock): * dfg/DFGBasicBlock.h: (JSC::DFG::BasicBlock::didLink): * dfg/DFGCFAPhase.cpp: (JSC::DFG::CFAPhase::performBlockCFA): * dfg/DFGCommon.h: (JSC::DFG::validationEnabled): * dfg/DFGCommonData.cpp: (JSC::DFG::CommonData::finalizeCatchEntrypoints): * dfg/DFGDesiredWatchpoints.h: * dfg/DFGDoesGC.cpp: (JSC::DFG::doesGC): * dfg/DFGEdge.h: (JSC::DFG::Edge::makeWord): * dfg/DFGFixupPhase.cpp: (JSC::DFG::FixupPhase::fixupNode): * dfg/DFGJITCode.cpp: (JSC::DFG::JITCode::finalizeOSREntrypoints): * dfg/DFGObjectAllocationSinkingPhase.cpp: * dfg/DFGSSAConversionPhase.cpp: (JSC::DFG::SSAConversionPhase::run): * dfg/DFGScoreBoard.h: (JSC::DFG::ScoreBoard::assertClear): * dfg/DFGSlowPathGenerator.h: (JSC::DFG::SlowPathGenerator::generate): * dfg/DFGSpeculativeJIT.cpp: (JSC::DFG::SpeculativeJIT::compileCurrentBlock): (JSC::DFG::SpeculativeJIT::emitBinarySwitchStringRecurse): (JSC::DFG::SpeculativeJIT::emitAllocateButterfly): (JSC::DFG::SpeculativeJIT::compileAllocateNewArrayWithSize): (JSC::DFG::SpeculativeJIT::compileMakeRope): * dfg/DFGSpeculativeJIT64.cpp: (JSC::DFG::SpeculativeJIT::fillSpeculateCell): * dfg/DFGStructureAbstractValue.cpp: * dfg/DFGStructureAbstractValue.h: (JSC::DFG::StructureAbstractValue::assertIsRegistered const): * dfg/DFGVarargsForwardingPhase.cpp: * dfg/DFGVirtualRegisterAllocationPhase.cpp: (JSC::DFG::VirtualRegisterAllocationPhase::run): * ftl/FTLLink.cpp: (JSC::FTL::link): * ftl/FTLLowerDFGToB3.cpp: (JSC::FTL::DFG::LowerDFGToB3::callPreflight): (JSC::FTL::DFG::LowerDFGToB3::callCheck): (JSC::FTL::DFG::LowerDFGToB3::crash): * ftl/FTLOperations.cpp: (JSC::FTL::operationMaterializeObjectInOSR): * heap/BlockDirectory.cpp: (JSC::BlockDirectory::assertNoUnswept): * heap/GCSegmentedArray.h: (JSC::GCArraySegment::GCArraySegment): * heap/GCSegmentedArrayInlines.h: (JSC::GCSegmentedArray<T>::clear): (JSC::GCSegmentedArray<T>::expand): (JSC::GCSegmentedArray<T>::validatePrevious): * heap/HandleSet.cpp: * heap/HandleSet.h: * heap/Heap.cpp: (JSC::Heap::updateAllocationLimits): * heap/Heap.h: * heap/MarkedBlock.cpp: * heap/MarkedBlock.h: (JSC::MarkedBlock::assertValidCell const): (JSC::MarkedBlock::assertMarksNotStale): * heap/MarkedSpace.cpp: (JSC::MarkedSpace::beginMarking): (JSC::MarkedSpace::endMarking): (JSC::MarkedSpace::assertNoUnswept): * heap/PreciseAllocation.cpp: * heap/PreciseAllocation.h: (JSC::PreciseAllocation::assertValidCell const): * heap/SlotVisitor.cpp: (JSC::SlotVisitor::SlotVisitor): (JSC::SlotVisitor::appendJSCellOrAuxiliary): * heap/SlotVisitor.h: * inspector/InspectorProtocolTypes.h: (Inspector::Protocol::BindingTraits<JSON::ArrayOf<T>>::assertValueHasExpectedType): * inspector/scripts/codegen/generate_cpp_protocol_types_implementation.py: (CppProtocolTypesImplementationGenerator._generate_assertion_for_object_declaration): (CppProtocolTypesImplementationGenerator): (CppProtocolTypesImplementationGenerator._generate_assertion_for_enum): * inspector/scripts/tests/generic/expected/type-requiring-runtime-casts.json-result: * interpreter/FrameTracers.h: (JSC::JITOperationPrologueCallFrameTracer::JITOperationPrologueCallFrameTracer): * interpreter/Interpreter.cpp: (JSC::Interpreter::Interpreter): * interpreter/Interpreter.h: * jit/AssemblyHelpers.cpp: (JSC::AssemblyHelpers::emitStoreStructureWithTypeInfo): * jit/AssemblyHelpers.h: (JSC::AssemblyHelpers::prepareCallOperation): * jit/BinarySwitch.cpp: (JSC::BinarySwitch::BinarySwitch): * jit/CCallHelpers.h: (JSC::CCallHelpers::setupStubArgs): * jit/CallFrameShuffler.cpp: (JSC::CallFrameShuffler::emitDeltaCheck): (JSC::CallFrameShuffler::prepareAny): * jit/JIT.cpp: (JSC::JIT::assertStackPointerOffset): (JSC::JIT::compileWithoutLinking): * jit/JITOpcodes.cpp: (JSC::JIT::emitSlow_op_loop_hint): * jit/JITPropertyAccess.cpp: (JSC::JIT::emit_op_get_from_scope): * jit/JITPropertyAccess32_64.cpp: (JSC::JIT::emit_op_get_from_scope): * jit/Repatch.cpp: (JSC::linkPolymorphicCall): * jit/ThunkGenerators.cpp: (JSC::emitPointerValidation): * llint/LLIntData.cpp: (JSC::LLInt::Data::performAssertions): * llint/LLIntOfflineAsmConfig.h: * parser/Lexer.cpp: * parser/Lexer.h: (JSC::isSafeBuiltinIdentifier): (JSC::Lexer<T>::lexExpectIdentifier): * runtime/ArgList.h: (JSC::MarkedArgumentBuffer::setNeedsOverflowCheck): (JSC::MarkedArgumentBuffer::clearNeedsOverflowCheck): * runtime/Butterfly.h: (JSC::ContiguousData::ContiguousData): (JSC::ContiguousData::Data::Data): * runtime/HashMapImpl.h: (JSC::HashMapImpl::checkConsistency const): (JSC::HashMapImpl::assertBufferIsEmpty const): * runtime/JSCellInlines.h: (JSC::JSCell::methodTable const): * runtime/JSFunction.cpp: * runtime/JSFunction.h: (JSC::JSFunction::assertTypeInfoFlagInvariants): * runtime/JSGlobalObject.cpp: (JSC::JSGlobalObject::init): * runtime/JSGlobalObject.h: * runtime/JSObject.cpp: (JSC::JSObject::visitChildren): (JSC::JSFinalObject::visitChildren): * runtime/JSObjectInlines.h: (JSC::JSObject::validatePutOwnDataProperty): * runtime/JSSegmentedVariableObject.h: (JSC::JSSegmentedVariableObject::assertVariableIsInThisObject): * runtime/LiteralParser.cpp: (JSC::LiteralParser<CharType>::Lexer::lex): * runtime/LiteralParser.h: * runtime/Operations.h: (JSC::scribbleFreeCells): * runtime/OptionsList.h: * runtime/VM.cpp: (JSC::VM::computeCanUseJIT): * runtime/VM.h: (JSC::VM::canUseJIT): * runtime/VarOffset.h: (JSC::VarOffset::checkSanity const): * runtime/WeakMapImpl.h: (JSC::WeakMapImpl::checkConsistency const): (JSC::WeakMapImpl::assertBufferIsEmpty const): * wasm/WasmAirIRGenerator.cpp: (JSC::Wasm::AirIRGenerator::validateInst): * wasm/WasmB3IRGenerator.cpp: (JSC::Wasm::parseAndCompile): * wasm/WasmFunctionParser.h: (JSC::Wasm::FunctionParser::validationFail const): * wasm/WasmLLIntGenerator.cpp: (JSC::Wasm::LLIntGenerator::checkConsistency): * wasm/WasmPlan.cpp: (JSC::Wasm::Plan::tryRemoveContextAndCancelIfLast): * wasm/WasmSectionParser.h: * wasm/WasmSections.h: * wasm/WasmSignatureInlines.h: (JSC::Wasm::SignatureInformation::get): * wasm/WasmWorklist.cpp: (JSC::Wasm::Worklist::enqueue): * wasm/js/JSToWasm.cpp: (JSC::Wasm::createJSToWasmWrapper): * wasm/js/WebAssemblyFunction.cpp: (JSC::WebAssemblyFunction::previousInstanceOffset const): Source/WebCore: Convert ASSERT_DISABLED to ASSERT_ENABLED, and fix some tests of NDEBUG that should actually test for ASSERT_ENABLED. https://bugs.webkit.org/show_bug.cgi?id=205776 Reviewed by Saam Barati. * Modules/fetch/FetchBodySource.cpp: (WebCore::FetchBodySource::close): * Modules/fetch/FetchBodySource.h: * Modules/webdatabase/DatabaseDetails.h: (WebCore::DatabaseDetails::DatabaseDetails): (WebCore::DatabaseDetails::operator=): * Modules/webdatabase/DatabaseTask.cpp: (WebCore::DatabaseTask::performTask): * Modules/webdatabase/DatabaseTask.h: * Modules/webdatabase/DatabaseThread.cpp: (WebCore::DatabaseThread::terminationRequested const): * Modules/webgpu/WHLSL/AST/WHLSLAddressSpace.h: (WebCore::WHLSL::AST::TypeAnnotation::TypeAnnotation): * Modules/webgpu/WHLSL/WHLSLHighZombieFinder.cpp: (WebCore::WHLSL::findHighZombies): * Modules/webgpu/WHLSL/WHLSLInferTypes.cpp: (WebCore::WHLSL::matches): * Modules/webgpu/WHLSL/WHLSLLiteralTypeChecker.cpp: (WebCore::WHLSL::checkLiteralTypes): * Modules/webgpu/WHLSL/WHLSLSynthesizeConstructors.cpp: (WebCore::WHLSL::FindAllTypes::appendNamedType): * bindings/js/JSCallbackData.h: * bindings/js/JSLazyEventListener.cpp: * bindings/js/JSLazyEventListener.h: * contentextensions/ContentExtensionCompiler.cpp: (WebCore::ContentExtensions::compileRuleList): * css/CSSCalculationValue.cpp: (WebCore::CSSCalcOperationNode::primitiveType const): * css/CSSComputedStyleDeclaration.cpp: (WebCore::ComputedStyleExtractor::valueForPropertyInStyle): * css/CSSPrimitiveValue.cpp: * css/CSSSelector.cpp: (WebCore::CSSSelector::selectorText const): * css/CSSStyleSheet.cpp: * dom/ActiveDOMObject.cpp: (WebCore::ActiveDOMObject::suspendIfNeeded): (WebCore::ActiveDOMObject::assertSuspendIfNeededWasCalled const): * dom/ActiveDOMObject.h: * dom/ContainerNode.cpp: * dom/ContainerNodeAlgorithms.cpp: * dom/ContainerNodeAlgorithms.h: * dom/CustomElementReactionQueue.cpp: * dom/CustomElementReactionQueue.h: (WebCore::CustomElementReactionDisallowedScope::CustomElementReactionDisallowedScope): (WebCore::CustomElementReactionDisallowedScope::~CustomElementReactionDisallowedScope): * dom/Document.cpp: (WebCore::Document::hitTest): * dom/Document.h: (WebCore::Document::decrementReferencingNodeCount): * dom/Element.cpp: (WebCore::Element::addShadowRoot): (WebCore::Element::getURLAttribute const): (WebCore::Element::getNonEmptyURLAttribute const): * dom/Element.h: * dom/ElementAndTextDescendantIterator.h: (WebCore::ElementAndTextDescendantIterator::ElementAndTextDescendantIterator): (WebCore::ElementAndTextDescendantIterator::dropAssertions): (WebCore::ElementAndTextDescendantIterator::popAncestorSiblingStack): (WebCore::ElementAndTextDescendantIterator::traverseNextSibling): (WebCore::ElementAndTextDescendantIterator::traversePreviousSibling): * dom/ElementDescendantIterator.h: (WebCore::ElementDescendantIterator::ElementDescendantIterator): (WebCore::ElementDescendantIterator::dropAssertions): (WebCore::ElementDescendantIterator::operator++): (WebCore::ElementDescendantIterator::operator--): (WebCore::ElementDescendantConstIterator::ElementDescendantConstIterator): (WebCore::ElementDescendantConstIterator::dropAssertions): (WebCore::ElementDescendantConstIterator::operator++): * dom/ElementIterator.h: (WebCore::ElementIterator<ElementType>::ElementIterator): (WebCore::ElementIterator<ElementType>::traverseNext): (WebCore::ElementIterator<ElementType>::traversePrevious): (WebCore::ElementIterator<ElementType>::traverseNextSibling): (WebCore::ElementIterator<ElementType>::traversePreviousSibling): (WebCore::ElementIterator<ElementType>::traverseNextSkippingChildren): (WebCore::ElementIterator<ElementType>::dropAssertions): (WebCore::ElementIterator<ElementType>::traverseAncestor): (WebCore::ElementConstIterator<ElementType>::ElementConstIterator): (WebCore::ElementConstIterator<ElementType>::traverseNext): (WebCore::ElementConstIterator<ElementType>::traversePrevious): (WebCore::ElementConstIterator<ElementType>::traverseNextSibling): (WebCore::ElementConstIterator<ElementType>::traversePreviousSibling): (WebCore::ElementConstIterator<ElementType>::traverseNextSkippingChildren): (WebCore::ElementConstIterator<ElementType>::traverseAncestor): (WebCore::ElementConstIterator<ElementType>::dropAssertions): * dom/EventContext.cpp: * dom/EventContext.h: * dom/EventListener.h: * dom/EventPath.cpp: * dom/EventSender.h: * dom/EventTarget.cpp: (WebCore::EventTarget::addEventListener): (WebCore::EventTarget::setAttributeEventListener): (WebCore::EventTarget::innerInvokeEventListeners): * dom/Node.cpp: (WebCore::Node::~Node): (WebCore::Node::moveNodeToNewDocument): (WebCore::Node::removedLastRef): * dom/Node.h: (WebCore::Node::deref const): * dom/ScriptDisallowedScope.h: (WebCore::ScriptDisallowedScope::InMainThread::isEventDispatchAllowedInSubtree): * dom/ScriptExecutionContext.cpp: (WebCore::ScriptExecutionContext::~ScriptExecutionContext): * dom/ScriptExecutionContext.h: * dom/SelectorQuery.cpp: (WebCore::SelectorDataList::execute const): * dom/SlotAssignment.cpp: (WebCore::SlotAssignment::addSlotElementByName): (WebCore::SlotAssignment::removeSlotElementByName): (WebCore::SlotAssignment::resolveSlotsAfterSlotMutation): (WebCore::SlotAssignment::findFirstSlotElement): * dom/SlotAssignment.h: * dom/TreeScopeOrderedMap.cpp: (WebCore::TreeScopeOrderedMap::add): (WebCore::TreeScopeOrderedMap::get const): * dom/TreeScopeOrderedMap.h: * fileapi/Blob.cpp: * fileapi/Blob.h: * history/BackForwardCache.cpp: (WebCore::BackForwardCache::removeAllItemsForPage): * history/BackForwardCache.h: * html/CanvasBase.cpp: (WebCore::CanvasBase::notifyObserversCanvasDestroyed): * html/CanvasBase.h: * html/HTMLCollection.h: (WebCore::CollectionNamedElementCache::didPopulate): * html/HTMLSelectElement.cpp: (WebCore:: const): * html/HTMLTableRowsCollection.cpp: (WebCore::assertRowIsInTable): * html/HTMLTextFormControlElement.cpp: (WebCore::HTMLTextFormControlElement::indexForPosition const): * html/canvas/CanvasRenderingContext2DBase.cpp: (WebCore::CanvasRenderingContext2DBase::~CanvasRenderingContext2DBase): * html/parser/HTMLParserScheduler.cpp: (WebCore::HTMLParserScheduler::HTMLParserScheduler): (WebCore::HTMLParserScheduler::suspend): (WebCore::HTMLParserScheduler::resume): * html/parser/HTMLParserScheduler.h: * html/parser/HTMLToken.h: (WebCore::HTMLToken::beginStartTag): (WebCore::HTMLToken::beginEndTag): (WebCore::HTMLToken::endAttribute): * html/parser/HTMLTreeBuilder.cpp: (WebCore::HTMLTreeBuilder::HTMLTreeBuilder): (WebCore::HTMLTreeBuilder::constructTree): * html/parser/HTMLTreeBuilder.h: (WebCore::HTMLTreeBuilder::~HTMLTreeBuilder): * layout/FormattingContext.cpp: (WebCore::Layout::FormattingContext::geometryForBox const): * layout/blockformatting/BlockFormattingContext.cpp: (WebCore::Layout::BlockFormattingContext::computeEstimatedVerticalPosition): * layout/blockformatting/BlockFormattingContext.h: * layout/displaytree/DisplayBox.cpp: (WebCore::Display::Box::Box): * layout/displaytree/DisplayBox.h: (WebCore::Display::Box::setTopLeft): (WebCore::Display::Box::setTop): (WebCore::Display::Box::setLeft): (WebCore::Display::Box::setContentBoxHeight): (WebCore::Display::Box::setContentBoxWidth): (WebCore::Display::Box::setHorizontalMargin): (WebCore::Display::Box::setVerticalMargin): (WebCore::Display::Box::setHorizontalComputedMargin): (WebCore::Display::Box::setBorder): (WebCore::Display::Box::setPadding): * layout/displaytree/DisplayInlineRect.h: (WebCore::Display::InlineRect::InlineRect): (WebCore::Display::InlineRect::setTopLeft): (WebCore::Display::InlineRect::setTop): (WebCore::Display::InlineRect::setBottom): (WebCore::Display::InlineRect::setLeft): (WebCore::Display::InlineRect::setWidth): (WebCore::Display::InlineRect::setHeight): * layout/displaytree/DisplayLineBox.h: (WebCore::Display::LineBox::LineBox): (WebCore::Display::LineBox::setBaselineOffsetIfGreater): (WebCore::Display::LineBox::resetBaseline): (WebCore::Display::LineBox::Baseline::Baseline): (WebCore::Display::LineBox::Baseline::setAscent): (WebCore::Display::LineBox::Baseline::setDescent): (WebCore::Display::LineBox::Baseline::reset): * layout/displaytree/DisplayRect.h: (WebCore::Display::Rect::Rect): (WebCore::Display::Rect::setTopLeft): (WebCore::Display::Rect::setTop): (WebCore::Display::Rect::setLeft): (WebCore::Display::Rect::setWidth): (WebCore::Display::Rect::setHeight): (WebCore::Display::Rect::setSize): (WebCore::Display::Rect::clone const): * layout/floats/FloatingContext.cpp: * layout/inlineformatting/InlineLineBuilder.cpp: (WebCore::Layout::LineBuilder::CollapsibleContent::collapse): * layout/tableformatting/TableGrid.cpp: (WebCore::Layout::TableGrid::Column::setWidthConstraints): (WebCore::Layout::TableGrid::Column::setLogicalWidth): (WebCore::Layout::TableGrid::Column::setLogicalLeft): * layout/tableformatting/TableGrid.h: * loader/DocumentLoader.cpp: (WebCore::DocumentLoader::continueAfterContentPolicy): (WebCore::DocumentLoader::attachToFrame): (WebCore::DocumentLoader::detachFromFrame): (WebCore::DocumentLoader::addSubresourceLoader): * loader/DocumentLoader.h: * loader/ImageLoader.cpp: * loader/cache/CachedResource.h: * loader/cache/MemoryCache.cpp: (WebCore::MemoryCache::lruListFor): (WebCore::MemoryCache::removeFromLRUList): * page/FrameView.cpp: (WebCore::FrameView::updateLayoutAndStyleIfNeededRecursive): * page/FrameViewLayoutContext.cpp: * page/FrameViewLayoutContext.h: * page/Page.cpp: * page/Page.h: * page/ViewportConfiguration.cpp: * page/ViewportConfiguration.h: * page/mac/EventHandlerMac.mm: (WebCore::CurrentEventScope::CurrentEventScope): * platform/DateComponents.cpp: (WebCore::DateComponents::toStringForTime const): * platform/ScrollableArea.cpp: * platform/SharedBuffer.cpp: (WebCore::SharedBuffer::combineIntoOneSegment const): * platform/SharedBuffer.h: * platform/Supplementable.h: * platform/Timer.cpp: (WebCore::TimerBase::checkHeapIndex const): (WebCore::TimerBase::updateHeapIfNeeded): * platform/graphics/BitmapImage.cpp: * platform/graphics/BitmapImage.h: * platform/graphics/Image.h: * platform/graphics/ShadowBlur.cpp: (WebCore::ScratchBuffer::ScratchBuffer): (WebCore::ScratchBuffer::getScratchBuffer): (WebCore::ScratchBuffer::scheduleScratchBufferPurge): * platform/graphics/ca/win/CACFLayerTreeHost.cpp: (WebCore::CACFLayerTreeHost::setWindow): * platform/graphics/ca/win/CACFLayerTreeHost.h: * platform/graphics/cg/ImageBufferDataCG.cpp: (WebCore::ImageBufferData::putData): * platform/graphics/cocoa/FontCacheCoreText.cpp: * platform/graphics/gstreamer/GstAllocatorFastMalloc.cpp: (gstAllocatorFastMallocFree): * platform/graphics/nicosia/cairo/NicosiaPaintingContextCairo.cpp: (Nicosia::PaintingContextCairo::ForPainting::ForPainting): * platform/graphics/nicosia/texmap/NicosiaBackingStoreTextureMapperImpl.cpp: (Nicosia::BackingStoreTextureMapperImpl::createTile): * platform/graphics/nicosia/texmap/NicosiaContentLayerTextureMapperImpl.cpp: (Nicosia::ContentLayerTextureMapperImpl::~ContentLayerTextureMapperImpl): * platform/graphics/win/GradientDirect2D.cpp: (WebCore::Gradient::fill): * platform/graphics/win/ImageBufferDataDirect2D.cpp: (WebCore::ImageBufferData::putData): * platform/graphics/win/PathDirect2D.cpp: (WebCore::Path::appendGeometry): (WebCore::Path::Path): (WebCore::Path::operator=): (WebCore::Path::strokeContains const): (WebCore::Path::transform): * platform/graphics/win/PlatformContextDirect2D.cpp: (WebCore::PlatformContextDirect2D::setTags): * platform/mediastream/MediaStreamTrackPrivate.h: * platform/mediastream/RealtimeOutgoingAudioSource.cpp: (WebCore::RealtimeOutgoingAudioSource::~RealtimeOutgoingAudioSource): * platform/mediastream/RealtimeOutgoingVideoSource.cpp: (WebCore::RealtimeOutgoingVideoSource::~RealtimeOutgoingVideoSource): * platform/network/HTTPParsers.cpp: (WebCore::isCrossOriginSafeHeader): * platform/sql/SQLiteDatabase.cpp: * platform/sql/SQLiteDatabase.h: * platform/sql/SQLiteStatement.cpp: (WebCore::SQLiteStatement::SQLiteStatement): (WebCore::SQLiteStatement::prepare): (WebCore::SQLiteStatement::finalize): * platform/sql/SQLiteStatement.h: * platform/win/COMPtr.h: * rendering/ComplexLineLayout.cpp: (WebCore::ComplexLineLayout::removeInlineBox const): * rendering/FloatingObjects.cpp: (WebCore::FloatingObject::FloatingObject): (WebCore::FloatingObjects::addPlacedObject): (WebCore::FloatingObjects::removePlacedObject): * rendering/FloatingObjects.h: * rendering/GridTrackSizingAlgorithm.cpp: * rendering/GridTrackSizingAlgorithm.h: * rendering/LayoutDisallowedScope.cpp: * rendering/LayoutDisallowedScope.h: * rendering/RenderBlock.cpp: * rendering/RenderBlock.h: * rendering/RenderBlockFlow.cpp: (WebCore::RenderBlockFlow::layoutBlockChild): (WebCore::RenderBlockFlow::removeFloatingObject): (WebCore::RenderBlockFlow::ensureLineBoxes): * rendering/RenderBoxModelObject.cpp: * rendering/RenderDeprecatedFlexibleBox.cpp: (WebCore::RenderDeprecatedFlexibleBox::layoutBlock): * rendering/RenderElement.cpp: * rendering/RenderGeometryMap.cpp: (WebCore::RenderGeometryMap::mapToContainer const): * rendering/RenderGrid.cpp: (WebCore::RenderGrid::placeItemsOnGrid const): (WebCore::RenderGrid::baselinePosition const): * rendering/RenderInline.cpp: (WebCore::RenderInline::willBeDestroyed): * rendering/RenderLayer.cpp: (WebCore::ClipRectsCache::ClipRectsCache): (WebCore::RenderLayer::RenderLayer): (WebCore::RenderLayer::paintList): (WebCore::RenderLayer::hitTestLayer): (WebCore::RenderLayer::updateClipRects): (WebCore::RenderLayer::calculateClipRects const): * rendering/RenderLayer.h: * rendering/RenderLayerBacking.cpp: (WebCore::traverseVisibleNonCompositedDescendantLayers): * rendering/RenderLayerCompositor.cpp: (WebCore::RenderLayerCompositor::computeCompositingRequirements): (WebCore::RenderLayerCompositor::traverseUnchangedSubtree): (WebCore::RenderLayerCompositor::updateBackingAndHierarchy): (WebCore::RenderLayerCompositor::addDescendantsToOverlapMapRecursive const): (WebCore::RenderLayerCompositor::recursiveRepaintLayer): (WebCore::RenderLayerCompositor::layerHas3DContent const): * rendering/RenderLayoutState.cpp: (WebCore::RenderLayoutState::RenderLayoutState): (WebCore::RenderLayoutState::computeOffsets): (WebCore::RenderLayoutState::addLayoutDelta): * rendering/RenderLayoutState.h: (WebCore::RenderLayoutState::RenderLayoutState): * rendering/RenderObject.cpp: (WebCore::RenderObject::RenderObject): (WebCore::RenderObject::~RenderObject): (WebCore::RenderObject::clearNeedsLayout): * rendering/RenderObject.h: * rendering/RenderQuote.cpp: (WebCore::quotesForLanguage): * rendering/RenderTableCell.h: * rendering/RenderTableSection.cpp: (WebCore::RenderTableSection::computeOverflowFromCells): * rendering/RenderTextLineBoxes.cpp: (WebCore::RenderTextLineBoxes::checkConsistency const): * rendering/RenderTextLineBoxes.h: * rendering/line/BreakingContext.h: (WebCore::tryHyphenating): * rendering/style/GridArea.h: (WebCore::GridSpan::GridSpan): * rendering/style/RenderStyle.cpp: (WebCore::RenderStyle::~RenderStyle): * rendering/style/RenderStyle.h: * rendering/updating/RenderTreeBuilderRuby.cpp: (WebCore::RenderTreeBuilder::Ruby::detach): * rendering/updating/RenderTreePosition.cpp: (WebCore::RenderTreePosition::computeNextSibling): * rendering/updating/RenderTreePosition.h: * svg/SVGToOTFFontConversion.cpp: (WebCore::SVGToOTFFontConverter::Placeholder::Placeholder): (WebCore::SVGToOTFFontConverter::Placeholder::populate): (WebCore::SVGToOTFFontConverter::appendCFFTable): (WebCore::SVGToOTFFontConverter::firstGlyph const): (WebCore::SVGToOTFFontConverter::appendKERNTable): * svg/SVGTransformDistance.cpp: (WebCore::SVGTransformDistance::SVGTransformDistance): (WebCore::SVGTransformDistance::scaledDistance const): (WebCore::SVGTransformDistance::addSVGTransforms): (WebCore::SVGTransformDistance::addToSVGTransform const): (WebCore::SVGTransformDistance::distance const): * svg/graphics/SVGImage.cpp: (WebCore::SVGImage::nativeImage): * testing/InternalSettings.cpp: * workers/service/ServiceWorkerJob.h: * worklets/PaintWorkletGlobalScope.h: (WebCore::PaintWorkletGlobalScope::~PaintWorkletGlobalScope): * xml/XPathStep.cpp: Source/WebKit: Convert ASSERT_DISABLED to ASSERT_ENABLED, and fix some tests of NDEBUG that should actually test for ASSERT_ENABLED. https://bugs.webkit.org/show_bug.cgi?id=205776 Reviewed by Saam Barati. * NetworkProcess/NetworkSession.cpp: (WebKit::NetworkSession::invalidateAndCancel): * NetworkProcess/NetworkSession.h: * NetworkProcess/cache/NetworkCacheStorage.cpp: (WebKit::NetworkCache::Storage::setCapacity): * NetworkProcess/cocoa/NetworkSessionCocoa.mm: (toNSURLSessionResponseDisposition): (WebKit::NetworkSessionCocoa::NetworkSessionCocoa): * Platform/IPC/Connection.cpp: (IPC::Connection::waitForMessage): * Platform/IPC/MessageReceiver.h: (IPC::MessageReceiver::willBeAddedToMessageReceiverMap): (IPC::MessageReceiver::willBeRemovedFromMessageReceiverMap): * Platform/IPC/cocoa/ConnectionCocoa.mm: (IPC::readFromMachPort): * Platform/mac/MachUtilities.cpp: (setMachExceptionPort): * Shared/API/APIClient.h: (API::Client::Client): * Shared/API/Cocoa/WKRemoteObjectCoder.mm: * Shared/Cocoa/ArgumentCodersCocoa.h: * Shared/SharedStringHashTableReadOnly.cpp: * UIProcess/BackingStore.cpp: (WebKit::BackingStore::incorporateUpdate): * UIProcess/GenericCallback.h: * UIProcess/Launcher/mac/ProcessLauncherMac.mm: (WebKit::ProcessLauncher::launchProcess): * UIProcess/PageLoadState.h: (WebKit::PageLoadState::Transaction::Token::Token): * UIProcess/WebPageProxy.cpp: (WebKit::WebPageProxy::~WebPageProxy): * WebProcess/Network/WebResourceLoader.cpp: (WebKit::WebResourceLoader::didReceiveResponse): * WebProcess/Network/WebResourceLoader.h: * WebProcess/Plugins/Netscape/NetscapePluginStream.cpp: (WebKit::NetscapePluginStream::NetscapePluginStream): (WebKit::NetscapePluginStream::notifyAndDestroyStream): * WebProcess/Plugins/Netscape/NetscapePluginStream.h: * WebProcess/WebPage/WebPage.cpp: (WebKit::WebPage::runModal): * WebProcess/WebProcess.cpp: (WebKit::checkDocumentsCaptureStateConsistency): * WebProcess/cocoa/WebProcessCocoa.mm: (WebKit::WebProcess::updateProcessName): Source/WebKitLegacy: Convert ASSERT_DISABLED to ASSERT_ENABLED, and fix some tests of NDEBUG that should actually test for ASSERT_ENABLED. https://bugs.webkit.org/show_bug.cgi?id=205776 Reviewed by Saam Barati. * Storage/StorageAreaImpl.cpp: (WebKit::StorageAreaImpl::StorageAreaImpl): (WebKit::StorageAreaImpl::close): * Storage/StorageAreaImpl.h: Source/WebKitLegacy/mac: Convert ASSERT_DISABLED to ASSERT_ENABLED, and fix some tests of NDEBUG that should actually test for ASSERT_ENABLED. https://bugs.webkit.org/show_bug.cgi?id=205776 Reviewed by Saam Barati. * History/WebHistory.mm: (-[WebHistoryPrivate removeItemForURLString:]): * WebView/WebFrame.mm: Source/WebKitLegacy/win: Convert ASSERT_DISABLED to ASSERT_ENABLED, and fix some tests of NDEBUG that should actually test for ASSERT_ENABLED. https://bugs.webkit.org/show_bug.cgi?id=205776 Reviewed by Saam Barati. * WebKitQuartzCoreAdditions/CAD3DRenderer.cpp: (WKQCA::CAD3DRenderer::swapChain): (WKQCA::CAD3DRenderer::initialize): * WebKitQuartzCoreAdditions/CAD3DRenderer.h: * WebView.cpp: (WebView::Release): * WebView.h: Source/WTF: Convert ASSERT_DISABLED to ASSERT_ENABLED, and fix some tests of NDEBUG that should actually test for ASSERT_ENABLED. https://bugs.webkit.org/show_bug.cgi?id=205776 Reviewed by Saam Barati. This patch did the following changes: 1. Replaced ASSERT_DISABLED with ASSERT_ENABLED. This change does away with the need for the double negative !ASSERT_DISABLED test that is commonly used all over the code, thereby improving code readability. In Assertions.h, there is also BACKTRACE_DISABLED, ASSERT_MSG_DISABLED, ASSERT_ARG_DISABLED, FATAL_DISABLED, ERROR_DISABLED, LOG_DISABLED, and RELEASE_LOG_DISABLED. We should replace those with ..._ENABLED equivalents as well. We'll do that in another patch. For now, they are left as is to minimize the size of this patch. See https://bugs.webkit.org/show_bug.cgi?id=205780. 2. Fixed some code was guarded with "#ifndef NDEBUG" that should actually be guarded by "#if ASSERT_ENABLED" instead. 3. In cases where the change is minimal, we move some code around so that we can test for "#if ASSERT_ENABLED" instead of "#if !ASSERT_ENABLED". * wtf/Assertions.h: * wtf/AutomaticThread.cpp: (WTF::AutomaticThread::start): * wtf/BitVector.h: * wtf/BlockObjCExceptions.mm: (ReportBlockedObjCException): * wtf/BloomFilter.h: * wtf/CallbackAggregator.h: (WTF::CallbackAggregator::CallbackAggregator): * wtf/CheckedArithmetic.h: (WTF::observesOverflow<AssertNoOverflow>): * wtf/CheckedBoolean.h: (CheckedBoolean::CheckedBoolean): (CheckedBoolean::operator bool): * wtf/CompletionHandler.h: (WTF::CompletionHandler<Out): * wtf/DateMath.cpp: (WTF::initializeDates): * wtf/Gigacage.cpp: (Gigacage::tryAllocateZeroedVirtualPages): * wtf/HashTable.h: (WTF::KeyTraits>::checkKey): (WTF::KeyTraits>::checkTableConsistencyExceptSize const): * wtf/LoggerHelper.h: * wtf/NaturalLoops.h: (WTF::NaturalLoops::headerOf const): * wtf/NeverDestroyed.h: (WTF::LazyNeverDestroyed::construct): * wtf/OptionSet.h: (WTF::OptionSet::OptionSet): * wtf/Platform.h: * wtf/PtrTag.h: * wtf/RefCounted.h: (WTF::RefCountedBase::disableThreadingChecks): (WTF::RefCountedBase::enableThreadingChecksGlobally): (WTF::RefCountedBase::RefCountedBase): (WTF::RefCountedBase::applyRefDerefThreadingCheck const): * wtf/SingleRootGraph.h: (WTF::SingleRootGraph::assertIsConsistent const): * wtf/SizeLimits.cpp: * wtf/StackBounds.h: (WTF::StackBounds::checkConsistency const): * wtf/URLParser.cpp: (WTF::URLParser::URLParser): (WTF::URLParser::domainToASCII): * wtf/ValueCheck.h: * wtf/Vector.h: (WTF::Malloc>::checkConsistency): * wtf/WeakHashSet.h: * wtf/WeakPtr.h: (WTF::WeakPtrImpl::WeakPtrImpl): (WTF::WeakPtrFactory::WeakPtrFactory): * wtf/text/AtomStringImpl.cpp: * wtf/text/AtomStringImpl.h: * wtf/text/StringBuilder.cpp: (WTF::StringBuilder::reifyString const): * wtf/text/StringBuilder.h: * wtf/text/StringCommon.h: (WTF::hasPrefixWithLettersIgnoringASCIICaseCommon): * wtf/text/StringHasher.h: (WTF::StringHasher::addCharacters): * wtf/text/StringImpl.h: * wtf/text/SymbolImpl.h: * wtf/text/UniquedStringImpl.h: Tools: Remove WebsiteDataStore::setServiceWorkerRegistrationDirectory https://bugs.webkit.org/show_bug.cgi?id=205754 Patch by Alex Christensen <achristensen@webkit.org> on 2020-01-06 Reviewed by Youenn Fablet. * TestWebKitAPI/Tests/WebKitCocoa/ServiceWorkerBasic.mm: * WebKitTestRunner/TestController.cpp: (WTR::TestController::websiteDataStore): (WTR::TestController::platformAdjustContext): * WebKitTestRunner/cocoa/TestControllerCocoa.mm: (WTR::initializeWebViewConfiguration): Canonical link: https://commits.webkit.org/218957@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@254087 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2020-01-06 22:24:50 +00:00
#if ASSERT_ENABLED
CheckedBoolean(const CheckedBoolean& other)
: m_value(other.m_value)
, m_checked(false)
{
other.m_checked = true;
}
#endif
Implement a new allocator for backing stores https://bugs.webkit.org/show_bug.cgi?id=75181 Reviewed by Filip Pizlo. Source/JavaScriptCore: We want to move away from using fastMalloc for the backing stores for some of our objects (e.g. JSArray, JSObject, JSString, etc). These backing stores have a nice property in that they only have a single owner (i.e. a single pointer to them at any one time). One way that we can take advantage of this property is to implement a simple bump allocator/copying collector, which will run alongside our normal mark/sweep collector, that only needs to update the single owner pointer rather than having to redirect an arbitrary number of pointers in from-space to to-space. This plan can give us a number of benefits. We can beat fastMalloc in terms of both performance and memory usage, we can track how much memory we're using far more accurately than our rough estimation now through the use of reportExtraMemoryCost, and we can allocate arbitrary size objects (as opposed to being limited to size classes like we have been historically). This is also another step toward moving away from lazy destruction, which will improve our memory footprint. We start by creating said allocator and moving the ArrayStorage for JSArray to use it rather than fastMalloc. The design of the collector is as follows: Allocation: -The collector allocates 64KB chunks from the OS to use for object allocation. -Each chunk contains an offset, a flag indicating if the block has been pinned, and a payload, along with next and prev pointers so that they can be put in DoublyLinkedLists. -Any allocation greater than 64KB gets its own separate oversize block, which is managed separately from the rest. -If the allocator receives a request for more than the remaining amount in the current block, it grabs a fresh block. -Grabbing a fresh block means grabbing one off of the global free list (which is now shared between the mark/sweep allocator and the bump allocator) if there is one. If there isn't a new one we do one of two things: allocate a new block from the OS if we're not ready for a GC yet, or run a GC and then try again. If we still don't have enough space after the GC, we allocate a new block from the OS. Garbage collection: -At the start of garbage collection during conservative stack scanning, if we encounter what appears to be a pointer to a bump-allocated block of memory, we pin that block so that it will not be copied for this round of collection. -We also pin any oversize blocks that we encounter, which effectively doubles as a "mark bit" for that block. Any oversize blocks that aren't pinned at the end of copying are given back to the OS. -Marking threads are now also responsible for copying bump-allocated objects to newSpace -Each marking thread has a private 64KB block into which it copies bump-allocated objects that it encounters. -When that block fills up, the marking thread gives it back to the allocator and requests a new one. -When all marking has concluded, each thread gives back its copy block, even if it isn't full. -At the conclusion of copying (which is done by the end of the marking phase), we un-pin any pinned blocks and give any blocks left in from-space to the global free list. * CMakeLists.txt: * GNUmakefile.list.am: * JavaScriptCore.gypi: * JavaScriptCore.vcproj/JavaScriptCore/JavaScriptCore.vcproj: * JavaScriptCore.vcproj/WTF/WTF.vcproj: * JavaScriptCore.xcodeproj/project.pbxproj: * Target.pri: * heap/AllocationSpace.cpp: (JSC::AllocationSpace::allocateSlowCase): (JSC::AllocationSpace::allocateBlock): (JSC::AllocationSpace::freeBlocks): * heap/AllocationSpace.h: (JSC::AllocationSpace::waterMark): * heap/BumpBlock.h: Added. (JSC::BumpBlock::BumpBlock): * heap/BumpSpace.cpp: Added. (JSC::BumpSpace::tryAllocateSlowCase): * heap/BumpSpace.h: Added. (JSC::BumpSpace::isInCopyPhase): (JSC::BumpSpace::totalMemoryAllocated): (JSC::BumpSpace::totalMemoryUtilized): * heap/BumpSpaceInlineMethods.h: Added. (JSC::BumpSpace::BumpSpace): (JSC::BumpSpace::init): (JSC::BumpSpace::contains): (JSC::BumpSpace::pin): (JSC::BumpSpace::startedCopying): (JSC::BumpSpace::doneCopying): (JSC::BumpSpace::doneFillingBlock): (JSC::BumpSpace::recycleBlock): (JSC::BumpSpace::getFreshBlock): (JSC::BumpSpace::borrowBlock): (JSC::BumpSpace::addNewBlock): (JSC::BumpSpace::allocateNewBlock): (JSC::BumpSpace::fitsInBlock): (JSC::BumpSpace::fitsInCurrentBlock): (JSC::BumpSpace::tryAllocate): (JSC::BumpSpace::tryAllocateOversize): (JSC::BumpSpace::allocateFromBlock): (JSC::BumpSpace::tryReallocate): (JSC::BumpSpace::tryReallocateOversize): (JSC::BumpSpace::isOversize): (JSC::BumpSpace::isPinned): (JSC::BumpSpace::oversizeBlockFor): (JSC::BumpSpace::blockFor): * heap/ConservativeRoots.cpp: (JSC::ConservativeRoots::ConservativeRoots): (JSC::ConservativeRoots::genericAddPointer): (JSC::ConservativeRoots::add): * heap/ConservativeRoots.h: * heap/Heap.cpp: (JSC::Heap::Heap): (JSC::Heap::blockFreeingThreadMain): (JSC::Heap::reportExtraMemoryCostSlowCase): (JSC::Heap::getConservativeRegisterRoots): (JSC::Heap::markRoots): (JSC::Heap::collect): (JSC::Heap::releaseFreeBlocks): * heap/Heap.h: (JSC::Heap::waterMark): (JSC::Heap::highWaterMark): (JSC::Heap::setHighWaterMark): (JSC::Heap::tryAllocateStorage): (JSC::Heap::tryReallocateStorage): * heap/HeapBlock.h: Added. (JSC::HeapBlock::HeapBlock): * heap/MarkStack.cpp: (JSC::MarkStackThreadSharedData::MarkStackThreadSharedData): (JSC::SlotVisitor::drain): (JSC::SlotVisitor::drainFromShared): (JSC::SlotVisitor::startCopying): (JSC::SlotVisitor::allocateNewSpace): (JSC::SlotVisitor::copy): (JSC::SlotVisitor::copyAndAppend): (JSC::SlotVisitor::doneCopying): * heap/MarkStack.h: * heap/MarkedBlock.cpp: (JSC::MarkedBlock::recycle): (JSC::MarkedBlock::MarkedBlock): * heap/MarkedBlock.h: * heap/MarkedSpace.cpp: (JSC::MarkedSpace::MarkedSpace): * heap/MarkedSpace.h: (JSC::MarkedSpace::allocate): (JSC::MarkedSpace::forEachBlock): (JSC::MarkedSpace::SizeClass::resetAllocator): * heap/SlotVisitor.h: (JSC::SlotVisitor::SlotVisitor): * heap/TinyBloomFilter.h: (JSC::TinyBloomFilter::reset): * runtime/JSArray.cpp: (JSC::JSArray::JSArray): (JSC::JSArray::finishCreation): (JSC::JSArray::tryFinishCreationUninitialized): (JSC::JSArray::~JSArray): (JSC::JSArray::enterSparseMode): (JSC::JSArray::defineOwnNumericProperty): (JSC::JSArray::setLengthWritable): (JSC::JSArray::getOwnPropertySlotByIndex): (JSC::JSArray::getOwnPropertyDescriptor): (JSC::JSArray::putByIndexBeyondVectorLength): (JSC::JSArray::deletePropertyByIndex): (JSC::JSArray::getOwnPropertyNames): (JSC::JSArray::increaseVectorLength): (JSC::JSArray::unshiftCountSlowCase): (JSC::JSArray::setLength): (JSC::JSArray::pop): (JSC::JSArray::unshiftCount): (JSC::JSArray::visitChildren): (JSC::JSArray::sortNumeric): (JSC::JSArray::sort): (JSC::JSArray::compactForSorting): (JSC::JSArray::subclassData): (JSC::JSArray::setSubclassData): (JSC::JSArray::checkConsistency): * runtime/JSArray.h: (JSC::JSArray::inSparseMode): (JSC::JSArray::isLengthWritable): * wtf/CheckedBoolean.h: Added. (CheckedBoolean::CheckedBoolean): (CheckedBoolean::~CheckedBoolean): (CheckedBoolean::operator bool): * wtf/DoublyLinkedList.h: (WTF::::push): * wtf/StdLibExtras.h: (WTF::isPointerAligned): Source/JavaScriptGlue: Added forwarding header for new CheckedBoolean used in the bump allocator. * ForwardingHeaders/wtf/CheckedBoolean.h: Added. Source/WebCore: No new tests. Added forwarding header for new CheckedBoolean used in the bump allocator. * ForwardingHeaders/wtf/CheckedBoolean.h: Added. Canonical link: https://commits.webkit.org/93486@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@105442 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2012-01-19 21:49:56 +00:00
CheckedBoolean(bool value)
: m_value(value)
PerformanceTests: Convert ASSERT_DISABLED to ASSERT_ENABLED, and fix some tests of NDEBUG that should actually test for ASSERT_ENABLED. https://bugs.webkit.org/show_bug.cgi?id=205776 Reviewed by Saam Barati. * JetStream2/wasm/HashSet.cpp: * StitchMarker/wtf/Assertions.h: * StitchMarker/wtf/DateMath.cpp: (WTF::initializeDates): * StitchMarker/wtf/HashTable.h: * StitchMarker/wtf/Hasher.h: (WTF::StringHasher::addCharacters): * StitchMarker/wtf/NeverDestroyed.h: (WTF::LazyNeverDestroyed::construct): * StitchMarker/wtf/StackBounds.h: (WTF::StackBounds::checkConsistency const): * StitchMarker/wtf/ValueCheck.h: * StitchMarker/wtf/Vector.h: (WTF::minCapacity>::checkConsistency): * StitchMarker/wtf/text/AtomicStringImpl.cpp: * StitchMarker/wtf/text/AtomicStringImpl.h: * StitchMarker/wtf/text/StringCommon.h: (WTF::hasPrefixWithLettersIgnoringASCIICaseCommon): * StitchMarker/wtf/text/StringImpl.h: * StitchMarker/wtf/text/SymbolImpl.h: * StitchMarker/wtf/text/UniquedStringImpl.h: Source/JavaScriptCore: Convert ASSERT_DISABLED to ASSERT_ENABLED, and fix some tests of NDEBUG that should actually test for ASSERT_ENABLED. https://bugs.webkit.org/show_bug.cgi?id=205776 Reviewed by Saam Barati. * API/tests/testapi.c: * assembler/ARM64Assembler.h: (JSC::ARM64Assembler::replaceWithLoad): (JSC::ARM64Assembler::replaceWithAddressComputation): * assembler/AssemblerBuffer.h: (JSC::AssemblerBuffer::LocalWriter::LocalWriter): * assembler/LinkBuffer.cpp: (JSC::LinkBuffer::copyCompactAndLinkCode): * assembler/ProbeStack.cpp: (JSC::Probe::Stack::Stack): * assembler/ProbeStack.h: * b3/B3FoldPathConstants.cpp: * b3/B3LowerToAir.cpp: * b3/B3MemoryValue.cpp: (JSC::B3::MemoryValue::MemoryValue): * b3/B3Opcode.cpp: * b3/B3Type.h: * b3/B3TypeMap.h: * b3/B3Width.h: * b3/air/AirAllocateRegistersAndStackAndGenerateCode.cpp: (JSC::B3::Air::GenerateAndAllocateRegisters::prepareForGeneration): (JSC::B3::Air::GenerateAndAllocateRegisters::generate): * b3/air/AirAllocateRegistersAndStackAndGenerateCode.h: * b3/air/AirAllocateRegistersByGraphColoring.cpp: * b3/air/AirArg.cpp: * b3/air/AirArg.h: * b3/air/AirCode.h: * b3/air/AirEmitShuffle.cpp: (JSC::B3::Air::emitShuffle): * builtins/BuiltinExecutables.cpp: (JSC::BuiltinExecutables::createExecutable): * bytecode/AccessCase.cpp: * bytecode/AccessCase.h: * bytecode/CallVariant.cpp: (JSC::variantListWithVariant): * bytecode/CodeBlock.cpp: (JSC::CodeBlock::ensureCatchLivenessIsComputedForBytecodeIndex): * bytecode/CodeBlockHash.cpp: (JSC::CodeBlockHash::dump const): * bytecode/StructureStubInfo.cpp: * bytecode/StructureStubInfo.h: * bytecompiler/NodesCodegen.cpp: (JSC::FunctionCallResolveNode::emitBytecode): * bytecompiler/RegisterID.h: (JSC::RegisterID::RegisterID): (JSC::RegisterID::setIndex): * debugger/Debugger.cpp: (JSC::Debugger::removeBreakpoint): * debugger/DebuggerEvalEnabler.h: (JSC::DebuggerEvalEnabler::DebuggerEvalEnabler): (JSC::DebuggerEvalEnabler::~DebuggerEvalEnabler): * dfg/DFGAbstractInterpreterInlines.h: (JSC::DFG::AbstractInterpreter<AbstractStateType>::observeTransitions): * dfg/DFGAbstractValue.cpp: * dfg/DFGAbstractValue.h: (JSC::DFG::AbstractValue::merge): (JSC::DFG::AbstractValue::checkConsistency const): (JSC::DFG::AbstractValue::assertIsRegistered const): * dfg/DFGArithMode.h: (JSC::DFG::doesOverflow): * dfg/DFGBasicBlock.cpp: (JSC::DFG::BasicBlock::BasicBlock): * dfg/DFGBasicBlock.h: (JSC::DFG::BasicBlock::didLink): * dfg/DFGCFAPhase.cpp: (JSC::DFG::CFAPhase::performBlockCFA): * dfg/DFGCommon.h: (JSC::DFG::validationEnabled): * dfg/DFGCommonData.cpp: (JSC::DFG::CommonData::finalizeCatchEntrypoints): * dfg/DFGDesiredWatchpoints.h: * dfg/DFGDoesGC.cpp: (JSC::DFG::doesGC): * dfg/DFGEdge.h: (JSC::DFG::Edge::makeWord): * dfg/DFGFixupPhase.cpp: (JSC::DFG::FixupPhase::fixupNode): * dfg/DFGJITCode.cpp: (JSC::DFG::JITCode::finalizeOSREntrypoints): * dfg/DFGObjectAllocationSinkingPhase.cpp: * dfg/DFGSSAConversionPhase.cpp: (JSC::DFG::SSAConversionPhase::run): * dfg/DFGScoreBoard.h: (JSC::DFG::ScoreBoard::assertClear): * dfg/DFGSlowPathGenerator.h: (JSC::DFG::SlowPathGenerator::generate): * dfg/DFGSpeculativeJIT.cpp: (JSC::DFG::SpeculativeJIT::compileCurrentBlock): (JSC::DFG::SpeculativeJIT::emitBinarySwitchStringRecurse): (JSC::DFG::SpeculativeJIT::emitAllocateButterfly): (JSC::DFG::SpeculativeJIT::compileAllocateNewArrayWithSize): (JSC::DFG::SpeculativeJIT::compileMakeRope): * dfg/DFGSpeculativeJIT64.cpp: (JSC::DFG::SpeculativeJIT::fillSpeculateCell): * dfg/DFGStructureAbstractValue.cpp: * dfg/DFGStructureAbstractValue.h: (JSC::DFG::StructureAbstractValue::assertIsRegistered const): * dfg/DFGVarargsForwardingPhase.cpp: * dfg/DFGVirtualRegisterAllocationPhase.cpp: (JSC::DFG::VirtualRegisterAllocationPhase::run): * ftl/FTLLink.cpp: (JSC::FTL::link): * ftl/FTLLowerDFGToB3.cpp: (JSC::FTL::DFG::LowerDFGToB3::callPreflight): (JSC::FTL::DFG::LowerDFGToB3::callCheck): (JSC::FTL::DFG::LowerDFGToB3::crash): * ftl/FTLOperations.cpp: (JSC::FTL::operationMaterializeObjectInOSR): * heap/BlockDirectory.cpp: (JSC::BlockDirectory::assertNoUnswept): * heap/GCSegmentedArray.h: (JSC::GCArraySegment::GCArraySegment): * heap/GCSegmentedArrayInlines.h: (JSC::GCSegmentedArray<T>::clear): (JSC::GCSegmentedArray<T>::expand): (JSC::GCSegmentedArray<T>::validatePrevious): * heap/HandleSet.cpp: * heap/HandleSet.h: * heap/Heap.cpp: (JSC::Heap::updateAllocationLimits): * heap/Heap.h: * heap/MarkedBlock.cpp: * heap/MarkedBlock.h: (JSC::MarkedBlock::assertValidCell const): (JSC::MarkedBlock::assertMarksNotStale): * heap/MarkedSpace.cpp: (JSC::MarkedSpace::beginMarking): (JSC::MarkedSpace::endMarking): (JSC::MarkedSpace::assertNoUnswept): * heap/PreciseAllocation.cpp: * heap/PreciseAllocation.h: (JSC::PreciseAllocation::assertValidCell const): * heap/SlotVisitor.cpp: (JSC::SlotVisitor::SlotVisitor): (JSC::SlotVisitor::appendJSCellOrAuxiliary): * heap/SlotVisitor.h: * inspector/InspectorProtocolTypes.h: (Inspector::Protocol::BindingTraits<JSON::ArrayOf<T>>::assertValueHasExpectedType): * inspector/scripts/codegen/generate_cpp_protocol_types_implementation.py: (CppProtocolTypesImplementationGenerator._generate_assertion_for_object_declaration): (CppProtocolTypesImplementationGenerator): (CppProtocolTypesImplementationGenerator._generate_assertion_for_enum): * inspector/scripts/tests/generic/expected/type-requiring-runtime-casts.json-result: * interpreter/FrameTracers.h: (JSC::JITOperationPrologueCallFrameTracer::JITOperationPrologueCallFrameTracer): * interpreter/Interpreter.cpp: (JSC::Interpreter::Interpreter): * interpreter/Interpreter.h: * jit/AssemblyHelpers.cpp: (JSC::AssemblyHelpers::emitStoreStructureWithTypeInfo): * jit/AssemblyHelpers.h: (JSC::AssemblyHelpers::prepareCallOperation): * jit/BinarySwitch.cpp: (JSC::BinarySwitch::BinarySwitch): * jit/CCallHelpers.h: (JSC::CCallHelpers::setupStubArgs): * jit/CallFrameShuffler.cpp: (JSC::CallFrameShuffler::emitDeltaCheck): (JSC::CallFrameShuffler::prepareAny): * jit/JIT.cpp: (JSC::JIT::assertStackPointerOffset): (JSC::JIT::compileWithoutLinking): * jit/JITOpcodes.cpp: (JSC::JIT::emitSlow_op_loop_hint): * jit/JITPropertyAccess.cpp: (JSC::JIT::emit_op_get_from_scope): * jit/JITPropertyAccess32_64.cpp: (JSC::JIT::emit_op_get_from_scope): * jit/Repatch.cpp: (JSC::linkPolymorphicCall): * jit/ThunkGenerators.cpp: (JSC::emitPointerValidation): * llint/LLIntData.cpp: (JSC::LLInt::Data::performAssertions): * llint/LLIntOfflineAsmConfig.h: * parser/Lexer.cpp: * parser/Lexer.h: (JSC::isSafeBuiltinIdentifier): (JSC::Lexer<T>::lexExpectIdentifier): * runtime/ArgList.h: (JSC::MarkedArgumentBuffer::setNeedsOverflowCheck): (JSC::MarkedArgumentBuffer::clearNeedsOverflowCheck): * runtime/Butterfly.h: (JSC::ContiguousData::ContiguousData): (JSC::ContiguousData::Data::Data): * runtime/HashMapImpl.h: (JSC::HashMapImpl::checkConsistency const): (JSC::HashMapImpl::assertBufferIsEmpty const): * runtime/JSCellInlines.h: (JSC::JSCell::methodTable const): * runtime/JSFunction.cpp: * runtime/JSFunction.h: (JSC::JSFunction::assertTypeInfoFlagInvariants): * runtime/JSGlobalObject.cpp: (JSC::JSGlobalObject::init): * runtime/JSGlobalObject.h: * runtime/JSObject.cpp: (JSC::JSObject::visitChildren): (JSC::JSFinalObject::visitChildren): * runtime/JSObjectInlines.h: (JSC::JSObject::validatePutOwnDataProperty): * runtime/JSSegmentedVariableObject.h: (JSC::JSSegmentedVariableObject::assertVariableIsInThisObject): * runtime/LiteralParser.cpp: (JSC::LiteralParser<CharType>::Lexer::lex): * runtime/LiteralParser.h: * runtime/Operations.h: (JSC::scribbleFreeCells): * runtime/OptionsList.h: * runtime/VM.cpp: (JSC::VM::computeCanUseJIT): * runtime/VM.h: (JSC::VM::canUseJIT): * runtime/VarOffset.h: (JSC::VarOffset::checkSanity const): * runtime/WeakMapImpl.h: (JSC::WeakMapImpl::checkConsistency const): (JSC::WeakMapImpl::assertBufferIsEmpty const): * wasm/WasmAirIRGenerator.cpp: (JSC::Wasm::AirIRGenerator::validateInst): * wasm/WasmB3IRGenerator.cpp: (JSC::Wasm::parseAndCompile): * wasm/WasmFunctionParser.h: (JSC::Wasm::FunctionParser::validationFail const): * wasm/WasmLLIntGenerator.cpp: (JSC::Wasm::LLIntGenerator::checkConsistency): * wasm/WasmPlan.cpp: (JSC::Wasm::Plan::tryRemoveContextAndCancelIfLast): * wasm/WasmSectionParser.h: * wasm/WasmSections.h: * wasm/WasmSignatureInlines.h: (JSC::Wasm::SignatureInformation::get): * wasm/WasmWorklist.cpp: (JSC::Wasm::Worklist::enqueue): * wasm/js/JSToWasm.cpp: (JSC::Wasm::createJSToWasmWrapper): * wasm/js/WebAssemblyFunction.cpp: (JSC::WebAssemblyFunction::previousInstanceOffset const): Source/WebCore: Convert ASSERT_DISABLED to ASSERT_ENABLED, and fix some tests of NDEBUG that should actually test for ASSERT_ENABLED. https://bugs.webkit.org/show_bug.cgi?id=205776 Reviewed by Saam Barati. * Modules/fetch/FetchBodySource.cpp: (WebCore::FetchBodySource::close): * Modules/fetch/FetchBodySource.h: * Modules/webdatabase/DatabaseDetails.h: (WebCore::DatabaseDetails::DatabaseDetails): (WebCore::DatabaseDetails::operator=): * Modules/webdatabase/DatabaseTask.cpp: (WebCore::DatabaseTask::performTask): * Modules/webdatabase/DatabaseTask.h: * Modules/webdatabase/DatabaseThread.cpp: (WebCore::DatabaseThread::terminationRequested const): * Modules/webgpu/WHLSL/AST/WHLSLAddressSpace.h: (WebCore::WHLSL::AST::TypeAnnotation::TypeAnnotation): * Modules/webgpu/WHLSL/WHLSLHighZombieFinder.cpp: (WebCore::WHLSL::findHighZombies): * Modules/webgpu/WHLSL/WHLSLInferTypes.cpp: (WebCore::WHLSL::matches): * Modules/webgpu/WHLSL/WHLSLLiteralTypeChecker.cpp: (WebCore::WHLSL::checkLiteralTypes): * Modules/webgpu/WHLSL/WHLSLSynthesizeConstructors.cpp: (WebCore::WHLSL::FindAllTypes::appendNamedType): * bindings/js/JSCallbackData.h: * bindings/js/JSLazyEventListener.cpp: * bindings/js/JSLazyEventListener.h: * contentextensions/ContentExtensionCompiler.cpp: (WebCore::ContentExtensions::compileRuleList): * css/CSSCalculationValue.cpp: (WebCore::CSSCalcOperationNode::primitiveType const): * css/CSSComputedStyleDeclaration.cpp: (WebCore::ComputedStyleExtractor::valueForPropertyInStyle): * css/CSSPrimitiveValue.cpp: * css/CSSSelector.cpp: (WebCore::CSSSelector::selectorText const): * css/CSSStyleSheet.cpp: * dom/ActiveDOMObject.cpp: (WebCore::ActiveDOMObject::suspendIfNeeded): (WebCore::ActiveDOMObject::assertSuspendIfNeededWasCalled const): * dom/ActiveDOMObject.h: * dom/ContainerNode.cpp: * dom/ContainerNodeAlgorithms.cpp: * dom/ContainerNodeAlgorithms.h: * dom/CustomElementReactionQueue.cpp: * dom/CustomElementReactionQueue.h: (WebCore::CustomElementReactionDisallowedScope::CustomElementReactionDisallowedScope): (WebCore::CustomElementReactionDisallowedScope::~CustomElementReactionDisallowedScope): * dom/Document.cpp: (WebCore::Document::hitTest): * dom/Document.h: (WebCore::Document::decrementReferencingNodeCount): * dom/Element.cpp: (WebCore::Element::addShadowRoot): (WebCore::Element::getURLAttribute const): (WebCore::Element::getNonEmptyURLAttribute const): * dom/Element.h: * dom/ElementAndTextDescendantIterator.h: (WebCore::ElementAndTextDescendantIterator::ElementAndTextDescendantIterator): (WebCore::ElementAndTextDescendantIterator::dropAssertions): (WebCore::ElementAndTextDescendantIterator::popAncestorSiblingStack): (WebCore::ElementAndTextDescendantIterator::traverseNextSibling): (WebCore::ElementAndTextDescendantIterator::traversePreviousSibling): * dom/ElementDescendantIterator.h: (WebCore::ElementDescendantIterator::ElementDescendantIterator): (WebCore::ElementDescendantIterator::dropAssertions): (WebCore::ElementDescendantIterator::operator++): (WebCore::ElementDescendantIterator::operator--): (WebCore::ElementDescendantConstIterator::ElementDescendantConstIterator): (WebCore::ElementDescendantConstIterator::dropAssertions): (WebCore::ElementDescendantConstIterator::operator++): * dom/ElementIterator.h: (WebCore::ElementIterator<ElementType>::ElementIterator): (WebCore::ElementIterator<ElementType>::traverseNext): (WebCore::ElementIterator<ElementType>::traversePrevious): (WebCore::ElementIterator<ElementType>::traverseNextSibling): (WebCore::ElementIterator<ElementType>::traversePreviousSibling): (WebCore::ElementIterator<ElementType>::traverseNextSkippingChildren): (WebCore::ElementIterator<ElementType>::dropAssertions): (WebCore::ElementIterator<ElementType>::traverseAncestor): (WebCore::ElementConstIterator<ElementType>::ElementConstIterator): (WebCore::ElementConstIterator<ElementType>::traverseNext): (WebCore::ElementConstIterator<ElementType>::traversePrevious): (WebCore::ElementConstIterator<ElementType>::traverseNextSibling): (WebCore::ElementConstIterator<ElementType>::traversePreviousSibling): (WebCore::ElementConstIterator<ElementType>::traverseNextSkippingChildren): (WebCore::ElementConstIterator<ElementType>::traverseAncestor): (WebCore::ElementConstIterator<ElementType>::dropAssertions): * dom/EventContext.cpp: * dom/EventContext.h: * dom/EventListener.h: * dom/EventPath.cpp: * dom/EventSender.h: * dom/EventTarget.cpp: (WebCore::EventTarget::addEventListener): (WebCore::EventTarget::setAttributeEventListener): (WebCore::EventTarget::innerInvokeEventListeners): * dom/Node.cpp: (WebCore::Node::~Node): (WebCore::Node::moveNodeToNewDocument): (WebCore::Node::removedLastRef): * dom/Node.h: (WebCore::Node::deref const): * dom/ScriptDisallowedScope.h: (WebCore::ScriptDisallowedScope::InMainThread::isEventDispatchAllowedInSubtree): * dom/ScriptExecutionContext.cpp: (WebCore::ScriptExecutionContext::~ScriptExecutionContext): * dom/ScriptExecutionContext.h: * dom/SelectorQuery.cpp: (WebCore::SelectorDataList::execute const): * dom/SlotAssignment.cpp: (WebCore::SlotAssignment::addSlotElementByName): (WebCore::SlotAssignment::removeSlotElementByName): (WebCore::SlotAssignment::resolveSlotsAfterSlotMutation): (WebCore::SlotAssignment::findFirstSlotElement): * dom/SlotAssignment.h: * dom/TreeScopeOrderedMap.cpp: (WebCore::TreeScopeOrderedMap::add): (WebCore::TreeScopeOrderedMap::get const): * dom/TreeScopeOrderedMap.h: * fileapi/Blob.cpp: * fileapi/Blob.h: * history/BackForwardCache.cpp: (WebCore::BackForwardCache::removeAllItemsForPage): * history/BackForwardCache.h: * html/CanvasBase.cpp: (WebCore::CanvasBase::notifyObserversCanvasDestroyed): * html/CanvasBase.h: * html/HTMLCollection.h: (WebCore::CollectionNamedElementCache::didPopulate): * html/HTMLSelectElement.cpp: (WebCore:: const): * html/HTMLTableRowsCollection.cpp: (WebCore::assertRowIsInTable): * html/HTMLTextFormControlElement.cpp: (WebCore::HTMLTextFormControlElement::indexForPosition const): * html/canvas/CanvasRenderingContext2DBase.cpp: (WebCore::CanvasRenderingContext2DBase::~CanvasRenderingContext2DBase): * html/parser/HTMLParserScheduler.cpp: (WebCore::HTMLParserScheduler::HTMLParserScheduler): (WebCore::HTMLParserScheduler::suspend): (WebCore::HTMLParserScheduler::resume): * html/parser/HTMLParserScheduler.h: * html/parser/HTMLToken.h: (WebCore::HTMLToken::beginStartTag): (WebCore::HTMLToken::beginEndTag): (WebCore::HTMLToken::endAttribute): * html/parser/HTMLTreeBuilder.cpp: (WebCore::HTMLTreeBuilder::HTMLTreeBuilder): (WebCore::HTMLTreeBuilder::constructTree): * html/parser/HTMLTreeBuilder.h: (WebCore::HTMLTreeBuilder::~HTMLTreeBuilder): * layout/FormattingContext.cpp: (WebCore::Layout::FormattingContext::geometryForBox const): * layout/blockformatting/BlockFormattingContext.cpp: (WebCore::Layout::BlockFormattingContext::computeEstimatedVerticalPosition): * layout/blockformatting/BlockFormattingContext.h: * layout/displaytree/DisplayBox.cpp: (WebCore::Display::Box::Box): * layout/displaytree/DisplayBox.h: (WebCore::Display::Box::setTopLeft): (WebCore::Display::Box::setTop): (WebCore::Display::Box::setLeft): (WebCore::Display::Box::setContentBoxHeight): (WebCore::Display::Box::setContentBoxWidth): (WebCore::Display::Box::setHorizontalMargin): (WebCore::Display::Box::setVerticalMargin): (WebCore::Display::Box::setHorizontalComputedMargin): (WebCore::Display::Box::setBorder): (WebCore::Display::Box::setPadding): * layout/displaytree/DisplayInlineRect.h: (WebCore::Display::InlineRect::InlineRect): (WebCore::Display::InlineRect::setTopLeft): (WebCore::Display::InlineRect::setTop): (WebCore::Display::InlineRect::setBottom): (WebCore::Display::InlineRect::setLeft): (WebCore::Display::InlineRect::setWidth): (WebCore::Display::InlineRect::setHeight): * layout/displaytree/DisplayLineBox.h: (WebCore::Display::LineBox::LineBox): (WebCore::Display::LineBox::setBaselineOffsetIfGreater): (WebCore::Display::LineBox::resetBaseline): (WebCore::Display::LineBox::Baseline::Baseline): (WebCore::Display::LineBox::Baseline::setAscent): (WebCore::Display::LineBox::Baseline::setDescent): (WebCore::Display::LineBox::Baseline::reset): * layout/displaytree/DisplayRect.h: (WebCore::Display::Rect::Rect): (WebCore::Display::Rect::setTopLeft): (WebCore::Display::Rect::setTop): (WebCore::Display::Rect::setLeft): (WebCore::Display::Rect::setWidth): (WebCore::Display::Rect::setHeight): (WebCore::Display::Rect::setSize): (WebCore::Display::Rect::clone const): * layout/floats/FloatingContext.cpp: * layout/inlineformatting/InlineLineBuilder.cpp: (WebCore::Layout::LineBuilder::CollapsibleContent::collapse): * layout/tableformatting/TableGrid.cpp: (WebCore::Layout::TableGrid::Column::setWidthConstraints): (WebCore::Layout::TableGrid::Column::setLogicalWidth): (WebCore::Layout::TableGrid::Column::setLogicalLeft): * layout/tableformatting/TableGrid.h: * loader/DocumentLoader.cpp: (WebCore::DocumentLoader::continueAfterContentPolicy): (WebCore::DocumentLoader::attachToFrame): (WebCore::DocumentLoader::detachFromFrame): (WebCore::DocumentLoader::addSubresourceLoader): * loader/DocumentLoader.h: * loader/ImageLoader.cpp: * loader/cache/CachedResource.h: * loader/cache/MemoryCache.cpp: (WebCore::MemoryCache::lruListFor): (WebCore::MemoryCache::removeFromLRUList): * page/FrameView.cpp: (WebCore::FrameView::updateLayoutAndStyleIfNeededRecursive): * page/FrameViewLayoutContext.cpp: * page/FrameViewLayoutContext.h: * page/Page.cpp: * page/Page.h: * page/ViewportConfiguration.cpp: * page/ViewportConfiguration.h: * page/mac/EventHandlerMac.mm: (WebCore::CurrentEventScope::CurrentEventScope): * platform/DateComponents.cpp: (WebCore::DateComponents::toStringForTime const): * platform/ScrollableArea.cpp: * platform/SharedBuffer.cpp: (WebCore::SharedBuffer::combineIntoOneSegment const): * platform/SharedBuffer.h: * platform/Supplementable.h: * platform/Timer.cpp: (WebCore::TimerBase::checkHeapIndex const): (WebCore::TimerBase::updateHeapIfNeeded): * platform/graphics/BitmapImage.cpp: * platform/graphics/BitmapImage.h: * platform/graphics/Image.h: * platform/graphics/ShadowBlur.cpp: (WebCore::ScratchBuffer::ScratchBuffer): (WebCore::ScratchBuffer::getScratchBuffer): (WebCore::ScratchBuffer::scheduleScratchBufferPurge): * platform/graphics/ca/win/CACFLayerTreeHost.cpp: (WebCore::CACFLayerTreeHost::setWindow): * platform/graphics/ca/win/CACFLayerTreeHost.h: * platform/graphics/cg/ImageBufferDataCG.cpp: (WebCore::ImageBufferData::putData): * platform/graphics/cocoa/FontCacheCoreText.cpp: * platform/graphics/gstreamer/GstAllocatorFastMalloc.cpp: (gstAllocatorFastMallocFree): * platform/graphics/nicosia/cairo/NicosiaPaintingContextCairo.cpp: (Nicosia::PaintingContextCairo::ForPainting::ForPainting): * platform/graphics/nicosia/texmap/NicosiaBackingStoreTextureMapperImpl.cpp: (Nicosia::BackingStoreTextureMapperImpl::createTile): * platform/graphics/nicosia/texmap/NicosiaContentLayerTextureMapperImpl.cpp: (Nicosia::ContentLayerTextureMapperImpl::~ContentLayerTextureMapperImpl): * platform/graphics/win/GradientDirect2D.cpp: (WebCore::Gradient::fill): * platform/graphics/win/ImageBufferDataDirect2D.cpp: (WebCore::ImageBufferData::putData): * platform/graphics/win/PathDirect2D.cpp: (WebCore::Path::appendGeometry): (WebCore::Path::Path): (WebCore::Path::operator=): (WebCore::Path::strokeContains const): (WebCore::Path::transform): * platform/graphics/win/PlatformContextDirect2D.cpp: (WebCore::PlatformContextDirect2D::setTags): * platform/mediastream/MediaStreamTrackPrivate.h: * platform/mediastream/RealtimeOutgoingAudioSource.cpp: (WebCore::RealtimeOutgoingAudioSource::~RealtimeOutgoingAudioSource): * platform/mediastream/RealtimeOutgoingVideoSource.cpp: (WebCore::RealtimeOutgoingVideoSource::~RealtimeOutgoingVideoSource): * platform/network/HTTPParsers.cpp: (WebCore::isCrossOriginSafeHeader): * platform/sql/SQLiteDatabase.cpp: * platform/sql/SQLiteDatabase.h: * platform/sql/SQLiteStatement.cpp: (WebCore::SQLiteStatement::SQLiteStatement): (WebCore::SQLiteStatement::prepare): (WebCore::SQLiteStatement::finalize): * platform/sql/SQLiteStatement.h: * platform/win/COMPtr.h: * rendering/ComplexLineLayout.cpp: (WebCore::ComplexLineLayout::removeInlineBox const): * rendering/FloatingObjects.cpp: (WebCore::FloatingObject::FloatingObject): (WebCore::FloatingObjects::addPlacedObject): (WebCore::FloatingObjects::removePlacedObject): * rendering/FloatingObjects.h: * rendering/GridTrackSizingAlgorithm.cpp: * rendering/GridTrackSizingAlgorithm.h: * rendering/LayoutDisallowedScope.cpp: * rendering/LayoutDisallowedScope.h: * rendering/RenderBlock.cpp: * rendering/RenderBlock.h: * rendering/RenderBlockFlow.cpp: (WebCore::RenderBlockFlow::layoutBlockChild): (WebCore::RenderBlockFlow::removeFloatingObject): (WebCore::RenderBlockFlow::ensureLineBoxes): * rendering/RenderBoxModelObject.cpp: * rendering/RenderDeprecatedFlexibleBox.cpp: (WebCore::RenderDeprecatedFlexibleBox::layoutBlock): * rendering/RenderElement.cpp: * rendering/RenderGeometryMap.cpp: (WebCore::RenderGeometryMap::mapToContainer const): * rendering/RenderGrid.cpp: (WebCore::RenderGrid::placeItemsOnGrid const): (WebCore::RenderGrid::baselinePosition const): * rendering/RenderInline.cpp: (WebCore::RenderInline::willBeDestroyed): * rendering/RenderLayer.cpp: (WebCore::ClipRectsCache::ClipRectsCache): (WebCore::RenderLayer::RenderLayer): (WebCore::RenderLayer::paintList): (WebCore::RenderLayer::hitTestLayer): (WebCore::RenderLayer::updateClipRects): (WebCore::RenderLayer::calculateClipRects const): * rendering/RenderLayer.h: * rendering/RenderLayerBacking.cpp: (WebCore::traverseVisibleNonCompositedDescendantLayers): * rendering/RenderLayerCompositor.cpp: (WebCore::RenderLayerCompositor::computeCompositingRequirements): (WebCore::RenderLayerCompositor::traverseUnchangedSubtree): (WebCore::RenderLayerCompositor::updateBackingAndHierarchy): (WebCore::RenderLayerCompositor::addDescendantsToOverlapMapRecursive const): (WebCore::RenderLayerCompositor::recursiveRepaintLayer): (WebCore::RenderLayerCompositor::layerHas3DContent const): * rendering/RenderLayoutState.cpp: (WebCore::RenderLayoutState::RenderLayoutState): (WebCore::RenderLayoutState::computeOffsets): (WebCore::RenderLayoutState::addLayoutDelta): * rendering/RenderLayoutState.h: (WebCore::RenderLayoutState::RenderLayoutState): * rendering/RenderObject.cpp: (WebCore::RenderObject::RenderObject): (WebCore::RenderObject::~RenderObject): (WebCore::RenderObject::clearNeedsLayout): * rendering/RenderObject.h: * rendering/RenderQuote.cpp: (WebCore::quotesForLanguage): * rendering/RenderTableCell.h: * rendering/RenderTableSection.cpp: (WebCore::RenderTableSection::computeOverflowFromCells): * rendering/RenderTextLineBoxes.cpp: (WebCore::RenderTextLineBoxes::checkConsistency const): * rendering/RenderTextLineBoxes.h: * rendering/line/BreakingContext.h: (WebCore::tryHyphenating): * rendering/style/GridArea.h: (WebCore::GridSpan::GridSpan): * rendering/style/RenderStyle.cpp: (WebCore::RenderStyle::~RenderStyle): * rendering/style/RenderStyle.h: * rendering/updating/RenderTreeBuilderRuby.cpp: (WebCore::RenderTreeBuilder::Ruby::detach): * rendering/updating/RenderTreePosition.cpp: (WebCore::RenderTreePosition::computeNextSibling): * rendering/updating/RenderTreePosition.h: * svg/SVGToOTFFontConversion.cpp: (WebCore::SVGToOTFFontConverter::Placeholder::Placeholder): (WebCore::SVGToOTFFontConverter::Placeholder::populate): (WebCore::SVGToOTFFontConverter::appendCFFTable): (WebCore::SVGToOTFFontConverter::firstGlyph const): (WebCore::SVGToOTFFontConverter::appendKERNTable): * svg/SVGTransformDistance.cpp: (WebCore::SVGTransformDistance::SVGTransformDistance): (WebCore::SVGTransformDistance::scaledDistance const): (WebCore::SVGTransformDistance::addSVGTransforms): (WebCore::SVGTransformDistance::addToSVGTransform const): (WebCore::SVGTransformDistance::distance const): * svg/graphics/SVGImage.cpp: (WebCore::SVGImage::nativeImage): * testing/InternalSettings.cpp: * workers/service/ServiceWorkerJob.h: * worklets/PaintWorkletGlobalScope.h: (WebCore::PaintWorkletGlobalScope::~PaintWorkletGlobalScope): * xml/XPathStep.cpp: Source/WebKit: Convert ASSERT_DISABLED to ASSERT_ENABLED, and fix some tests of NDEBUG that should actually test for ASSERT_ENABLED. https://bugs.webkit.org/show_bug.cgi?id=205776 Reviewed by Saam Barati. * NetworkProcess/NetworkSession.cpp: (WebKit::NetworkSession::invalidateAndCancel): * NetworkProcess/NetworkSession.h: * NetworkProcess/cache/NetworkCacheStorage.cpp: (WebKit::NetworkCache::Storage::setCapacity): * NetworkProcess/cocoa/NetworkSessionCocoa.mm: (toNSURLSessionResponseDisposition): (WebKit::NetworkSessionCocoa::NetworkSessionCocoa): * Platform/IPC/Connection.cpp: (IPC::Connection::waitForMessage): * Platform/IPC/MessageReceiver.h: (IPC::MessageReceiver::willBeAddedToMessageReceiverMap): (IPC::MessageReceiver::willBeRemovedFromMessageReceiverMap): * Platform/IPC/cocoa/ConnectionCocoa.mm: (IPC::readFromMachPort): * Platform/mac/MachUtilities.cpp: (setMachExceptionPort): * Shared/API/APIClient.h: (API::Client::Client): * Shared/API/Cocoa/WKRemoteObjectCoder.mm: * Shared/Cocoa/ArgumentCodersCocoa.h: * Shared/SharedStringHashTableReadOnly.cpp: * UIProcess/BackingStore.cpp: (WebKit::BackingStore::incorporateUpdate): * UIProcess/GenericCallback.h: * UIProcess/Launcher/mac/ProcessLauncherMac.mm: (WebKit::ProcessLauncher::launchProcess): * UIProcess/PageLoadState.h: (WebKit::PageLoadState::Transaction::Token::Token): * UIProcess/WebPageProxy.cpp: (WebKit::WebPageProxy::~WebPageProxy): * WebProcess/Network/WebResourceLoader.cpp: (WebKit::WebResourceLoader::didReceiveResponse): * WebProcess/Network/WebResourceLoader.h: * WebProcess/Plugins/Netscape/NetscapePluginStream.cpp: (WebKit::NetscapePluginStream::NetscapePluginStream): (WebKit::NetscapePluginStream::notifyAndDestroyStream): * WebProcess/Plugins/Netscape/NetscapePluginStream.h: * WebProcess/WebPage/WebPage.cpp: (WebKit::WebPage::runModal): * WebProcess/WebProcess.cpp: (WebKit::checkDocumentsCaptureStateConsistency): * WebProcess/cocoa/WebProcessCocoa.mm: (WebKit::WebProcess::updateProcessName): Source/WebKitLegacy: Convert ASSERT_DISABLED to ASSERT_ENABLED, and fix some tests of NDEBUG that should actually test for ASSERT_ENABLED. https://bugs.webkit.org/show_bug.cgi?id=205776 Reviewed by Saam Barati. * Storage/StorageAreaImpl.cpp: (WebKit::StorageAreaImpl::StorageAreaImpl): (WebKit::StorageAreaImpl::close): * Storage/StorageAreaImpl.h: Source/WebKitLegacy/mac: Convert ASSERT_DISABLED to ASSERT_ENABLED, and fix some tests of NDEBUG that should actually test for ASSERT_ENABLED. https://bugs.webkit.org/show_bug.cgi?id=205776 Reviewed by Saam Barati. * History/WebHistory.mm: (-[WebHistoryPrivate removeItemForURLString:]): * WebView/WebFrame.mm: Source/WebKitLegacy/win: Convert ASSERT_DISABLED to ASSERT_ENABLED, and fix some tests of NDEBUG that should actually test for ASSERT_ENABLED. https://bugs.webkit.org/show_bug.cgi?id=205776 Reviewed by Saam Barati. * WebKitQuartzCoreAdditions/CAD3DRenderer.cpp: (WKQCA::CAD3DRenderer::swapChain): (WKQCA::CAD3DRenderer::initialize): * WebKitQuartzCoreAdditions/CAD3DRenderer.h: * WebView.cpp: (WebView::Release): * WebView.h: Source/WTF: Convert ASSERT_DISABLED to ASSERT_ENABLED, and fix some tests of NDEBUG that should actually test for ASSERT_ENABLED. https://bugs.webkit.org/show_bug.cgi?id=205776 Reviewed by Saam Barati. This patch did the following changes: 1. Replaced ASSERT_DISABLED with ASSERT_ENABLED. This change does away with the need for the double negative !ASSERT_DISABLED test that is commonly used all over the code, thereby improving code readability. In Assertions.h, there is also BACKTRACE_DISABLED, ASSERT_MSG_DISABLED, ASSERT_ARG_DISABLED, FATAL_DISABLED, ERROR_DISABLED, LOG_DISABLED, and RELEASE_LOG_DISABLED. We should replace those with ..._ENABLED equivalents as well. We'll do that in another patch. For now, they are left as is to minimize the size of this patch. See https://bugs.webkit.org/show_bug.cgi?id=205780. 2. Fixed some code was guarded with "#ifndef NDEBUG" that should actually be guarded by "#if ASSERT_ENABLED" instead. 3. In cases where the change is minimal, we move some code around so that we can test for "#if ASSERT_ENABLED" instead of "#if !ASSERT_ENABLED". * wtf/Assertions.h: * wtf/AutomaticThread.cpp: (WTF::AutomaticThread::start): * wtf/BitVector.h: * wtf/BlockObjCExceptions.mm: (ReportBlockedObjCException): * wtf/BloomFilter.h: * wtf/CallbackAggregator.h: (WTF::CallbackAggregator::CallbackAggregator): * wtf/CheckedArithmetic.h: (WTF::observesOverflow<AssertNoOverflow>): * wtf/CheckedBoolean.h: (CheckedBoolean::CheckedBoolean): (CheckedBoolean::operator bool): * wtf/CompletionHandler.h: (WTF::CompletionHandler<Out): * wtf/DateMath.cpp: (WTF::initializeDates): * wtf/Gigacage.cpp: (Gigacage::tryAllocateZeroedVirtualPages): * wtf/HashTable.h: (WTF::KeyTraits>::checkKey): (WTF::KeyTraits>::checkTableConsistencyExceptSize const): * wtf/LoggerHelper.h: * wtf/NaturalLoops.h: (WTF::NaturalLoops::headerOf const): * wtf/NeverDestroyed.h: (WTF::LazyNeverDestroyed::construct): * wtf/OptionSet.h: (WTF::OptionSet::OptionSet): * wtf/Platform.h: * wtf/PtrTag.h: * wtf/RefCounted.h: (WTF::RefCountedBase::disableThreadingChecks): (WTF::RefCountedBase::enableThreadingChecksGlobally): (WTF::RefCountedBase::RefCountedBase): (WTF::RefCountedBase::applyRefDerefThreadingCheck const): * wtf/SingleRootGraph.h: (WTF::SingleRootGraph::assertIsConsistent const): * wtf/SizeLimits.cpp: * wtf/StackBounds.h: (WTF::StackBounds::checkConsistency const): * wtf/URLParser.cpp: (WTF::URLParser::URLParser): (WTF::URLParser::domainToASCII): * wtf/ValueCheck.h: * wtf/Vector.h: (WTF::Malloc>::checkConsistency): * wtf/WeakHashSet.h: * wtf/WeakPtr.h: (WTF::WeakPtrImpl::WeakPtrImpl): (WTF::WeakPtrFactory::WeakPtrFactory): * wtf/text/AtomStringImpl.cpp: * wtf/text/AtomStringImpl.h: * wtf/text/StringBuilder.cpp: (WTF::StringBuilder::reifyString const): * wtf/text/StringBuilder.h: * wtf/text/StringCommon.h: (WTF::hasPrefixWithLettersIgnoringASCIICaseCommon): * wtf/text/StringHasher.h: (WTF::StringHasher::addCharacters): * wtf/text/StringImpl.h: * wtf/text/SymbolImpl.h: * wtf/text/UniquedStringImpl.h: Tools: Remove WebsiteDataStore::setServiceWorkerRegistrationDirectory https://bugs.webkit.org/show_bug.cgi?id=205754 Patch by Alex Christensen <achristensen@webkit.org> on 2020-01-06 Reviewed by Youenn Fablet. * TestWebKitAPI/Tests/WebKitCocoa/ServiceWorkerBasic.mm: * WebKitTestRunner/TestController.cpp: (WTR::TestController::websiteDataStore): (WTR::TestController::platformAdjustContext): * WebKitTestRunner/cocoa/TestControllerCocoa.mm: (WTR::initializeWebViewConfiguration): Canonical link: https://commits.webkit.org/218957@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@254087 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2020-01-06 22:24:50 +00:00
#if ASSERT_ENABLED
, m_checked(false)
Implement a new allocator for backing stores https://bugs.webkit.org/show_bug.cgi?id=75181 Reviewed by Filip Pizlo. Source/JavaScriptCore: We want to move away from using fastMalloc for the backing stores for some of our objects (e.g. JSArray, JSObject, JSString, etc). These backing stores have a nice property in that they only have a single owner (i.e. a single pointer to them at any one time). One way that we can take advantage of this property is to implement a simple bump allocator/copying collector, which will run alongside our normal mark/sweep collector, that only needs to update the single owner pointer rather than having to redirect an arbitrary number of pointers in from-space to to-space. This plan can give us a number of benefits. We can beat fastMalloc in terms of both performance and memory usage, we can track how much memory we're using far more accurately than our rough estimation now through the use of reportExtraMemoryCost, and we can allocate arbitrary size objects (as opposed to being limited to size classes like we have been historically). This is also another step toward moving away from lazy destruction, which will improve our memory footprint. We start by creating said allocator and moving the ArrayStorage for JSArray to use it rather than fastMalloc. The design of the collector is as follows: Allocation: -The collector allocates 64KB chunks from the OS to use for object allocation. -Each chunk contains an offset, a flag indicating if the block has been pinned, and a payload, along with next and prev pointers so that they can be put in DoublyLinkedLists. -Any allocation greater than 64KB gets its own separate oversize block, which is managed separately from the rest. -If the allocator receives a request for more than the remaining amount in the current block, it grabs a fresh block. -Grabbing a fresh block means grabbing one off of the global free list (which is now shared between the mark/sweep allocator and the bump allocator) if there is one. If there isn't a new one we do one of two things: allocate a new block from the OS if we're not ready for a GC yet, or run a GC and then try again. If we still don't have enough space after the GC, we allocate a new block from the OS. Garbage collection: -At the start of garbage collection during conservative stack scanning, if we encounter what appears to be a pointer to a bump-allocated block of memory, we pin that block so that it will not be copied for this round of collection. -We also pin any oversize blocks that we encounter, which effectively doubles as a "mark bit" for that block. Any oversize blocks that aren't pinned at the end of copying are given back to the OS. -Marking threads are now also responsible for copying bump-allocated objects to newSpace -Each marking thread has a private 64KB block into which it copies bump-allocated objects that it encounters. -When that block fills up, the marking thread gives it back to the allocator and requests a new one. -When all marking has concluded, each thread gives back its copy block, even if it isn't full. -At the conclusion of copying (which is done by the end of the marking phase), we un-pin any pinned blocks and give any blocks left in from-space to the global free list. * CMakeLists.txt: * GNUmakefile.list.am: * JavaScriptCore.gypi: * JavaScriptCore.vcproj/JavaScriptCore/JavaScriptCore.vcproj: * JavaScriptCore.vcproj/WTF/WTF.vcproj: * JavaScriptCore.xcodeproj/project.pbxproj: * Target.pri: * heap/AllocationSpace.cpp: (JSC::AllocationSpace::allocateSlowCase): (JSC::AllocationSpace::allocateBlock): (JSC::AllocationSpace::freeBlocks): * heap/AllocationSpace.h: (JSC::AllocationSpace::waterMark): * heap/BumpBlock.h: Added. (JSC::BumpBlock::BumpBlock): * heap/BumpSpace.cpp: Added. (JSC::BumpSpace::tryAllocateSlowCase): * heap/BumpSpace.h: Added. (JSC::BumpSpace::isInCopyPhase): (JSC::BumpSpace::totalMemoryAllocated): (JSC::BumpSpace::totalMemoryUtilized): * heap/BumpSpaceInlineMethods.h: Added. (JSC::BumpSpace::BumpSpace): (JSC::BumpSpace::init): (JSC::BumpSpace::contains): (JSC::BumpSpace::pin): (JSC::BumpSpace::startedCopying): (JSC::BumpSpace::doneCopying): (JSC::BumpSpace::doneFillingBlock): (JSC::BumpSpace::recycleBlock): (JSC::BumpSpace::getFreshBlock): (JSC::BumpSpace::borrowBlock): (JSC::BumpSpace::addNewBlock): (JSC::BumpSpace::allocateNewBlock): (JSC::BumpSpace::fitsInBlock): (JSC::BumpSpace::fitsInCurrentBlock): (JSC::BumpSpace::tryAllocate): (JSC::BumpSpace::tryAllocateOversize): (JSC::BumpSpace::allocateFromBlock): (JSC::BumpSpace::tryReallocate): (JSC::BumpSpace::tryReallocateOversize): (JSC::BumpSpace::isOversize): (JSC::BumpSpace::isPinned): (JSC::BumpSpace::oversizeBlockFor): (JSC::BumpSpace::blockFor): * heap/ConservativeRoots.cpp: (JSC::ConservativeRoots::ConservativeRoots): (JSC::ConservativeRoots::genericAddPointer): (JSC::ConservativeRoots::add): * heap/ConservativeRoots.h: * heap/Heap.cpp: (JSC::Heap::Heap): (JSC::Heap::blockFreeingThreadMain): (JSC::Heap::reportExtraMemoryCostSlowCase): (JSC::Heap::getConservativeRegisterRoots): (JSC::Heap::markRoots): (JSC::Heap::collect): (JSC::Heap::releaseFreeBlocks): * heap/Heap.h: (JSC::Heap::waterMark): (JSC::Heap::highWaterMark): (JSC::Heap::setHighWaterMark): (JSC::Heap::tryAllocateStorage): (JSC::Heap::tryReallocateStorage): * heap/HeapBlock.h: Added. (JSC::HeapBlock::HeapBlock): * heap/MarkStack.cpp: (JSC::MarkStackThreadSharedData::MarkStackThreadSharedData): (JSC::SlotVisitor::drain): (JSC::SlotVisitor::drainFromShared): (JSC::SlotVisitor::startCopying): (JSC::SlotVisitor::allocateNewSpace): (JSC::SlotVisitor::copy): (JSC::SlotVisitor::copyAndAppend): (JSC::SlotVisitor::doneCopying): * heap/MarkStack.h: * heap/MarkedBlock.cpp: (JSC::MarkedBlock::recycle): (JSC::MarkedBlock::MarkedBlock): * heap/MarkedBlock.h: * heap/MarkedSpace.cpp: (JSC::MarkedSpace::MarkedSpace): * heap/MarkedSpace.h: (JSC::MarkedSpace::allocate): (JSC::MarkedSpace::forEachBlock): (JSC::MarkedSpace::SizeClass::resetAllocator): * heap/SlotVisitor.h: (JSC::SlotVisitor::SlotVisitor): * heap/TinyBloomFilter.h: (JSC::TinyBloomFilter::reset): * runtime/JSArray.cpp: (JSC::JSArray::JSArray): (JSC::JSArray::finishCreation): (JSC::JSArray::tryFinishCreationUninitialized): (JSC::JSArray::~JSArray): (JSC::JSArray::enterSparseMode): (JSC::JSArray::defineOwnNumericProperty): (JSC::JSArray::setLengthWritable): (JSC::JSArray::getOwnPropertySlotByIndex): (JSC::JSArray::getOwnPropertyDescriptor): (JSC::JSArray::putByIndexBeyondVectorLength): (JSC::JSArray::deletePropertyByIndex): (JSC::JSArray::getOwnPropertyNames): (JSC::JSArray::increaseVectorLength): (JSC::JSArray::unshiftCountSlowCase): (JSC::JSArray::setLength): (JSC::JSArray::pop): (JSC::JSArray::unshiftCount): (JSC::JSArray::visitChildren): (JSC::JSArray::sortNumeric): (JSC::JSArray::sort): (JSC::JSArray::compactForSorting): (JSC::JSArray::subclassData): (JSC::JSArray::setSubclassData): (JSC::JSArray::checkConsistency): * runtime/JSArray.h: (JSC::JSArray::inSparseMode): (JSC::JSArray::isLengthWritable): * wtf/CheckedBoolean.h: Added. (CheckedBoolean::CheckedBoolean): (CheckedBoolean::~CheckedBoolean): (CheckedBoolean::operator bool): * wtf/DoublyLinkedList.h: (WTF::::push): * wtf/StdLibExtras.h: (WTF::isPointerAligned): Source/JavaScriptGlue: Added forwarding header for new CheckedBoolean used in the bump allocator. * ForwardingHeaders/wtf/CheckedBoolean.h: Added. Source/WebCore: No new tests. Added forwarding header for new CheckedBoolean used in the bump allocator. * ForwardingHeaders/wtf/CheckedBoolean.h: Added. Canonical link: https://commits.webkit.org/93486@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@105442 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2012-01-19 21:49:56 +00:00
#endif
{
}
~CheckedBoolean()
{
ASSERT(m_checked);
}
operator bool()
{
PerformanceTests: Convert ASSERT_DISABLED to ASSERT_ENABLED, and fix some tests of NDEBUG that should actually test for ASSERT_ENABLED. https://bugs.webkit.org/show_bug.cgi?id=205776 Reviewed by Saam Barati. * JetStream2/wasm/HashSet.cpp: * StitchMarker/wtf/Assertions.h: * StitchMarker/wtf/DateMath.cpp: (WTF::initializeDates): * StitchMarker/wtf/HashTable.h: * StitchMarker/wtf/Hasher.h: (WTF::StringHasher::addCharacters): * StitchMarker/wtf/NeverDestroyed.h: (WTF::LazyNeverDestroyed::construct): * StitchMarker/wtf/StackBounds.h: (WTF::StackBounds::checkConsistency const): * StitchMarker/wtf/ValueCheck.h: * StitchMarker/wtf/Vector.h: (WTF::minCapacity>::checkConsistency): * StitchMarker/wtf/text/AtomicStringImpl.cpp: * StitchMarker/wtf/text/AtomicStringImpl.h: * StitchMarker/wtf/text/StringCommon.h: (WTF::hasPrefixWithLettersIgnoringASCIICaseCommon): * StitchMarker/wtf/text/StringImpl.h: * StitchMarker/wtf/text/SymbolImpl.h: * StitchMarker/wtf/text/UniquedStringImpl.h: Source/JavaScriptCore: Convert ASSERT_DISABLED to ASSERT_ENABLED, and fix some tests of NDEBUG that should actually test for ASSERT_ENABLED. https://bugs.webkit.org/show_bug.cgi?id=205776 Reviewed by Saam Barati. * API/tests/testapi.c: * assembler/ARM64Assembler.h: (JSC::ARM64Assembler::replaceWithLoad): (JSC::ARM64Assembler::replaceWithAddressComputation): * assembler/AssemblerBuffer.h: (JSC::AssemblerBuffer::LocalWriter::LocalWriter): * assembler/LinkBuffer.cpp: (JSC::LinkBuffer::copyCompactAndLinkCode): * assembler/ProbeStack.cpp: (JSC::Probe::Stack::Stack): * assembler/ProbeStack.h: * b3/B3FoldPathConstants.cpp: * b3/B3LowerToAir.cpp: * b3/B3MemoryValue.cpp: (JSC::B3::MemoryValue::MemoryValue): * b3/B3Opcode.cpp: * b3/B3Type.h: * b3/B3TypeMap.h: * b3/B3Width.h: * b3/air/AirAllocateRegistersAndStackAndGenerateCode.cpp: (JSC::B3::Air::GenerateAndAllocateRegisters::prepareForGeneration): (JSC::B3::Air::GenerateAndAllocateRegisters::generate): * b3/air/AirAllocateRegistersAndStackAndGenerateCode.h: * b3/air/AirAllocateRegistersByGraphColoring.cpp: * b3/air/AirArg.cpp: * b3/air/AirArg.h: * b3/air/AirCode.h: * b3/air/AirEmitShuffle.cpp: (JSC::B3::Air::emitShuffle): * builtins/BuiltinExecutables.cpp: (JSC::BuiltinExecutables::createExecutable): * bytecode/AccessCase.cpp: * bytecode/AccessCase.h: * bytecode/CallVariant.cpp: (JSC::variantListWithVariant): * bytecode/CodeBlock.cpp: (JSC::CodeBlock::ensureCatchLivenessIsComputedForBytecodeIndex): * bytecode/CodeBlockHash.cpp: (JSC::CodeBlockHash::dump const): * bytecode/StructureStubInfo.cpp: * bytecode/StructureStubInfo.h: * bytecompiler/NodesCodegen.cpp: (JSC::FunctionCallResolveNode::emitBytecode): * bytecompiler/RegisterID.h: (JSC::RegisterID::RegisterID): (JSC::RegisterID::setIndex): * debugger/Debugger.cpp: (JSC::Debugger::removeBreakpoint): * debugger/DebuggerEvalEnabler.h: (JSC::DebuggerEvalEnabler::DebuggerEvalEnabler): (JSC::DebuggerEvalEnabler::~DebuggerEvalEnabler): * dfg/DFGAbstractInterpreterInlines.h: (JSC::DFG::AbstractInterpreter<AbstractStateType>::observeTransitions): * dfg/DFGAbstractValue.cpp: * dfg/DFGAbstractValue.h: (JSC::DFG::AbstractValue::merge): (JSC::DFG::AbstractValue::checkConsistency const): (JSC::DFG::AbstractValue::assertIsRegistered const): * dfg/DFGArithMode.h: (JSC::DFG::doesOverflow): * dfg/DFGBasicBlock.cpp: (JSC::DFG::BasicBlock::BasicBlock): * dfg/DFGBasicBlock.h: (JSC::DFG::BasicBlock::didLink): * dfg/DFGCFAPhase.cpp: (JSC::DFG::CFAPhase::performBlockCFA): * dfg/DFGCommon.h: (JSC::DFG::validationEnabled): * dfg/DFGCommonData.cpp: (JSC::DFG::CommonData::finalizeCatchEntrypoints): * dfg/DFGDesiredWatchpoints.h: * dfg/DFGDoesGC.cpp: (JSC::DFG::doesGC): * dfg/DFGEdge.h: (JSC::DFG::Edge::makeWord): * dfg/DFGFixupPhase.cpp: (JSC::DFG::FixupPhase::fixupNode): * dfg/DFGJITCode.cpp: (JSC::DFG::JITCode::finalizeOSREntrypoints): * dfg/DFGObjectAllocationSinkingPhase.cpp: * dfg/DFGSSAConversionPhase.cpp: (JSC::DFG::SSAConversionPhase::run): * dfg/DFGScoreBoard.h: (JSC::DFG::ScoreBoard::assertClear): * dfg/DFGSlowPathGenerator.h: (JSC::DFG::SlowPathGenerator::generate): * dfg/DFGSpeculativeJIT.cpp: (JSC::DFG::SpeculativeJIT::compileCurrentBlock): (JSC::DFG::SpeculativeJIT::emitBinarySwitchStringRecurse): (JSC::DFG::SpeculativeJIT::emitAllocateButterfly): (JSC::DFG::SpeculativeJIT::compileAllocateNewArrayWithSize): (JSC::DFG::SpeculativeJIT::compileMakeRope): * dfg/DFGSpeculativeJIT64.cpp: (JSC::DFG::SpeculativeJIT::fillSpeculateCell): * dfg/DFGStructureAbstractValue.cpp: * dfg/DFGStructureAbstractValue.h: (JSC::DFG::StructureAbstractValue::assertIsRegistered const): * dfg/DFGVarargsForwardingPhase.cpp: * dfg/DFGVirtualRegisterAllocationPhase.cpp: (JSC::DFG::VirtualRegisterAllocationPhase::run): * ftl/FTLLink.cpp: (JSC::FTL::link): * ftl/FTLLowerDFGToB3.cpp: (JSC::FTL::DFG::LowerDFGToB3::callPreflight): (JSC::FTL::DFG::LowerDFGToB3::callCheck): (JSC::FTL::DFG::LowerDFGToB3::crash): * ftl/FTLOperations.cpp: (JSC::FTL::operationMaterializeObjectInOSR): * heap/BlockDirectory.cpp: (JSC::BlockDirectory::assertNoUnswept): * heap/GCSegmentedArray.h: (JSC::GCArraySegment::GCArraySegment): * heap/GCSegmentedArrayInlines.h: (JSC::GCSegmentedArray<T>::clear): (JSC::GCSegmentedArray<T>::expand): (JSC::GCSegmentedArray<T>::validatePrevious): * heap/HandleSet.cpp: * heap/HandleSet.h: * heap/Heap.cpp: (JSC::Heap::updateAllocationLimits): * heap/Heap.h: * heap/MarkedBlock.cpp: * heap/MarkedBlock.h: (JSC::MarkedBlock::assertValidCell const): (JSC::MarkedBlock::assertMarksNotStale): * heap/MarkedSpace.cpp: (JSC::MarkedSpace::beginMarking): (JSC::MarkedSpace::endMarking): (JSC::MarkedSpace::assertNoUnswept): * heap/PreciseAllocation.cpp: * heap/PreciseAllocation.h: (JSC::PreciseAllocation::assertValidCell const): * heap/SlotVisitor.cpp: (JSC::SlotVisitor::SlotVisitor): (JSC::SlotVisitor::appendJSCellOrAuxiliary): * heap/SlotVisitor.h: * inspector/InspectorProtocolTypes.h: (Inspector::Protocol::BindingTraits<JSON::ArrayOf<T>>::assertValueHasExpectedType): * inspector/scripts/codegen/generate_cpp_protocol_types_implementation.py: (CppProtocolTypesImplementationGenerator._generate_assertion_for_object_declaration): (CppProtocolTypesImplementationGenerator): (CppProtocolTypesImplementationGenerator._generate_assertion_for_enum): * inspector/scripts/tests/generic/expected/type-requiring-runtime-casts.json-result: * interpreter/FrameTracers.h: (JSC::JITOperationPrologueCallFrameTracer::JITOperationPrologueCallFrameTracer): * interpreter/Interpreter.cpp: (JSC::Interpreter::Interpreter): * interpreter/Interpreter.h: * jit/AssemblyHelpers.cpp: (JSC::AssemblyHelpers::emitStoreStructureWithTypeInfo): * jit/AssemblyHelpers.h: (JSC::AssemblyHelpers::prepareCallOperation): * jit/BinarySwitch.cpp: (JSC::BinarySwitch::BinarySwitch): * jit/CCallHelpers.h: (JSC::CCallHelpers::setupStubArgs): * jit/CallFrameShuffler.cpp: (JSC::CallFrameShuffler::emitDeltaCheck): (JSC::CallFrameShuffler::prepareAny): * jit/JIT.cpp: (JSC::JIT::assertStackPointerOffset): (JSC::JIT::compileWithoutLinking): * jit/JITOpcodes.cpp: (JSC::JIT::emitSlow_op_loop_hint): * jit/JITPropertyAccess.cpp: (JSC::JIT::emit_op_get_from_scope): * jit/JITPropertyAccess32_64.cpp: (JSC::JIT::emit_op_get_from_scope): * jit/Repatch.cpp: (JSC::linkPolymorphicCall): * jit/ThunkGenerators.cpp: (JSC::emitPointerValidation): * llint/LLIntData.cpp: (JSC::LLInt::Data::performAssertions): * llint/LLIntOfflineAsmConfig.h: * parser/Lexer.cpp: * parser/Lexer.h: (JSC::isSafeBuiltinIdentifier): (JSC::Lexer<T>::lexExpectIdentifier): * runtime/ArgList.h: (JSC::MarkedArgumentBuffer::setNeedsOverflowCheck): (JSC::MarkedArgumentBuffer::clearNeedsOverflowCheck): * runtime/Butterfly.h: (JSC::ContiguousData::ContiguousData): (JSC::ContiguousData::Data::Data): * runtime/HashMapImpl.h: (JSC::HashMapImpl::checkConsistency const): (JSC::HashMapImpl::assertBufferIsEmpty const): * runtime/JSCellInlines.h: (JSC::JSCell::methodTable const): * runtime/JSFunction.cpp: * runtime/JSFunction.h: (JSC::JSFunction::assertTypeInfoFlagInvariants): * runtime/JSGlobalObject.cpp: (JSC::JSGlobalObject::init): * runtime/JSGlobalObject.h: * runtime/JSObject.cpp: (JSC::JSObject::visitChildren): (JSC::JSFinalObject::visitChildren): * runtime/JSObjectInlines.h: (JSC::JSObject::validatePutOwnDataProperty): * runtime/JSSegmentedVariableObject.h: (JSC::JSSegmentedVariableObject::assertVariableIsInThisObject): * runtime/LiteralParser.cpp: (JSC::LiteralParser<CharType>::Lexer::lex): * runtime/LiteralParser.h: * runtime/Operations.h: (JSC::scribbleFreeCells): * runtime/OptionsList.h: * runtime/VM.cpp: (JSC::VM::computeCanUseJIT): * runtime/VM.h: (JSC::VM::canUseJIT): * runtime/VarOffset.h: (JSC::VarOffset::checkSanity const): * runtime/WeakMapImpl.h: (JSC::WeakMapImpl::checkConsistency const): (JSC::WeakMapImpl::assertBufferIsEmpty const): * wasm/WasmAirIRGenerator.cpp: (JSC::Wasm::AirIRGenerator::validateInst): * wasm/WasmB3IRGenerator.cpp: (JSC::Wasm::parseAndCompile): * wasm/WasmFunctionParser.h: (JSC::Wasm::FunctionParser::validationFail const): * wasm/WasmLLIntGenerator.cpp: (JSC::Wasm::LLIntGenerator::checkConsistency): * wasm/WasmPlan.cpp: (JSC::Wasm::Plan::tryRemoveContextAndCancelIfLast): * wasm/WasmSectionParser.h: * wasm/WasmSections.h: * wasm/WasmSignatureInlines.h: (JSC::Wasm::SignatureInformation::get): * wasm/WasmWorklist.cpp: (JSC::Wasm::Worklist::enqueue): * wasm/js/JSToWasm.cpp: (JSC::Wasm::createJSToWasmWrapper): * wasm/js/WebAssemblyFunction.cpp: (JSC::WebAssemblyFunction::previousInstanceOffset const): Source/WebCore: Convert ASSERT_DISABLED to ASSERT_ENABLED, and fix some tests of NDEBUG that should actually test for ASSERT_ENABLED. https://bugs.webkit.org/show_bug.cgi?id=205776 Reviewed by Saam Barati. * Modules/fetch/FetchBodySource.cpp: (WebCore::FetchBodySource::close): * Modules/fetch/FetchBodySource.h: * Modules/webdatabase/DatabaseDetails.h: (WebCore::DatabaseDetails::DatabaseDetails): (WebCore::DatabaseDetails::operator=): * Modules/webdatabase/DatabaseTask.cpp: (WebCore::DatabaseTask::performTask): * Modules/webdatabase/DatabaseTask.h: * Modules/webdatabase/DatabaseThread.cpp: (WebCore::DatabaseThread::terminationRequested const): * Modules/webgpu/WHLSL/AST/WHLSLAddressSpace.h: (WebCore::WHLSL::AST::TypeAnnotation::TypeAnnotation): * Modules/webgpu/WHLSL/WHLSLHighZombieFinder.cpp: (WebCore::WHLSL::findHighZombies): * Modules/webgpu/WHLSL/WHLSLInferTypes.cpp: (WebCore::WHLSL::matches): * Modules/webgpu/WHLSL/WHLSLLiteralTypeChecker.cpp: (WebCore::WHLSL::checkLiteralTypes): * Modules/webgpu/WHLSL/WHLSLSynthesizeConstructors.cpp: (WebCore::WHLSL::FindAllTypes::appendNamedType): * bindings/js/JSCallbackData.h: * bindings/js/JSLazyEventListener.cpp: * bindings/js/JSLazyEventListener.h: * contentextensions/ContentExtensionCompiler.cpp: (WebCore::ContentExtensions::compileRuleList): * css/CSSCalculationValue.cpp: (WebCore::CSSCalcOperationNode::primitiveType const): * css/CSSComputedStyleDeclaration.cpp: (WebCore::ComputedStyleExtractor::valueForPropertyInStyle): * css/CSSPrimitiveValue.cpp: * css/CSSSelector.cpp: (WebCore::CSSSelector::selectorText const): * css/CSSStyleSheet.cpp: * dom/ActiveDOMObject.cpp: (WebCore::ActiveDOMObject::suspendIfNeeded): (WebCore::ActiveDOMObject::assertSuspendIfNeededWasCalled const): * dom/ActiveDOMObject.h: * dom/ContainerNode.cpp: * dom/ContainerNodeAlgorithms.cpp: * dom/ContainerNodeAlgorithms.h: * dom/CustomElementReactionQueue.cpp: * dom/CustomElementReactionQueue.h: (WebCore::CustomElementReactionDisallowedScope::CustomElementReactionDisallowedScope): (WebCore::CustomElementReactionDisallowedScope::~CustomElementReactionDisallowedScope): * dom/Document.cpp: (WebCore::Document::hitTest): * dom/Document.h: (WebCore::Document::decrementReferencingNodeCount): * dom/Element.cpp: (WebCore::Element::addShadowRoot): (WebCore::Element::getURLAttribute const): (WebCore::Element::getNonEmptyURLAttribute const): * dom/Element.h: * dom/ElementAndTextDescendantIterator.h: (WebCore::ElementAndTextDescendantIterator::ElementAndTextDescendantIterator): (WebCore::ElementAndTextDescendantIterator::dropAssertions): (WebCore::ElementAndTextDescendantIterator::popAncestorSiblingStack): (WebCore::ElementAndTextDescendantIterator::traverseNextSibling): (WebCore::ElementAndTextDescendantIterator::traversePreviousSibling): * dom/ElementDescendantIterator.h: (WebCore::ElementDescendantIterator::ElementDescendantIterator): (WebCore::ElementDescendantIterator::dropAssertions): (WebCore::ElementDescendantIterator::operator++): (WebCore::ElementDescendantIterator::operator--): (WebCore::ElementDescendantConstIterator::ElementDescendantConstIterator): (WebCore::ElementDescendantConstIterator::dropAssertions): (WebCore::ElementDescendantConstIterator::operator++): * dom/ElementIterator.h: (WebCore::ElementIterator<ElementType>::ElementIterator): (WebCore::ElementIterator<ElementType>::traverseNext): (WebCore::ElementIterator<ElementType>::traversePrevious): (WebCore::ElementIterator<ElementType>::traverseNextSibling): (WebCore::ElementIterator<ElementType>::traversePreviousSibling): (WebCore::ElementIterator<ElementType>::traverseNextSkippingChildren): (WebCore::ElementIterator<ElementType>::dropAssertions): (WebCore::ElementIterator<ElementType>::traverseAncestor): (WebCore::ElementConstIterator<ElementType>::ElementConstIterator): (WebCore::ElementConstIterator<ElementType>::traverseNext): (WebCore::ElementConstIterator<ElementType>::traversePrevious): (WebCore::ElementConstIterator<ElementType>::traverseNextSibling): (WebCore::ElementConstIterator<ElementType>::traversePreviousSibling): (WebCore::ElementConstIterator<ElementType>::traverseNextSkippingChildren): (WebCore::ElementConstIterator<ElementType>::traverseAncestor): (WebCore::ElementConstIterator<ElementType>::dropAssertions): * dom/EventContext.cpp: * dom/EventContext.h: * dom/EventListener.h: * dom/EventPath.cpp: * dom/EventSender.h: * dom/EventTarget.cpp: (WebCore::EventTarget::addEventListener): (WebCore::EventTarget::setAttributeEventListener): (WebCore::EventTarget::innerInvokeEventListeners): * dom/Node.cpp: (WebCore::Node::~Node): (WebCore::Node::moveNodeToNewDocument): (WebCore::Node::removedLastRef): * dom/Node.h: (WebCore::Node::deref const): * dom/ScriptDisallowedScope.h: (WebCore::ScriptDisallowedScope::InMainThread::isEventDispatchAllowedInSubtree): * dom/ScriptExecutionContext.cpp: (WebCore::ScriptExecutionContext::~ScriptExecutionContext): * dom/ScriptExecutionContext.h: * dom/SelectorQuery.cpp: (WebCore::SelectorDataList::execute const): * dom/SlotAssignment.cpp: (WebCore::SlotAssignment::addSlotElementByName): (WebCore::SlotAssignment::removeSlotElementByName): (WebCore::SlotAssignment::resolveSlotsAfterSlotMutation): (WebCore::SlotAssignment::findFirstSlotElement): * dom/SlotAssignment.h: * dom/TreeScopeOrderedMap.cpp: (WebCore::TreeScopeOrderedMap::add): (WebCore::TreeScopeOrderedMap::get const): * dom/TreeScopeOrderedMap.h: * fileapi/Blob.cpp: * fileapi/Blob.h: * history/BackForwardCache.cpp: (WebCore::BackForwardCache::removeAllItemsForPage): * history/BackForwardCache.h: * html/CanvasBase.cpp: (WebCore::CanvasBase::notifyObserversCanvasDestroyed): * html/CanvasBase.h: * html/HTMLCollection.h: (WebCore::CollectionNamedElementCache::didPopulate): * html/HTMLSelectElement.cpp: (WebCore:: const): * html/HTMLTableRowsCollection.cpp: (WebCore::assertRowIsInTable): * html/HTMLTextFormControlElement.cpp: (WebCore::HTMLTextFormControlElement::indexForPosition const): * html/canvas/CanvasRenderingContext2DBase.cpp: (WebCore::CanvasRenderingContext2DBase::~CanvasRenderingContext2DBase): * html/parser/HTMLParserScheduler.cpp: (WebCore::HTMLParserScheduler::HTMLParserScheduler): (WebCore::HTMLParserScheduler::suspend): (WebCore::HTMLParserScheduler::resume): * html/parser/HTMLParserScheduler.h: * html/parser/HTMLToken.h: (WebCore::HTMLToken::beginStartTag): (WebCore::HTMLToken::beginEndTag): (WebCore::HTMLToken::endAttribute): * html/parser/HTMLTreeBuilder.cpp: (WebCore::HTMLTreeBuilder::HTMLTreeBuilder): (WebCore::HTMLTreeBuilder::constructTree): * html/parser/HTMLTreeBuilder.h: (WebCore::HTMLTreeBuilder::~HTMLTreeBuilder): * layout/FormattingContext.cpp: (WebCore::Layout::FormattingContext::geometryForBox const): * layout/blockformatting/BlockFormattingContext.cpp: (WebCore::Layout::BlockFormattingContext::computeEstimatedVerticalPosition): * layout/blockformatting/BlockFormattingContext.h: * layout/displaytree/DisplayBox.cpp: (WebCore::Display::Box::Box): * layout/displaytree/DisplayBox.h: (WebCore::Display::Box::setTopLeft): (WebCore::Display::Box::setTop): (WebCore::Display::Box::setLeft): (WebCore::Display::Box::setContentBoxHeight): (WebCore::Display::Box::setContentBoxWidth): (WebCore::Display::Box::setHorizontalMargin): (WebCore::Display::Box::setVerticalMargin): (WebCore::Display::Box::setHorizontalComputedMargin): (WebCore::Display::Box::setBorder): (WebCore::Display::Box::setPadding): * layout/displaytree/DisplayInlineRect.h: (WebCore::Display::InlineRect::InlineRect): (WebCore::Display::InlineRect::setTopLeft): (WebCore::Display::InlineRect::setTop): (WebCore::Display::InlineRect::setBottom): (WebCore::Display::InlineRect::setLeft): (WebCore::Display::InlineRect::setWidth): (WebCore::Display::InlineRect::setHeight): * layout/displaytree/DisplayLineBox.h: (WebCore::Display::LineBox::LineBox): (WebCore::Display::LineBox::setBaselineOffsetIfGreater): (WebCore::Display::LineBox::resetBaseline): (WebCore::Display::LineBox::Baseline::Baseline): (WebCore::Display::LineBox::Baseline::setAscent): (WebCore::Display::LineBox::Baseline::setDescent): (WebCore::Display::LineBox::Baseline::reset): * layout/displaytree/DisplayRect.h: (WebCore::Display::Rect::Rect): (WebCore::Display::Rect::setTopLeft): (WebCore::Display::Rect::setTop): (WebCore::Display::Rect::setLeft): (WebCore::Display::Rect::setWidth): (WebCore::Display::Rect::setHeight): (WebCore::Display::Rect::setSize): (WebCore::Display::Rect::clone const): * layout/floats/FloatingContext.cpp: * layout/inlineformatting/InlineLineBuilder.cpp: (WebCore::Layout::LineBuilder::CollapsibleContent::collapse): * layout/tableformatting/TableGrid.cpp: (WebCore::Layout::TableGrid::Column::setWidthConstraints): (WebCore::Layout::TableGrid::Column::setLogicalWidth): (WebCore::Layout::TableGrid::Column::setLogicalLeft): * layout/tableformatting/TableGrid.h: * loader/DocumentLoader.cpp: (WebCore::DocumentLoader::continueAfterContentPolicy): (WebCore::DocumentLoader::attachToFrame): (WebCore::DocumentLoader::detachFromFrame): (WebCore::DocumentLoader::addSubresourceLoader): * loader/DocumentLoader.h: * loader/ImageLoader.cpp: * loader/cache/CachedResource.h: * loader/cache/MemoryCache.cpp: (WebCore::MemoryCache::lruListFor): (WebCore::MemoryCache::removeFromLRUList): * page/FrameView.cpp: (WebCore::FrameView::updateLayoutAndStyleIfNeededRecursive): * page/FrameViewLayoutContext.cpp: * page/FrameViewLayoutContext.h: * page/Page.cpp: * page/Page.h: * page/ViewportConfiguration.cpp: * page/ViewportConfiguration.h: * page/mac/EventHandlerMac.mm: (WebCore::CurrentEventScope::CurrentEventScope): * platform/DateComponents.cpp: (WebCore::DateComponents::toStringForTime const): * platform/ScrollableArea.cpp: * platform/SharedBuffer.cpp: (WebCore::SharedBuffer::combineIntoOneSegment const): * platform/SharedBuffer.h: * platform/Supplementable.h: * platform/Timer.cpp: (WebCore::TimerBase::checkHeapIndex const): (WebCore::TimerBase::updateHeapIfNeeded): * platform/graphics/BitmapImage.cpp: * platform/graphics/BitmapImage.h: * platform/graphics/Image.h: * platform/graphics/ShadowBlur.cpp: (WebCore::ScratchBuffer::ScratchBuffer): (WebCore::ScratchBuffer::getScratchBuffer): (WebCore::ScratchBuffer::scheduleScratchBufferPurge): * platform/graphics/ca/win/CACFLayerTreeHost.cpp: (WebCore::CACFLayerTreeHost::setWindow): * platform/graphics/ca/win/CACFLayerTreeHost.h: * platform/graphics/cg/ImageBufferDataCG.cpp: (WebCore::ImageBufferData::putData): * platform/graphics/cocoa/FontCacheCoreText.cpp: * platform/graphics/gstreamer/GstAllocatorFastMalloc.cpp: (gstAllocatorFastMallocFree): * platform/graphics/nicosia/cairo/NicosiaPaintingContextCairo.cpp: (Nicosia::PaintingContextCairo::ForPainting::ForPainting): * platform/graphics/nicosia/texmap/NicosiaBackingStoreTextureMapperImpl.cpp: (Nicosia::BackingStoreTextureMapperImpl::createTile): * platform/graphics/nicosia/texmap/NicosiaContentLayerTextureMapperImpl.cpp: (Nicosia::ContentLayerTextureMapperImpl::~ContentLayerTextureMapperImpl): * platform/graphics/win/GradientDirect2D.cpp: (WebCore::Gradient::fill): * platform/graphics/win/ImageBufferDataDirect2D.cpp: (WebCore::ImageBufferData::putData): * platform/graphics/win/PathDirect2D.cpp: (WebCore::Path::appendGeometry): (WebCore::Path::Path): (WebCore::Path::operator=): (WebCore::Path::strokeContains const): (WebCore::Path::transform): * platform/graphics/win/PlatformContextDirect2D.cpp: (WebCore::PlatformContextDirect2D::setTags): * platform/mediastream/MediaStreamTrackPrivate.h: * platform/mediastream/RealtimeOutgoingAudioSource.cpp: (WebCore::RealtimeOutgoingAudioSource::~RealtimeOutgoingAudioSource): * platform/mediastream/RealtimeOutgoingVideoSource.cpp: (WebCore::RealtimeOutgoingVideoSource::~RealtimeOutgoingVideoSource): * platform/network/HTTPParsers.cpp: (WebCore::isCrossOriginSafeHeader): * platform/sql/SQLiteDatabase.cpp: * platform/sql/SQLiteDatabase.h: * platform/sql/SQLiteStatement.cpp: (WebCore::SQLiteStatement::SQLiteStatement): (WebCore::SQLiteStatement::prepare): (WebCore::SQLiteStatement::finalize): * platform/sql/SQLiteStatement.h: * platform/win/COMPtr.h: * rendering/ComplexLineLayout.cpp: (WebCore::ComplexLineLayout::removeInlineBox const): * rendering/FloatingObjects.cpp: (WebCore::FloatingObject::FloatingObject): (WebCore::FloatingObjects::addPlacedObject): (WebCore::FloatingObjects::removePlacedObject): * rendering/FloatingObjects.h: * rendering/GridTrackSizingAlgorithm.cpp: * rendering/GridTrackSizingAlgorithm.h: * rendering/LayoutDisallowedScope.cpp: * rendering/LayoutDisallowedScope.h: * rendering/RenderBlock.cpp: * rendering/RenderBlock.h: * rendering/RenderBlockFlow.cpp: (WebCore::RenderBlockFlow::layoutBlockChild): (WebCore::RenderBlockFlow::removeFloatingObject): (WebCore::RenderBlockFlow::ensureLineBoxes): * rendering/RenderBoxModelObject.cpp: * rendering/RenderDeprecatedFlexibleBox.cpp: (WebCore::RenderDeprecatedFlexibleBox::layoutBlock): * rendering/RenderElement.cpp: * rendering/RenderGeometryMap.cpp: (WebCore::RenderGeometryMap::mapToContainer const): * rendering/RenderGrid.cpp: (WebCore::RenderGrid::placeItemsOnGrid const): (WebCore::RenderGrid::baselinePosition const): * rendering/RenderInline.cpp: (WebCore::RenderInline::willBeDestroyed): * rendering/RenderLayer.cpp: (WebCore::ClipRectsCache::ClipRectsCache): (WebCore::RenderLayer::RenderLayer): (WebCore::RenderLayer::paintList): (WebCore::RenderLayer::hitTestLayer): (WebCore::RenderLayer::updateClipRects): (WebCore::RenderLayer::calculateClipRects const): * rendering/RenderLayer.h: * rendering/RenderLayerBacking.cpp: (WebCore::traverseVisibleNonCompositedDescendantLayers): * rendering/RenderLayerCompositor.cpp: (WebCore::RenderLayerCompositor::computeCompositingRequirements): (WebCore::RenderLayerCompositor::traverseUnchangedSubtree): (WebCore::RenderLayerCompositor::updateBackingAndHierarchy): (WebCore::RenderLayerCompositor::addDescendantsToOverlapMapRecursive const): (WebCore::RenderLayerCompositor::recursiveRepaintLayer): (WebCore::RenderLayerCompositor::layerHas3DContent const): * rendering/RenderLayoutState.cpp: (WebCore::RenderLayoutState::RenderLayoutState): (WebCore::RenderLayoutState::computeOffsets): (WebCore::RenderLayoutState::addLayoutDelta): * rendering/RenderLayoutState.h: (WebCore::RenderLayoutState::RenderLayoutState): * rendering/RenderObject.cpp: (WebCore::RenderObject::RenderObject): (WebCore::RenderObject::~RenderObject): (WebCore::RenderObject::clearNeedsLayout): * rendering/RenderObject.h: * rendering/RenderQuote.cpp: (WebCore::quotesForLanguage): * rendering/RenderTableCell.h: * rendering/RenderTableSection.cpp: (WebCore::RenderTableSection::computeOverflowFromCells): * rendering/RenderTextLineBoxes.cpp: (WebCore::RenderTextLineBoxes::checkConsistency const): * rendering/RenderTextLineBoxes.h: * rendering/line/BreakingContext.h: (WebCore::tryHyphenating): * rendering/style/GridArea.h: (WebCore::GridSpan::GridSpan): * rendering/style/RenderStyle.cpp: (WebCore::RenderStyle::~RenderStyle): * rendering/style/RenderStyle.h: * rendering/updating/RenderTreeBuilderRuby.cpp: (WebCore::RenderTreeBuilder::Ruby::detach): * rendering/updating/RenderTreePosition.cpp: (WebCore::RenderTreePosition::computeNextSibling): * rendering/updating/RenderTreePosition.h: * svg/SVGToOTFFontConversion.cpp: (WebCore::SVGToOTFFontConverter::Placeholder::Placeholder): (WebCore::SVGToOTFFontConverter::Placeholder::populate): (WebCore::SVGToOTFFontConverter::appendCFFTable): (WebCore::SVGToOTFFontConverter::firstGlyph const): (WebCore::SVGToOTFFontConverter::appendKERNTable): * svg/SVGTransformDistance.cpp: (WebCore::SVGTransformDistance::SVGTransformDistance): (WebCore::SVGTransformDistance::scaledDistance const): (WebCore::SVGTransformDistance::addSVGTransforms): (WebCore::SVGTransformDistance::addToSVGTransform const): (WebCore::SVGTransformDistance::distance const): * svg/graphics/SVGImage.cpp: (WebCore::SVGImage::nativeImage): * testing/InternalSettings.cpp: * workers/service/ServiceWorkerJob.h: * worklets/PaintWorkletGlobalScope.h: (WebCore::PaintWorkletGlobalScope::~PaintWorkletGlobalScope): * xml/XPathStep.cpp: Source/WebKit: Convert ASSERT_DISABLED to ASSERT_ENABLED, and fix some tests of NDEBUG that should actually test for ASSERT_ENABLED. https://bugs.webkit.org/show_bug.cgi?id=205776 Reviewed by Saam Barati. * NetworkProcess/NetworkSession.cpp: (WebKit::NetworkSession::invalidateAndCancel): * NetworkProcess/NetworkSession.h: * NetworkProcess/cache/NetworkCacheStorage.cpp: (WebKit::NetworkCache::Storage::setCapacity): * NetworkProcess/cocoa/NetworkSessionCocoa.mm: (toNSURLSessionResponseDisposition): (WebKit::NetworkSessionCocoa::NetworkSessionCocoa): * Platform/IPC/Connection.cpp: (IPC::Connection::waitForMessage): * Platform/IPC/MessageReceiver.h: (IPC::MessageReceiver::willBeAddedToMessageReceiverMap): (IPC::MessageReceiver::willBeRemovedFromMessageReceiverMap): * Platform/IPC/cocoa/ConnectionCocoa.mm: (IPC::readFromMachPort): * Platform/mac/MachUtilities.cpp: (setMachExceptionPort): * Shared/API/APIClient.h: (API::Client::Client): * Shared/API/Cocoa/WKRemoteObjectCoder.mm: * Shared/Cocoa/ArgumentCodersCocoa.h: * Shared/SharedStringHashTableReadOnly.cpp: * UIProcess/BackingStore.cpp: (WebKit::BackingStore::incorporateUpdate): * UIProcess/GenericCallback.h: * UIProcess/Launcher/mac/ProcessLauncherMac.mm: (WebKit::ProcessLauncher::launchProcess): * UIProcess/PageLoadState.h: (WebKit::PageLoadState::Transaction::Token::Token): * UIProcess/WebPageProxy.cpp: (WebKit::WebPageProxy::~WebPageProxy): * WebProcess/Network/WebResourceLoader.cpp: (WebKit::WebResourceLoader::didReceiveResponse): * WebProcess/Network/WebResourceLoader.h: * WebProcess/Plugins/Netscape/NetscapePluginStream.cpp: (WebKit::NetscapePluginStream::NetscapePluginStream): (WebKit::NetscapePluginStream::notifyAndDestroyStream): * WebProcess/Plugins/Netscape/NetscapePluginStream.h: * WebProcess/WebPage/WebPage.cpp: (WebKit::WebPage::runModal): * WebProcess/WebProcess.cpp: (WebKit::checkDocumentsCaptureStateConsistency): * WebProcess/cocoa/WebProcessCocoa.mm: (WebKit::WebProcess::updateProcessName): Source/WebKitLegacy: Convert ASSERT_DISABLED to ASSERT_ENABLED, and fix some tests of NDEBUG that should actually test for ASSERT_ENABLED. https://bugs.webkit.org/show_bug.cgi?id=205776 Reviewed by Saam Barati. * Storage/StorageAreaImpl.cpp: (WebKit::StorageAreaImpl::StorageAreaImpl): (WebKit::StorageAreaImpl::close): * Storage/StorageAreaImpl.h: Source/WebKitLegacy/mac: Convert ASSERT_DISABLED to ASSERT_ENABLED, and fix some tests of NDEBUG that should actually test for ASSERT_ENABLED. https://bugs.webkit.org/show_bug.cgi?id=205776 Reviewed by Saam Barati. * History/WebHistory.mm: (-[WebHistoryPrivate removeItemForURLString:]): * WebView/WebFrame.mm: Source/WebKitLegacy/win: Convert ASSERT_DISABLED to ASSERT_ENABLED, and fix some tests of NDEBUG that should actually test for ASSERT_ENABLED. https://bugs.webkit.org/show_bug.cgi?id=205776 Reviewed by Saam Barati. * WebKitQuartzCoreAdditions/CAD3DRenderer.cpp: (WKQCA::CAD3DRenderer::swapChain): (WKQCA::CAD3DRenderer::initialize): * WebKitQuartzCoreAdditions/CAD3DRenderer.h: * WebView.cpp: (WebView::Release): * WebView.h: Source/WTF: Convert ASSERT_DISABLED to ASSERT_ENABLED, and fix some tests of NDEBUG that should actually test for ASSERT_ENABLED. https://bugs.webkit.org/show_bug.cgi?id=205776 Reviewed by Saam Barati. This patch did the following changes: 1. Replaced ASSERT_DISABLED with ASSERT_ENABLED. This change does away with the need for the double negative !ASSERT_DISABLED test that is commonly used all over the code, thereby improving code readability. In Assertions.h, there is also BACKTRACE_DISABLED, ASSERT_MSG_DISABLED, ASSERT_ARG_DISABLED, FATAL_DISABLED, ERROR_DISABLED, LOG_DISABLED, and RELEASE_LOG_DISABLED. We should replace those with ..._ENABLED equivalents as well. We'll do that in another patch. For now, they are left as is to minimize the size of this patch. See https://bugs.webkit.org/show_bug.cgi?id=205780. 2. Fixed some code was guarded with "#ifndef NDEBUG" that should actually be guarded by "#if ASSERT_ENABLED" instead. 3. In cases where the change is minimal, we move some code around so that we can test for "#if ASSERT_ENABLED" instead of "#if !ASSERT_ENABLED". * wtf/Assertions.h: * wtf/AutomaticThread.cpp: (WTF::AutomaticThread::start): * wtf/BitVector.h: * wtf/BlockObjCExceptions.mm: (ReportBlockedObjCException): * wtf/BloomFilter.h: * wtf/CallbackAggregator.h: (WTF::CallbackAggregator::CallbackAggregator): * wtf/CheckedArithmetic.h: (WTF::observesOverflow<AssertNoOverflow>): * wtf/CheckedBoolean.h: (CheckedBoolean::CheckedBoolean): (CheckedBoolean::operator bool): * wtf/CompletionHandler.h: (WTF::CompletionHandler<Out): * wtf/DateMath.cpp: (WTF::initializeDates): * wtf/Gigacage.cpp: (Gigacage::tryAllocateZeroedVirtualPages): * wtf/HashTable.h: (WTF::KeyTraits>::checkKey): (WTF::KeyTraits>::checkTableConsistencyExceptSize const): * wtf/LoggerHelper.h: * wtf/NaturalLoops.h: (WTF::NaturalLoops::headerOf const): * wtf/NeverDestroyed.h: (WTF::LazyNeverDestroyed::construct): * wtf/OptionSet.h: (WTF::OptionSet::OptionSet): * wtf/Platform.h: * wtf/PtrTag.h: * wtf/RefCounted.h: (WTF::RefCountedBase::disableThreadingChecks): (WTF::RefCountedBase::enableThreadingChecksGlobally): (WTF::RefCountedBase::RefCountedBase): (WTF::RefCountedBase::applyRefDerefThreadingCheck const): * wtf/SingleRootGraph.h: (WTF::SingleRootGraph::assertIsConsistent const): * wtf/SizeLimits.cpp: * wtf/StackBounds.h: (WTF::StackBounds::checkConsistency const): * wtf/URLParser.cpp: (WTF::URLParser::URLParser): (WTF::URLParser::domainToASCII): * wtf/ValueCheck.h: * wtf/Vector.h: (WTF::Malloc>::checkConsistency): * wtf/WeakHashSet.h: * wtf/WeakPtr.h: (WTF::WeakPtrImpl::WeakPtrImpl): (WTF::WeakPtrFactory::WeakPtrFactory): * wtf/text/AtomStringImpl.cpp: * wtf/text/AtomStringImpl.h: * wtf/text/StringBuilder.cpp: (WTF::StringBuilder::reifyString const): * wtf/text/StringBuilder.h: * wtf/text/StringCommon.h: (WTF::hasPrefixWithLettersIgnoringASCIICaseCommon): * wtf/text/StringHasher.h: (WTF::StringHasher::addCharacters): * wtf/text/StringImpl.h: * wtf/text/SymbolImpl.h: * wtf/text/UniquedStringImpl.h: Tools: Remove WebsiteDataStore::setServiceWorkerRegistrationDirectory https://bugs.webkit.org/show_bug.cgi?id=205754 Patch by Alex Christensen <achristensen@webkit.org> on 2020-01-06 Reviewed by Youenn Fablet. * TestWebKitAPI/Tests/WebKitCocoa/ServiceWorkerBasic.mm: * WebKitTestRunner/TestController.cpp: (WTR::TestController::websiteDataStore): (WTR::TestController::platformAdjustContext): * WebKitTestRunner/cocoa/TestControllerCocoa.mm: (WTR::initializeWebViewConfiguration): Canonical link: https://commits.webkit.org/218957@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@254087 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2020-01-06 22:24:50 +00:00
#if ASSERT_ENABLED
Implement a new allocator for backing stores https://bugs.webkit.org/show_bug.cgi?id=75181 Reviewed by Filip Pizlo. Source/JavaScriptCore: We want to move away from using fastMalloc for the backing stores for some of our objects (e.g. JSArray, JSObject, JSString, etc). These backing stores have a nice property in that they only have a single owner (i.e. a single pointer to them at any one time). One way that we can take advantage of this property is to implement a simple bump allocator/copying collector, which will run alongside our normal mark/sweep collector, that only needs to update the single owner pointer rather than having to redirect an arbitrary number of pointers in from-space to to-space. This plan can give us a number of benefits. We can beat fastMalloc in terms of both performance and memory usage, we can track how much memory we're using far more accurately than our rough estimation now through the use of reportExtraMemoryCost, and we can allocate arbitrary size objects (as opposed to being limited to size classes like we have been historically). This is also another step toward moving away from lazy destruction, which will improve our memory footprint. We start by creating said allocator and moving the ArrayStorage for JSArray to use it rather than fastMalloc. The design of the collector is as follows: Allocation: -The collector allocates 64KB chunks from the OS to use for object allocation. -Each chunk contains an offset, a flag indicating if the block has been pinned, and a payload, along with next and prev pointers so that they can be put in DoublyLinkedLists. -Any allocation greater than 64KB gets its own separate oversize block, which is managed separately from the rest. -If the allocator receives a request for more than the remaining amount in the current block, it grabs a fresh block. -Grabbing a fresh block means grabbing one off of the global free list (which is now shared between the mark/sweep allocator and the bump allocator) if there is one. If there isn't a new one we do one of two things: allocate a new block from the OS if we're not ready for a GC yet, or run a GC and then try again. If we still don't have enough space after the GC, we allocate a new block from the OS. Garbage collection: -At the start of garbage collection during conservative stack scanning, if we encounter what appears to be a pointer to a bump-allocated block of memory, we pin that block so that it will not be copied for this round of collection. -We also pin any oversize blocks that we encounter, which effectively doubles as a "mark bit" for that block. Any oversize blocks that aren't pinned at the end of copying are given back to the OS. -Marking threads are now also responsible for copying bump-allocated objects to newSpace -Each marking thread has a private 64KB block into which it copies bump-allocated objects that it encounters. -When that block fills up, the marking thread gives it back to the allocator and requests a new one. -When all marking has concluded, each thread gives back its copy block, even if it isn't full. -At the conclusion of copying (which is done by the end of the marking phase), we un-pin any pinned blocks and give any blocks left in from-space to the global free list. * CMakeLists.txt: * GNUmakefile.list.am: * JavaScriptCore.gypi: * JavaScriptCore.vcproj/JavaScriptCore/JavaScriptCore.vcproj: * JavaScriptCore.vcproj/WTF/WTF.vcproj: * JavaScriptCore.xcodeproj/project.pbxproj: * Target.pri: * heap/AllocationSpace.cpp: (JSC::AllocationSpace::allocateSlowCase): (JSC::AllocationSpace::allocateBlock): (JSC::AllocationSpace::freeBlocks): * heap/AllocationSpace.h: (JSC::AllocationSpace::waterMark): * heap/BumpBlock.h: Added. (JSC::BumpBlock::BumpBlock): * heap/BumpSpace.cpp: Added. (JSC::BumpSpace::tryAllocateSlowCase): * heap/BumpSpace.h: Added. (JSC::BumpSpace::isInCopyPhase): (JSC::BumpSpace::totalMemoryAllocated): (JSC::BumpSpace::totalMemoryUtilized): * heap/BumpSpaceInlineMethods.h: Added. (JSC::BumpSpace::BumpSpace): (JSC::BumpSpace::init): (JSC::BumpSpace::contains): (JSC::BumpSpace::pin): (JSC::BumpSpace::startedCopying): (JSC::BumpSpace::doneCopying): (JSC::BumpSpace::doneFillingBlock): (JSC::BumpSpace::recycleBlock): (JSC::BumpSpace::getFreshBlock): (JSC::BumpSpace::borrowBlock): (JSC::BumpSpace::addNewBlock): (JSC::BumpSpace::allocateNewBlock): (JSC::BumpSpace::fitsInBlock): (JSC::BumpSpace::fitsInCurrentBlock): (JSC::BumpSpace::tryAllocate): (JSC::BumpSpace::tryAllocateOversize): (JSC::BumpSpace::allocateFromBlock): (JSC::BumpSpace::tryReallocate): (JSC::BumpSpace::tryReallocateOversize): (JSC::BumpSpace::isOversize): (JSC::BumpSpace::isPinned): (JSC::BumpSpace::oversizeBlockFor): (JSC::BumpSpace::blockFor): * heap/ConservativeRoots.cpp: (JSC::ConservativeRoots::ConservativeRoots): (JSC::ConservativeRoots::genericAddPointer): (JSC::ConservativeRoots::add): * heap/ConservativeRoots.h: * heap/Heap.cpp: (JSC::Heap::Heap): (JSC::Heap::blockFreeingThreadMain): (JSC::Heap::reportExtraMemoryCostSlowCase): (JSC::Heap::getConservativeRegisterRoots): (JSC::Heap::markRoots): (JSC::Heap::collect): (JSC::Heap::releaseFreeBlocks): * heap/Heap.h: (JSC::Heap::waterMark): (JSC::Heap::highWaterMark): (JSC::Heap::setHighWaterMark): (JSC::Heap::tryAllocateStorage): (JSC::Heap::tryReallocateStorage): * heap/HeapBlock.h: Added. (JSC::HeapBlock::HeapBlock): * heap/MarkStack.cpp: (JSC::MarkStackThreadSharedData::MarkStackThreadSharedData): (JSC::SlotVisitor::drain): (JSC::SlotVisitor::drainFromShared): (JSC::SlotVisitor::startCopying): (JSC::SlotVisitor::allocateNewSpace): (JSC::SlotVisitor::copy): (JSC::SlotVisitor::copyAndAppend): (JSC::SlotVisitor::doneCopying): * heap/MarkStack.h: * heap/MarkedBlock.cpp: (JSC::MarkedBlock::recycle): (JSC::MarkedBlock::MarkedBlock): * heap/MarkedBlock.h: * heap/MarkedSpace.cpp: (JSC::MarkedSpace::MarkedSpace): * heap/MarkedSpace.h: (JSC::MarkedSpace::allocate): (JSC::MarkedSpace::forEachBlock): (JSC::MarkedSpace::SizeClass::resetAllocator): * heap/SlotVisitor.h: (JSC::SlotVisitor::SlotVisitor): * heap/TinyBloomFilter.h: (JSC::TinyBloomFilter::reset): * runtime/JSArray.cpp: (JSC::JSArray::JSArray): (JSC::JSArray::finishCreation): (JSC::JSArray::tryFinishCreationUninitialized): (JSC::JSArray::~JSArray): (JSC::JSArray::enterSparseMode): (JSC::JSArray::defineOwnNumericProperty): (JSC::JSArray::setLengthWritable): (JSC::JSArray::getOwnPropertySlotByIndex): (JSC::JSArray::getOwnPropertyDescriptor): (JSC::JSArray::putByIndexBeyondVectorLength): (JSC::JSArray::deletePropertyByIndex): (JSC::JSArray::getOwnPropertyNames): (JSC::JSArray::increaseVectorLength): (JSC::JSArray::unshiftCountSlowCase): (JSC::JSArray::setLength): (JSC::JSArray::pop): (JSC::JSArray::unshiftCount): (JSC::JSArray::visitChildren): (JSC::JSArray::sortNumeric): (JSC::JSArray::sort): (JSC::JSArray::compactForSorting): (JSC::JSArray::subclassData): (JSC::JSArray::setSubclassData): (JSC::JSArray::checkConsistency): * runtime/JSArray.h: (JSC::JSArray::inSparseMode): (JSC::JSArray::isLengthWritable): * wtf/CheckedBoolean.h: Added. (CheckedBoolean::CheckedBoolean): (CheckedBoolean::~CheckedBoolean): (CheckedBoolean::operator bool): * wtf/DoublyLinkedList.h: (WTF::::push): * wtf/StdLibExtras.h: (WTF::isPointerAligned): Source/JavaScriptGlue: Added forwarding header for new CheckedBoolean used in the bump allocator. * ForwardingHeaders/wtf/CheckedBoolean.h: Added. Source/WebCore: No new tests. Added forwarding header for new CheckedBoolean used in the bump allocator. * ForwardingHeaders/wtf/CheckedBoolean.h: Added. Canonical link: https://commits.webkit.org/93486@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@105442 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2012-01-19 21:49:56 +00:00
m_checked = true;
#endif
return m_value;
}
private:
bool m_value;
PerformanceTests: Convert ASSERT_DISABLED to ASSERT_ENABLED, and fix some tests of NDEBUG that should actually test for ASSERT_ENABLED. https://bugs.webkit.org/show_bug.cgi?id=205776 Reviewed by Saam Barati. * JetStream2/wasm/HashSet.cpp: * StitchMarker/wtf/Assertions.h: * StitchMarker/wtf/DateMath.cpp: (WTF::initializeDates): * StitchMarker/wtf/HashTable.h: * StitchMarker/wtf/Hasher.h: (WTF::StringHasher::addCharacters): * StitchMarker/wtf/NeverDestroyed.h: (WTF::LazyNeverDestroyed::construct): * StitchMarker/wtf/StackBounds.h: (WTF::StackBounds::checkConsistency const): * StitchMarker/wtf/ValueCheck.h: * StitchMarker/wtf/Vector.h: (WTF::minCapacity>::checkConsistency): * StitchMarker/wtf/text/AtomicStringImpl.cpp: * StitchMarker/wtf/text/AtomicStringImpl.h: * StitchMarker/wtf/text/StringCommon.h: (WTF::hasPrefixWithLettersIgnoringASCIICaseCommon): * StitchMarker/wtf/text/StringImpl.h: * StitchMarker/wtf/text/SymbolImpl.h: * StitchMarker/wtf/text/UniquedStringImpl.h: Source/JavaScriptCore: Convert ASSERT_DISABLED to ASSERT_ENABLED, and fix some tests of NDEBUG that should actually test for ASSERT_ENABLED. https://bugs.webkit.org/show_bug.cgi?id=205776 Reviewed by Saam Barati. * API/tests/testapi.c: * assembler/ARM64Assembler.h: (JSC::ARM64Assembler::replaceWithLoad): (JSC::ARM64Assembler::replaceWithAddressComputation): * assembler/AssemblerBuffer.h: (JSC::AssemblerBuffer::LocalWriter::LocalWriter): * assembler/LinkBuffer.cpp: (JSC::LinkBuffer::copyCompactAndLinkCode): * assembler/ProbeStack.cpp: (JSC::Probe::Stack::Stack): * assembler/ProbeStack.h: * b3/B3FoldPathConstants.cpp: * b3/B3LowerToAir.cpp: * b3/B3MemoryValue.cpp: (JSC::B3::MemoryValue::MemoryValue): * b3/B3Opcode.cpp: * b3/B3Type.h: * b3/B3TypeMap.h: * b3/B3Width.h: * b3/air/AirAllocateRegistersAndStackAndGenerateCode.cpp: (JSC::B3::Air::GenerateAndAllocateRegisters::prepareForGeneration): (JSC::B3::Air::GenerateAndAllocateRegisters::generate): * b3/air/AirAllocateRegistersAndStackAndGenerateCode.h: * b3/air/AirAllocateRegistersByGraphColoring.cpp: * b3/air/AirArg.cpp: * b3/air/AirArg.h: * b3/air/AirCode.h: * b3/air/AirEmitShuffle.cpp: (JSC::B3::Air::emitShuffle): * builtins/BuiltinExecutables.cpp: (JSC::BuiltinExecutables::createExecutable): * bytecode/AccessCase.cpp: * bytecode/AccessCase.h: * bytecode/CallVariant.cpp: (JSC::variantListWithVariant): * bytecode/CodeBlock.cpp: (JSC::CodeBlock::ensureCatchLivenessIsComputedForBytecodeIndex): * bytecode/CodeBlockHash.cpp: (JSC::CodeBlockHash::dump const): * bytecode/StructureStubInfo.cpp: * bytecode/StructureStubInfo.h: * bytecompiler/NodesCodegen.cpp: (JSC::FunctionCallResolveNode::emitBytecode): * bytecompiler/RegisterID.h: (JSC::RegisterID::RegisterID): (JSC::RegisterID::setIndex): * debugger/Debugger.cpp: (JSC::Debugger::removeBreakpoint): * debugger/DebuggerEvalEnabler.h: (JSC::DebuggerEvalEnabler::DebuggerEvalEnabler): (JSC::DebuggerEvalEnabler::~DebuggerEvalEnabler): * dfg/DFGAbstractInterpreterInlines.h: (JSC::DFG::AbstractInterpreter<AbstractStateType>::observeTransitions): * dfg/DFGAbstractValue.cpp: * dfg/DFGAbstractValue.h: (JSC::DFG::AbstractValue::merge): (JSC::DFG::AbstractValue::checkConsistency const): (JSC::DFG::AbstractValue::assertIsRegistered const): * dfg/DFGArithMode.h: (JSC::DFG::doesOverflow): * dfg/DFGBasicBlock.cpp: (JSC::DFG::BasicBlock::BasicBlock): * dfg/DFGBasicBlock.h: (JSC::DFG::BasicBlock::didLink): * dfg/DFGCFAPhase.cpp: (JSC::DFG::CFAPhase::performBlockCFA): * dfg/DFGCommon.h: (JSC::DFG::validationEnabled): * dfg/DFGCommonData.cpp: (JSC::DFG::CommonData::finalizeCatchEntrypoints): * dfg/DFGDesiredWatchpoints.h: * dfg/DFGDoesGC.cpp: (JSC::DFG::doesGC): * dfg/DFGEdge.h: (JSC::DFG::Edge::makeWord): * dfg/DFGFixupPhase.cpp: (JSC::DFG::FixupPhase::fixupNode): * dfg/DFGJITCode.cpp: (JSC::DFG::JITCode::finalizeOSREntrypoints): * dfg/DFGObjectAllocationSinkingPhase.cpp: * dfg/DFGSSAConversionPhase.cpp: (JSC::DFG::SSAConversionPhase::run): * dfg/DFGScoreBoard.h: (JSC::DFG::ScoreBoard::assertClear): * dfg/DFGSlowPathGenerator.h: (JSC::DFG::SlowPathGenerator::generate): * dfg/DFGSpeculativeJIT.cpp: (JSC::DFG::SpeculativeJIT::compileCurrentBlock): (JSC::DFG::SpeculativeJIT::emitBinarySwitchStringRecurse): (JSC::DFG::SpeculativeJIT::emitAllocateButterfly): (JSC::DFG::SpeculativeJIT::compileAllocateNewArrayWithSize): (JSC::DFG::SpeculativeJIT::compileMakeRope): * dfg/DFGSpeculativeJIT64.cpp: (JSC::DFG::SpeculativeJIT::fillSpeculateCell): * dfg/DFGStructureAbstractValue.cpp: * dfg/DFGStructureAbstractValue.h: (JSC::DFG::StructureAbstractValue::assertIsRegistered const): * dfg/DFGVarargsForwardingPhase.cpp: * dfg/DFGVirtualRegisterAllocationPhase.cpp: (JSC::DFG::VirtualRegisterAllocationPhase::run): * ftl/FTLLink.cpp: (JSC::FTL::link): * ftl/FTLLowerDFGToB3.cpp: (JSC::FTL::DFG::LowerDFGToB3::callPreflight): (JSC::FTL::DFG::LowerDFGToB3::callCheck): (JSC::FTL::DFG::LowerDFGToB3::crash): * ftl/FTLOperations.cpp: (JSC::FTL::operationMaterializeObjectInOSR): * heap/BlockDirectory.cpp: (JSC::BlockDirectory::assertNoUnswept): * heap/GCSegmentedArray.h: (JSC::GCArraySegment::GCArraySegment): * heap/GCSegmentedArrayInlines.h: (JSC::GCSegmentedArray<T>::clear): (JSC::GCSegmentedArray<T>::expand): (JSC::GCSegmentedArray<T>::validatePrevious): * heap/HandleSet.cpp: * heap/HandleSet.h: * heap/Heap.cpp: (JSC::Heap::updateAllocationLimits): * heap/Heap.h: * heap/MarkedBlock.cpp: * heap/MarkedBlock.h: (JSC::MarkedBlock::assertValidCell const): (JSC::MarkedBlock::assertMarksNotStale): * heap/MarkedSpace.cpp: (JSC::MarkedSpace::beginMarking): (JSC::MarkedSpace::endMarking): (JSC::MarkedSpace::assertNoUnswept): * heap/PreciseAllocation.cpp: * heap/PreciseAllocation.h: (JSC::PreciseAllocation::assertValidCell const): * heap/SlotVisitor.cpp: (JSC::SlotVisitor::SlotVisitor): (JSC::SlotVisitor::appendJSCellOrAuxiliary): * heap/SlotVisitor.h: * inspector/InspectorProtocolTypes.h: (Inspector::Protocol::BindingTraits<JSON::ArrayOf<T>>::assertValueHasExpectedType): * inspector/scripts/codegen/generate_cpp_protocol_types_implementation.py: (CppProtocolTypesImplementationGenerator._generate_assertion_for_object_declaration): (CppProtocolTypesImplementationGenerator): (CppProtocolTypesImplementationGenerator._generate_assertion_for_enum): * inspector/scripts/tests/generic/expected/type-requiring-runtime-casts.json-result: * interpreter/FrameTracers.h: (JSC::JITOperationPrologueCallFrameTracer::JITOperationPrologueCallFrameTracer): * interpreter/Interpreter.cpp: (JSC::Interpreter::Interpreter): * interpreter/Interpreter.h: * jit/AssemblyHelpers.cpp: (JSC::AssemblyHelpers::emitStoreStructureWithTypeInfo): * jit/AssemblyHelpers.h: (JSC::AssemblyHelpers::prepareCallOperation): * jit/BinarySwitch.cpp: (JSC::BinarySwitch::BinarySwitch): * jit/CCallHelpers.h: (JSC::CCallHelpers::setupStubArgs): * jit/CallFrameShuffler.cpp: (JSC::CallFrameShuffler::emitDeltaCheck): (JSC::CallFrameShuffler::prepareAny): * jit/JIT.cpp: (JSC::JIT::assertStackPointerOffset): (JSC::JIT::compileWithoutLinking): * jit/JITOpcodes.cpp: (JSC::JIT::emitSlow_op_loop_hint): * jit/JITPropertyAccess.cpp: (JSC::JIT::emit_op_get_from_scope): * jit/JITPropertyAccess32_64.cpp: (JSC::JIT::emit_op_get_from_scope): * jit/Repatch.cpp: (JSC::linkPolymorphicCall): * jit/ThunkGenerators.cpp: (JSC::emitPointerValidation): * llint/LLIntData.cpp: (JSC::LLInt::Data::performAssertions): * llint/LLIntOfflineAsmConfig.h: * parser/Lexer.cpp: * parser/Lexer.h: (JSC::isSafeBuiltinIdentifier): (JSC::Lexer<T>::lexExpectIdentifier): * runtime/ArgList.h: (JSC::MarkedArgumentBuffer::setNeedsOverflowCheck): (JSC::MarkedArgumentBuffer::clearNeedsOverflowCheck): * runtime/Butterfly.h: (JSC::ContiguousData::ContiguousData): (JSC::ContiguousData::Data::Data): * runtime/HashMapImpl.h: (JSC::HashMapImpl::checkConsistency const): (JSC::HashMapImpl::assertBufferIsEmpty const): * runtime/JSCellInlines.h: (JSC::JSCell::methodTable const): * runtime/JSFunction.cpp: * runtime/JSFunction.h: (JSC::JSFunction::assertTypeInfoFlagInvariants): * runtime/JSGlobalObject.cpp: (JSC::JSGlobalObject::init): * runtime/JSGlobalObject.h: * runtime/JSObject.cpp: (JSC::JSObject::visitChildren): (JSC::JSFinalObject::visitChildren): * runtime/JSObjectInlines.h: (JSC::JSObject::validatePutOwnDataProperty): * runtime/JSSegmentedVariableObject.h: (JSC::JSSegmentedVariableObject::assertVariableIsInThisObject): * runtime/LiteralParser.cpp: (JSC::LiteralParser<CharType>::Lexer::lex): * runtime/LiteralParser.h: * runtime/Operations.h: (JSC::scribbleFreeCells): * runtime/OptionsList.h: * runtime/VM.cpp: (JSC::VM::computeCanUseJIT): * runtime/VM.h: (JSC::VM::canUseJIT): * runtime/VarOffset.h: (JSC::VarOffset::checkSanity const): * runtime/WeakMapImpl.h: (JSC::WeakMapImpl::checkConsistency const): (JSC::WeakMapImpl::assertBufferIsEmpty const): * wasm/WasmAirIRGenerator.cpp: (JSC::Wasm::AirIRGenerator::validateInst): * wasm/WasmB3IRGenerator.cpp: (JSC::Wasm::parseAndCompile): * wasm/WasmFunctionParser.h: (JSC::Wasm::FunctionParser::validationFail const): * wasm/WasmLLIntGenerator.cpp: (JSC::Wasm::LLIntGenerator::checkConsistency): * wasm/WasmPlan.cpp: (JSC::Wasm::Plan::tryRemoveContextAndCancelIfLast): * wasm/WasmSectionParser.h: * wasm/WasmSections.h: * wasm/WasmSignatureInlines.h: (JSC::Wasm::SignatureInformation::get): * wasm/WasmWorklist.cpp: (JSC::Wasm::Worklist::enqueue): * wasm/js/JSToWasm.cpp: (JSC::Wasm::createJSToWasmWrapper): * wasm/js/WebAssemblyFunction.cpp: (JSC::WebAssemblyFunction::previousInstanceOffset const): Source/WebCore: Convert ASSERT_DISABLED to ASSERT_ENABLED, and fix some tests of NDEBUG that should actually test for ASSERT_ENABLED. https://bugs.webkit.org/show_bug.cgi?id=205776 Reviewed by Saam Barati. * Modules/fetch/FetchBodySource.cpp: (WebCore::FetchBodySource::close): * Modules/fetch/FetchBodySource.h: * Modules/webdatabase/DatabaseDetails.h: (WebCore::DatabaseDetails::DatabaseDetails): (WebCore::DatabaseDetails::operator=): * Modules/webdatabase/DatabaseTask.cpp: (WebCore::DatabaseTask::performTask): * Modules/webdatabase/DatabaseTask.h: * Modules/webdatabase/DatabaseThread.cpp: (WebCore::DatabaseThread::terminationRequested const): * Modules/webgpu/WHLSL/AST/WHLSLAddressSpace.h: (WebCore::WHLSL::AST::TypeAnnotation::TypeAnnotation): * Modules/webgpu/WHLSL/WHLSLHighZombieFinder.cpp: (WebCore::WHLSL::findHighZombies): * Modules/webgpu/WHLSL/WHLSLInferTypes.cpp: (WebCore::WHLSL::matches): * Modules/webgpu/WHLSL/WHLSLLiteralTypeChecker.cpp: (WebCore::WHLSL::checkLiteralTypes): * Modules/webgpu/WHLSL/WHLSLSynthesizeConstructors.cpp: (WebCore::WHLSL::FindAllTypes::appendNamedType): * bindings/js/JSCallbackData.h: * bindings/js/JSLazyEventListener.cpp: * bindings/js/JSLazyEventListener.h: * contentextensions/ContentExtensionCompiler.cpp: (WebCore::ContentExtensions::compileRuleList): * css/CSSCalculationValue.cpp: (WebCore::CSSCalcOperationNode::primitiveType const): * css/CSSComputedStyleDeclaration.cpp: (WebCore::ComputedStyleExtractor::valueForPropertyInStyle): * css/CSSPrimitiveValue.cpp: * css/CSSSelector.cpp: (WebCore::CSSSelector::selectorText const): * css/CSSStyleSheet.cpp: * dom/ActiveDOMObject.cpp: (WebCore::ActiveDOMObject::suspendIfNeeded): (WebCore::ActiveDOMObject::assertSuspendIfNeededWasCalled const): * dom/ActiveDOMObject.h: * dom/ContainerNode.cpp: * dom/ContainerNodeAlgorithms.cpp: * dom/ContainerNodeAlgorithms.h: * dom/CustomElementReactionQueue.cpp: * dom/CustomElementReactionQueue.h: (WebCore::CustomElementReactionDisallowedScope::CustomElementReactionDisallowedScope): (WebCore::CustomElementReactionDisallowedScope::~CustomElementReactionDisallowedScope): * dom/Document.cpp: (WebCore::Document::hitTest): * dom/Document.h: (WebCore::Document::decrementReferencingNodeCount): * dom/Element.cpp: (WebCore::Element::addShadowRoot): (WebCore::Element::getURLAttribute const): (WebCore::Element::getNonEmptyURLAttribute const): * dom/Element.h: * dom/ElementAndTextDescendantIterator.h: (WebCore::ElementAndTextDescendantIterator::ElementAndTextDescendantIterator): (WebCore::ElementAndTextDescendantIterator::dropAssertions): (WebCore::ElementAndTextDescendantIterator::popAncestorSiblingStack): (WebCore::ElementAndTextDescendantIterator::traverseNextSibling): (WebCore::ElementAndTextDescendantIterator::traversePreviousSibling): * dom/ElementDescendantIterator.h: (WebCore::ElementDescendantIterator::ElementDescendantIterator): (WebCore::ElementDescendantIterator::dropAssertions): (WebCore::ElementDescendantIterator::operator++): (WebCore::ElementDescendantIterator::operator--): (WebCore::ElementDescendantConstIterator::ElementDescendantConstIterator): (WebCore::ElementDescendantConstIterator::dropAssertions): (WebCore::ElementDescendantConstIterator::operator++): * dom/ElementIterator.h: (WebCore::ElementIterator<ElementType>::ElementIterator): (WebCore::ElementIterator<ElementType>::traverseNext): (WebCore::ElementIterator<ElementType>::traversePrevious): (WebCore::ElementIterator<ElementType>::traverseNextSibling): (WebCore::ElementIterator<ElementType>::traversePreviousSibling): (WebCore::ElementIterator<ElementType>::traverseNextSkippingChildren): (WebCore::ElementIterator<ElementType>::dropAssertions): (WebCore::ElementIterator<ElementType>::traverseAncestor): (WebCore::ElementConstIterator<ElementType>::ElementConstIterator): (WebCore::ElementConstIterator<ElementType>::traverseNext): (WebCore::ElementConstIterator<ElementType>::traversePrevious): (WebCore::ElementConstIterator<ElementType>::traverseNextSibling): (WebCore::ElementConstIterator<ElementType>::traversePreviousSibling): (WebCore::ElementConstIterator<ElementType>::traverseNextSkippingChildren): (WebCore::ElementConstIterator<ElementType>::traverseAncestor): (WebCore::ElementConstIterator<ElementType>::dropAssertions): * dom/EventContext.cpp: * dom/EventContext.h: * dom/EventListener.h: * dom/EventPath.cpp: * dom/EventSender.h: * dom/EventTarget.cpp: (WebCore::EventTarget::addEventListener): (WebCore::EventTarget::setAttributeEventListener): (WebCore::EventTarget::innerInvokeEventListeners): * dom/Node.cpp: (WebCore::Node::~Node): (WebCore::Node::moveNodeToNewDocument): (WebCore::Node::removedLastRef): * dom/Node.h: (WebCore::Node::deref const): * dom/ScriptDisallowedScope.h: (WebCore::ScriptDisallowedScope::InMainThread::isEventDispatchAllowedInSubtree): * dom/ScriptExecutionContext.cpp: (WebCore::ScriptExecutionContext::~ScriptExecutionContext): * dom/ScriptExecutionContext.h: * dom/SelectorQuery.cpp: (WebCore::SelectorDataList::execute const): * dom/SlotAssignment.cpp: (WebCore::SlotAssignment::addSlotElementByName): (WebCore::SlotAssignment::removeSlotElementByName): (WebCore::SlotAssignment::resolveSlotsAfterSlotMutation): (WebCore::SlotAssignment::findFirstSlotElement): * dom/SlotAssignment.h: * dom/TreeScopeOrderedMap.cpp: (WebCore::TreeScopeOrderedMap::add): (WebCore::TreeScopeOrderedMap::get const): * dom/TreeScopeOrderedMap.h: * fileapi/Blob.cpp: * fileapi/Blob.h: * history/BackForwardCache.cpp: (WebCore::BackForwardCache::removeAllItemsForPage): * history/BackForwardCache.h: * html/CanvasBase.cpp: (WebCore::CanvasBase::notifyObserversCanvasDestroyed): * html/CanvasBase.h: * html/HTMLCollection.h: (WebCore::CollectionNamedElementCache::didPopulate): * html/HTMLSelectElement.cpp: (WebCore:: const): * html/HTMLTableRowsCollection.cpp: (WebCore::assertRowIsInTable): * html/HTMLTextFormControlElement.cpp: (WebCore::HTMLTextFormControlElement::indexForPosition const): * html/canvas/CanvasRenderingContext2DBase.cpp: (WebCore::CanvasRenderingContext2DBase::~CanvasRenderingContext2DBase): * html/parser/HTMLParserScheduler.cpp: (WebCore::HTMLParserScheduler::HTMLParserScheduler): (WebCore::HTMLParserScheduler::suspend): (WebCore::HTMLParserScheduler::resume): * html/parser/HTMLParserScheduler.h: * html/parser/HTMLToken.h: (WebCore::HTMLToken::beginStartTag): (WebCore::HTMLToken::beginEndTag): (WebCore::HTMLToken::endAttribute): * html/parser/HTMLTreeBuilder.cpp: (WebCore::HTMLTreeBuilder::HTMLTreeBuilder): (WebCore::HTMLTreeBuilder::constructTree): * html/parser/HTMLTreeBuilder.h: (WebCore::HTMLTreeBuilder::~HTMLTreeBuilder): * layout/FormattingContext.cpp: (WebCore::Layout::FormattingContext::geometryForBox const): * layout/blockformatting/BlockFormattingContext.cpp: (WebCore::Layout::BlockFormattingContext::computeEstimatedVerticalPosition): * layout/blockformatting/BlockFormattingContext.h: * layout/displaytree/DisplayBox.cpp: (WebCore::Display::Box::Box): * layout/displaytree/DisplayBox.h: (WebCore::Display::Box::setTopLeft): (WebCore::Display::Box::setTop): (WebCore::Display::Box::setLeft): (WebCore::Display::Box::setContentBoxHeight): (WebCore::Display::Box::setContentBoxWidth): (WebCore::Display::Box::setHorizontalMargin): (WebCore::Display::Box::setVerticalMargin): (WebCore::Display::Box::setHorizontalComputedMargin): (WebCore::Display::Box::setBorder): (WebCore::Display::Box::setPadding): * layout/displaytree/DisplayInlineRect.h: (WebCore::Display::InlineRect::InlineRect): (WebCore::Display::InlineRect::setTopLeft): (WebCore::Display::InlineRect::setTop): (WebCore::Display::InlineRect::setBottom): (WebCore::Display::InlineRect::setLeft): (WebCore::Display::InlineRect::setWidth): (WebCore::Display::InlineRect::setHeight): * layout/displaytree/DisplayLineBox.h: (WebCore::Display::LineBox::LineBox): (WebCore::Display::LineBox::setBaselineOffsetIfGreater): (WebCore::Display::LineBox::resetBaseline): (WebCore::Display::LineBox::Baseline::Baseline): (WebCore::Display::LineBox::Baseline::setAscent): (WebCore::Display::LineBox::Baseline::setDescent): (WebCore::Display::LineBox::Baseline::reset): * layout/displaytree/DisplayRect.h: (WebCore::Display::Rect::Rect): (WebCore::Display::Rect::setTopLeft): (WebCore::Display::Rect::setTop): (WebCore::Display::Rect::setLeft): (WebCore::Display::Rect::setWidth): (WebCore::Display::Rect::setHeight): (WebCore::Display::Rect::setSize): (WebCore::Display::Rect::clone const): * layout/floats/FloatingContext.cpp: * layout/inlineformatting/InlineLineBuilder.cpp: (WebCore::Layout::LineBuilder::CollapsibleContent::collapse): * layout/tableformatting/TableGrid.cpp: (WebCore::Layout::TableGrid::Column::setWidthConstraints): (WebCore::Layout::TableGrid::Column::setLogicalWidth): (WebCore::Layout::TableGrid::Column::setLogicalLeft): * layout/tableformatting/TableGrid.h: * loader/DocumentLoader.cpp: (WebCore::DocumentLoader::continueAfterContentPolicy): (WebCore::DocumentLoader::attachToFrame): (WebCore::DocumentLoader::detachFromFrame): (WebCore::DocumentLoader::addSubresourceLoader): * loader/DocumentLoader.h: * loader/ImageLoader.cpp: * loader/cache/CachedResource.h: * loader/cache/MemoryCache.cpp: (WebCore::MemoryCache::lruListFor): (WebCore::MemoryCache::removeFromLRUList): * page/FrameView.cpp: (WebCore::FrameView::updateLayoutAndStyleIfNeededRecursive): * page/FrameViewLayoutContext.cpp: * page/FrameViewLayoutContext.h: * page/Page.cpp: * page/Page.h: * page/ViewportConfiguration.cpp: * page/ViewportConfiguration.h: * page/mac/EventHandlerMac.mm: (WebCore::CurrentEventScope::CurrentEventScope): * platform/DateComponents.cpp: (WebCore::DateComponents::toStringForTime const): * platform/ScrollableArea.cpp: * platform/SharedBuffer.cpp: (WebCore::SharedBuffer::combineIntoOneSegment const): * platform/SharedBuffer.h: * platform/Supplementable.h: * platform/Timer.cpp: (WebCore::TimerBase::checkHeapIndex const): (WebCore::TimerBase::updateHeapIfNeeded): * platform/graphics/BitmapImage.cpp: * platform/graphics/BitmapImage.h: * platform/graphics/Image.h: * platform/graphics/ShadowBlur.cpp: (WebCore::ScratchBuffer::ScratchBuffer): (WebCore::ScratchBuffer::getScratchBuffer): (WebCore::ScratchBuffer::scheduleScratchBufferPurge): * platform/graphics/ca/win/CACFLayerTreeHost.cpp: (WebCore::CACFLayerTreeHost::setWindow): * platform/graphics/ca/win/CACFLayerTreeHost.h: * platform/graphics/cg/ImageBufferDataCG.cpp: (WebCore::ImageBufferData::putData): * platform/graphics/cocoa/FontCacheCoreText.cpp: * platform/graphics/gstreamer/GstAllocatorFastMalloc.cpp: (gstAllocatorFastMallocFree): * platform/graphics/nicosia/cairo/NicosiaPaintingContextCairo.cpp: (Nicosia::PaintingContextCairo::ForPainting::ForPainting): * platform/graphics/nicosia/texmap/NicosiaBackingStoreTextureMapperImpl.cpp: (Nicosia::BackingStoreTextureMapperImpl::createTile): * platform/graphics/nicosia/texmap/NicosiaContentLayerTextureMapperImpl.cpp: (Nicosia::ContentLayerTextureMapperImpl::~ContentLayerTextureMapperImpl): * platform/graphics/win/GradientDirect2D.cpp: (WebCore::Gradient::fill): * platform/graphics/win/ImageBufferDataDirect2D.cpp: (WebCore::ImageBufferData::putData): * platform/graphics/win/PathDirect2D.cpp: (WebCore::Path::appendGeometry): (WebCore::Path::Path): (WebCore::Path::operator=): (WebCore::Path::strokeContains const): (WebCore::Path::transform): * platform/graphics/win/PlatformContextDirect2D.cpp: (WebCore::PlatformContextDirect2D::setTags): * platform/mediastream/MediaStreamTrackPrivate.h: * platform/mediastream/RealtimeOutgoingAudioSource.cpp: (WebCore::RealtimeOutgoingAudioSource::~RealtimeOutgoingAudioSource): * platform/mediastream/RealtimeOutgoingVideoSource.cpp: (WebCore::RealtimeOutgoingVideoSource::~RealtimeOutgoingVideoSource): * platform/network/HTTPParsers.cpp: (WebCore::isCrossOriginSafeHeader): * platform/sql/SQLiteDatabase.cpp: * platform/sql/SQLiteDatabase.h: * platform/sql/SQLiteStatement.cpp: (WebCore::SQLiteStatement::SQLiteStatement): (WebCore::SQLiteStatement::prepare): (WebCore::SQLiteStatement::finalize): * platform/sql/SQLiteStatement.h: * platform/win/COMPtr.h: * rendering/ComplexLineLayout.cpp: (WebCore::ComplexLineLayout::removeInlineBox const): * rendering/FloatingObjects.cpp: (WebCore::FloatingObject::FloatingObject): (WebCore::FloatingObjects::addPlacedObject): (WebCore::FloatingObjects::removePlacedObject): * rendering/FloatingObjects.h: * rendering/GridTrackSizingAlgorithm.cpp: * rendering/GridTrackSizingAlgorithm.h: * rendering/LayoutDisallowedScope.cpp: * rendering/LayoutDisallowedScope.h: * rendering/RenderBlock.cpp: * rendering/RenderBlock.h: * rendering/RenderBlockFlow.cpp: (WebCore::RenderBlockFlow::layoutBlockChild): (WebCore::RenderBlockFlow::removeFloatingObject): (WebCore::RenderBlockFlow::ensureLineBoxes): * rendering/RenderBoxModelObject.cpp: * rendering/RenderDeprecatedFlexibleBox.cpp: (WebCore::RenderDeprecatedFlexibleBox::layoutBlock): * rendering/RenderElement.cpp: * rendering/RenderGeometryMap.cpp: (WebCore::RenderGeometryMap::mapToContainer const): * rendering/RenderGrid.cpp: (WebCore::RenderGrid::placeItemsOnGrid const): (WebCore::RenderGrid::baselinePosition const): * rendering/RenderInline.cpp: (WebCore::RenderInline::willBeDestroyed): * rendering/RenderLayer.cpp: (WebCore::ClipRectsCache::ClipRectsCache): (WebCore::RenderLayer::RenderLayer): (WebCore::RenderLayer::paintList): (WebCore::RenderLayer::hitTestLayer): (WebCore::RenderLayer::updateClipRects): (WebCore::RenderLayer::calculateClipRects const): * rendering/RenderLayer.h: * rendering/RenderLayerBacking.cpp: (WebCore::traverseVisibleNonCompositedDescendantLayers): * rendering/RenderLayerCompositor.cpp: (WebCore::RenderLayerCompositor::computeCompositingRequirements): (WebCore::RenderLayerCompositor::traverseUnchangedSubtree): (WebCore::RenderLayerCompositor::updateBackingAndHierarchy): (WebCore::RenderLayerCompositor::addDescendantsToOverlapMapRecursive const): (WebCore::RenderLayerCompositor::recursiveRepaintLayer): (WebCore::RenderLayerCompositor::layerHas3DContent const): * rendering/RenderLayoutState.cpp: (WebCore::RenderLayoutState::RenderLayoutState): (WebCore::RenderLayoutState::computeOffsets): (WebCore::RenderLayoutState::addLayoutDelta): * rendering/RenderLayoutState.h: (WebCore::RenderLayoutState::RenderLayoutState): * rendering/RenderObject.cpp: (WebCore::RenderObject::RenderObject): (WebCore::RenderObject::~RenderObject): (WebCore::RenderObject::clearNeedsLayout): * rendering/RenderObject.h: * rendering/RenderQuote.cpp: (WebCore::quotesForLanguage): * rendering/RenderTableCell.h: * rendering/RenderTableSection.cpp: (WebCore::RenderTableSection::computeOverflowFromCells): * rendering/RenderTextLineBoxes.cpp: (WebCore::RenderTextLineBoxes::checkConsistency const): * rendering/RenderTextLineBoxes.h: * rendering/line/BreakingContext.h: (WebCore::tryHyphenating): * rendering/style/GridArea.h: (WebCore::GridSpan::GridSpan): * rendering/style/RenderStyle.cpp: (WebCore::RenderStyle::~RenderStyle): * rendering/style/RenderStyle.h: * rendering/updating/RenderTreeBuilderRuby.cpp: (WebCore::RenderTreeBuilder::Ruby::detach): * rendering/updating/RenderTreePosition.cpp: (WebCore::RenderTreePosition::computeNextSibling): * rendering/updating/RenderTreePosition.h: * svg/SVGToOTFFontConversion.cpp: (WebCore::SVGToOTFFontConverter::Placeholder::Placeholder): (WebCore::SVGToOTFFontConverter::Placeholder::populate): (WebCore::SVGToOTFFontConverter::appendCFFTable): (WebCore::SVGToOTFFontConverter::firstGlyph const): (WebCore::SVGToOTFFontConverter::appendKERNTable): * svg/SVGTransformDistance.cpp: (WebCore::SVGTransformDistance::SVGTransformDistance): (WebCore::SVGTransformDistance::scaledDistance const): (WebCore::SVGTransformDistance::addSVGTransforms): (WebCore::SVGTransformDistance::addToSVGTransform const): (WebCore::SVGTransformDistance::distance const): * svg/graphics/SVGImage.cpp: (WebCore::SVGImage::nativeImage): * testing/InternalSettings.cpp: * workers/service/ServiceWorkerJob.h: * worklets/PaintWorkletGlobalScope.h: (WebCore::PaintWorkletGlobalScope::~PaintWorkletGlobalScope): * xml/XPathStep.cpp: Source/WebKit: Convert ASSERT_DISABLED to ASSERT_ENABLED, and fix some tests of NDEBUG that should actually test for ASSERT_ENABLED. https://bugs.webkit.org/show_bug.cgi?id=205776 Reviewed by Saam Barati. * NetworkProcess/NetworkSession.cpp: (WebKit::NetworkSession::invalidateAndCancel): * NetworkProcess/NetworkSession.h: * NetworkProcess/cache/NetworkCacheStorage.cpp: (WebKit::NetworkCache::Storage::setCapacity): * NetworkProcess/cocoa/NetworkSessionCocoa.mm: (toNSURLSessionResponseDisposition): (WebKit::NetworkSessionCocoa::NetworkSessionCocoa): * Platform/IPC/Connection.cpp: (IPC::Connection::waitForMessage): * Platform/IPC/MessageReceiver.h: (IPC::MessageReceiver::willBeAddedToMessageReceiverMap): (IPC::MessageReceiver::willBeRemovedFromMessageReceiverMap): * Platform/IPC/cocoa/ConnectionCocoa.mm: (IPC::readFromMachPort): * Platform/mac/MachUtilities.cpp: (setMachExceptionPort): * Shared/API/APIClient.h: (API::Client::Client): * Shared/API/Cocoa/WKRemoteObjectCoder.mm: * Shared/Cocoa/ArgumentCodersCocoa.h: * Shared/SharedStringHashTableReadOnly.cpp: * UIProcess/BackingStore.cpp: (WebKit::BackingStore::incorporateUpdate): * UIProcess/GenericCallback.h: * UIProcess/Launcher/mac/ProcessLauncherMac.mm: (WebKit::ProcessLauncher::launchProcess): * UIProcess/PageLoadState.h: (WebKit::PageLoadState::Transaction::Token::Token): * UIProcess/WebPageProxy.cpp: (WebKit::WebPageProxy::~WebPageProxy): * WebProcess/Network/WebResourceLoader.cpp: (WebKit::WebResourceLoader::didReceiveResponse): * WebProcess/Network/WebResourceLoader.h: * WebProcess/Plugins/Netscape/NetscapePluginStream.cpp: (WebKit::NetscapePluginStream::NetscapePluginStream): (WebKit::NetscapePluginStream::notifyAndDestroyStream): * WebProcess/Plugins/Netscape/NetscapePluginStream.h: * WebProcess/WebPage/WebPage.cpp: (WebKit::WebPage::runModal): * WebProcess/WebProcess.cpp: (WebKit::checkDocumentsCaptureStateConsistency): * WebProcess/cocoa/WebProcessCocoa.mm: (WebKit::WebProcess::updateProcessName): Source/WebKitLegacy: Convert ASSERT_DISABLED to ASSERT_ENABLED, and fix some tests of NDEBUG that should actually test for ASSERT_ENABLED. https://bugs.webkit.org/show_bug.cgi?id=205776 Reviewed by Saam Barati. * Storage/StorageAreaImpl.cpp: (WebKit::StorageAreaImpl::StorageAreaImpl): (WebKit::StorageAreaImpl::close): * Storage/StorageAreaImpl.h: Source/WebKitLegacy/mac: Convert ASSERT_DISABLED to ASSERT_ENABLED, and fix some tests of NDEBUG that should actually test for ASSERT_ENABLED. https://bugs.webkit.org/show_bug.cgi?id=205776 Reviewed by Saam Barati. * History/WebHistory.mm: (-[WebHistoryPrivate removeItemForURLString:]): * WebView/WebFrame.mm: Source/WebKitLegacy/win: Convert ASSERT_DISABLED to ASSERT_ENABLED, and fix some tests of NDEBUG that should actually test for ASSERT_ENABLED. https://bugs.webkit.org/show_bug.cgi?id=205776 Reviewed by Saam Barati. * WebKitQuartzCoreAdditions/CAD3DRenderer.cpp: (WKQCA::CAD3DRenderer::swapChain): (WKQCA::CAD3DRenderer::initialize): * WebKitQuartzCoreAdditions/CAD3DRenderer.h: * WebView.cpp: (WebView::Release): * WebView.h: Source/WTF: Convert ASSERT_DISABLED to ASSERT_ENABLED, and fix some tests of NDEBUG that should actually test for ASSERT_ENABLED. https://bugs.webkit.org/show_bug.cgi?id=205776 Reviewed by Saam Barati. This patch did the following changes: 1. Replaced ASSERT_DISABLED with ASSERT_ENABLED. This change does away with the need for the double negative !ASSERT_DISABLED test that is commonly used all over the code, thereby improving code readability. In Assertions.h, there is also BACKTRACE_DISABLED, ASSERT_MSG_DISABLED, ASSERT_ARG_DISABLED, FATAL_DISABLED, ERROR_DISABLED, LOG_DISABLED, and RELEASE_LOG_DISABLED. We should replace those with ..._ENABLED equivalents as well. We'll do that in another patch. For now, they are left as is to minimize the size of this patch. See https://bugs.webkit.org/show_bug.cgi?id=205780. 2. Fixed some code was guarded with "#ifndef NDEBUG" that should actually be guarded by "#if ASSERT_ENABLED" instead. 3. In cases where the change is minimal, we move some code around so that we can test for "#if ASSERT_ENABLED" instead of "#if !ASSERT_ENABLED". * wtf/Assertions.h: * wtf/AutomaticThread.cpp: (WTF::AutomaticThread::start): * wtf/BitVector.h: * wtf/BlockObjCExceptions.mm: (ReportBlockedObjCException): * wtf/BloomFilter.h: * wtf/CallbackAggregator.h: (WTF::CallbackAggregator::CallbackAggregator): * wtf/CheckedArithmetic.h: (WTF::observesOverflow<AssertNoOverflow>): * wtf/CheckedBoolean.h: (CheckedBoolean::CheckedBoolean): (CheckedBoolean::operator bool): * wtf/CompletionHandler.h: (WTF::CompletionHandler<Out): * wtf/DateMath.cpp: (WTF::initializeDates): * wtf/Gigacage.cpp: (Gigacage::tryAllocateZeroedVirtualPages): * wtf/HashTable.h: (WTF::KeyTraits>::checkKey): (WTF::KeyTraits>::checkTableConsistencyExceptSize const): * wtf/LoggerHelper.h: * wtf/NaturalLoops.h: (WTF::NaturalLoops::headerOf const): * wtf/NeverDestroyed.h: (WTF::LazyNeverDestroyed::construct): * wtf/OptionSet.h: (WTF::OptionSet::OptionSet): * wtf/Platform.h: * wtf/PtrTag.h: * wtf/RefCounted.h: (WTF::RefCountedBase::disableThreadingChecks): (WTF::RefCountedBase::enableThreadingChecksGlobally): (WTF::RefCountedBase::RefCountedBase): (WTF::RefCountedBase::applyRefDerefThreadingCheck const): * wtf/SingleRootGraph.h: (WTF::SingleRootGraph::assertIsConsistent const): * wtf/SizeLimits.cpp: * wtf/StackBounds.h: (WTF::StackBounds::checkConsistency const): * wtf/URLParser.cpp: (WTF::URLParser::URLParser): (WTF::URLParser::domainToASCII): * wtf/ValueCheck.h: * wtf/Vector.h: (WTF::Malloc>::checkConsistency): * wtf/WeakHashSet.h: * wtf/WeakPtr.h: (WTF::WeakPtrImpl::WeakPtrImpl): (WTF::WeakPtrFactory::WeakPtrFactory): * wtf/text/AtomStringImpl.cpp: * wtf/text/AtomStringImpl.h: * wtf/text/StringBuilder.cpp: (WTF::StringBuilder::reifyString const): * wtf/text/StringBuilder.h: * wtf/text/StringCommon.h: (WTF::hasPrefixWithLettersIgnoringASCIICaseCommon): * wtf/text/StringHasher.h: (WTF::StringHasher::addCharacters): * wtf/text/StringImpl.h: * wtf/text/SymbolImpl.h: * wtf/text/UniquedStringImpl.h: Tools: Remove WebsiteDataStore::setServiceWorkerRegistrationDirectory https://bugs.webkit.org/show_bug.cgi?id=205754 Patch by Alex Christensen <achristensen@webkit.org> on 2020-01-06 Reviewed by Youenn Fablet. * TestWebKitAPI/Tests/WebKitCocoa/ServiceWorkerBasic.mm: * WebKitTestRunner/TestController.cpp: (WTR::TestController::websiteDataStore): (WTR::TestController::platformAdjustContext): * WebKitTestRunner/cocoa/TestControllerCocoa.mm: (WTR::initializeWebViewConfiguration): Canonical link: https://commits.webkit.org/218957@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@254087 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2020-01-06 22:24:50 +00:00
#if ASSERT_ENABLED
mutable bool m_checked;
Implement a new allocator for backing stores https://bugs.webkit.org/show_bug.cgi?id=75181 Reviewed by Filip Pizlo. Source/JavaScriptCore: We want to move away from using fastMalloc for the backing stores for some of our objects (e.g. JSArray, JSObject, JSString, etc). These backing stores have a nice property in that they only have a single owner (i.e. a single pointer to them at any one time). One way that we can take advantage of this property is to implement a simple bump allocator/copying collector, which will run alongside our normal mark/sweep collector, that only needs to update the single owner pointer rather than having to redirect an arbitrary number of pointers in from-space to to-space. This plan can give us a number of benefits. We can beat fastMalloc in terms of both performance and memory usage, we can track how much memory we're using far more accurately than our rough estimation now through the use of reportExtraMemoryCost, and we can allocate arbitrary size objects (as opposed to being limited to size classes like we have been historically). This is also another step toward moving away from lazy destruction, which will improve our memory footprint. We start by creating said allocator and moving the ArrayStorage for JSArray to use it rather than fastMalloc. The design of the collector is as follows: Allocation: -The collector allocates 64KB chunks from the OS to use for object allocation. -Each chunk contains an offset, a flag indicating if the block has been pinned, and a payload, along with next and prev pointers so that they can be put in DoublyLinkedLists. -Any allocation greater than 64KB gets its own separate oversize block, which is managed separately from the rest. -If the allocator receives a request for more than the remaining amount in the current block, it grabs a fresh block. -Grabbing a fresh block means grabbing one off of the global free list (which is now shared between the mark/sweep allocator and the bump allocator) if there is one. If there isn't a new one we do one of two things: allocate a new block from the OS if we're not ready for a GC yet, or run a GC and then try again. If we still don't have enough space after the GC, we allocate a new block from the OS. Garbage collection: -At the start of garbage collection during conservative stack scanning, if we encounter what appears to be a pointer to a bump-allocated block of memory, we pin that block so that it will not be copied for this round of collection. -We also pin any oversize blocks that we encounter, which effectively doubles as a "mark bit" for that block. Any oversize blocks that aren't pinned at the end of copying are given back to the OS. -Marking threads are now also responsible for copying bump-allocated objects to newSpace -Each marking thread has a private 64KB block into which it copies bump-allocated objects that it encounters. -When that block fills up, the marking thread gives it back to the allocator and requests a new one. -When all marking has concluded, each thread gives back its copy block, even if it isn't full. -At the conclusion of copying (which is done by the end of the marking phase), we un-pin any pinned blocks and give any blocks left in from-space to the global free list. * CMakeLists.txt: * GNUmakefile.list.am: * JavaScriptCore.gypi: * JavaScriptCore.vcproj/JavaScriptCore/JavaScriptCore.vcproj: * JavaScriptCore.vcproj/WTF/WTF.vcproj: * JavaScriptCore.xcodeproj/project.pbxproj: * Target.pri: * heap/AllocationSpace.cpp: (JSC::AllocationSpace::allocateSlowCase): (JSC::AllocationSpace::allocateBlock): (JSC::AllocationSpace::freeBlocks): * heap/AllocationSpace.h: (JSC::AllocationSpace::waterMark): * heap/BumpBlock.h: Added. (JSC::BumpBlock::BumpBlock): * heap/BumpSpace.cpp: Added. (JSC::BumpSpace::tryAllocateSlowCase): * heap/BumpSpace.h: Added. (JSC::BumpSpace::isInCopyPhase): (JSC::BumpSpace::totalMemoryAllocated): (JSC::BumpSpace::totalMemoryUtilized): * heap/BumpSpaceInlineMethods.h: Added. (JSC::BumpSpace::BumpSpace): (JSC::BumpSpace::init): (JSC::BumpSpace::contains): (JSC::BumpSpace::pin): (JSC::BumpSpace::startedCopying): (JSC::BumpSpace::doneCopying): (JSC::BumpSpace::doneFillingBlock): (JSC::BumpSpace::recycleBlock): (JSC::BumpSpace::getFreshBlock): (JSC::BumpSpace::borrowBlock): (JSC::BumpSpace::addNewBlock): (JSC::BumpSpace::allocateNewBlock): (JSC::BumpSpace::fitsInBlock): (JSC::BumpSpace::fitsInCurrentBlock): (JSC::BumpSpace::tryAllocate): (JSC::BumpSpace::tryAllocateOversize): (JSC::BumpSpace::allocateFromBlock): (JSC::BumpSpace::tryReallocate): (JSC::BumpSpace::tryReallocateOversize): (JSC::BumpSpace::isOversize): (JSC::BumpSpace::isPinned): (JSC::BumpSpace::oversizeBlockFor): (JSC::BumpSpace::blockFor): * heap/ConservativeRoots.cpp: (JSC::ConservativeRoots::ConservativeRoots): (JSC::ConservativeRoots::genericAddPointer): (JSC::ConservativeRoots::add): * heap/ConservativeRoots.h: * heap/Heap.cpp: (JSC::Heap::Heap): (JSC::Heap::blockFreeingThreadMain): (JSC::Heap::reportExtraMemoryCostSlowCase): (JSC::Heap::getConservativeRegisterRoots): (JSC::Heap::markRoots): (JSC::Heap::collect): (JSC::Heap::releaseFreeBlocks): * heap/Heap.h: (JSC::Heap::waterMark): (JSC::Heap::highWaterMark): (JSC::Heap::setHighWaterMark): (JSC::Heap::tryAllocateStorage): (JSC::Heap::tryReallocateStorage): * heap/HeapBlock.h: Added. (JSC::HeapBlock::HeapBlock): * heap/MarkStack.cpp: (JSC::MarkStackThreadSharedData::MarkStackThreadSharedData): (JSC::SlotVisitor::drain): (JSC::SlotVisitor::drainFromShared): (JSC::SlotVisitor::startCopying): (JSC::SlotVisitor::allocateNewSpace): (JSC::SlotVisitor::copy): (JSC::SlotVisitor::copyAndAppend): (JSC::SlotVisitor::doneCopying): * heap/MarkStack.h: * heap/MarkedBlock.cpp: (JSC::MarkedBlock::recycle): (JSC::MarkedBlock::MarkedBlock): * heap/MarkedBlock.h: * heap/MarkedSpace.cpp: (JSC::MarkedSpace::MarkedSpace): * heap/MarkedSpace.h: (JSC::MarkedSpace::allocate): (JSC::MarkedSpace::forEachBlock): (JSC::MarkedSpace::SizeClass::resetAllocator): * heap/SlotVisitor.h: (JSC::SlotVisitor::SlotVisitor): * heap/TinyBloomFilter.h: (JSC::TinyBloomFilter::reset): * runtime/JSArray.cpp: (JSC::JSArray::JSArray): (JSC::JSArray::finishCreation): (JSC::JSArray::tryFinishCreationUninitialized): (JSC::JSArray::~JSArray): (JSC::JSArray::enterSparseMode): (JSC::JSArray::defineOwnNumericProperty): (JSC::JSArray::setLengthWritable): (JSC::JSArray::getOwnPropertySlotByIndex): (JSC::JSArray::getOwnPropertyDescriptor): (JSC::JSArray::putByIndexBeyondVectorLength): (JSC::JSArray::deletePropertyByIndex): (JSC::JSArray::getOwnPropertyNames): (JSC::JSArray::increaseVectorLength): (JSC::JSArray::unshiftCountSlowCase): (JSC::JSArray::setLength): (JSC::JSArray::pop): (JSC::JSArray::unshiftCount): (JSC::JSArray::visitChildren): (JSC::JSArray::sortNumeric): (JSC::JSArray::sort): (JSC::JSArray::compactForSorting): (JSC::JSArray::subclassData): (JSC::JSArray::setSubclassData): (JSC::JSArray::checkConsistency): * runtime/JSArray.h: (JSC::JSArray::inSparseMode): (JSC::JSArray::isLengthWritable): * wtf/CheckedBoolean.h: Added. (CheckedBoolean::CheckedBoolean): (CheckedBoolean::~CheckedBoolean): (CheckedBoolean::operator bool): * wtf/DoublyLinkedList.h: (WTF::::push): * wtf/StdLibExtras.h: (WTF::isPointerAligned): Source/JavaScriptGlue: Added forwarding header for new CheckedBoolean used in the bump allocator. * ForwardingHeaders/wtf/CheckedBoolean.h: Added. Source/WebCore: No new tests. Added forwarding header for new CheckedBoolean used in the bump allocator. * ForwardingHeaders/wtf/CheckedBoolean.h: Added. Canonical link: https://commits.webkit.org/93486@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@105442 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2012-01-19 21:49:56 +00:00
#endif
};