[JSC] Compress Watchpoint size by using enum type and Packed<> data structure
https://bugs.webkit.org/show_bug.cgi?id=197730
Reviewed by Filip Pizlo.
Source/JavaScriptCore:
Watchpoint takes 5~ MB memory in Gmail (total memory starts with 400 - 500 MB), so 1~%. Since it is allocated massively,
reducing each size of Watchpoint reduces memory footprint significantly.
As a first step, this patch uses Packed<> and enum to reduce the size of Watchpoint.
1. Watchpoint should have enum type and should not use vtable. vtable takes one pointer, and it is too costly for such a
memory sensitive objects. We perform downcast and dispatch the method of the derived classes based on this enum. Since
the # of derived Watchpoint classes are limited (Only 8), we can list up them easily. One unfortunate thing is that
we cannot do this for destructor so long as we use "delete" for deleting objects. If we dispatch the destructor of derived
class in the destructor of the base class, we call the destructor of the base class multiple times. delete operator override
does not help since custom delete operator is called after the destructor is called. While we can fix this issue by always
using custom deleter, currently we do not since all the watchpoints do not have members which have non trivial destructor.
Once it is strongly required, we can start using custom deleter, but for now, we do not need to do this.
2. We use Packed<> to compact pointers in Watchpoint. Since Watchpoint is a node of doubly linked list, each one has two
pointers for prev and next. This is also too costly. PackedPtr reduces the size and makes alignment 1.S
3. We use PackedCellPtr<> for JSCells in Watchpoint. This leverages alignment information and makes pointers smaller in
Darwin ARM64. One important thing to note here is that since this pointer is packed, it cannot be found by conservative
GC scan. It is OK for watchpoint since they are allocated in the heap anyway.
We applied this change to Watchpoint and get the following memory reduction. The highlight is that CodeBlockJettisoningWatchpoint in
ARM64 only takes 2 pointers size.
ORIGINAL X86_64 ARM64
WatchpointSet: 40 32 28
CodeBlockJettisoningWatchpoint: 32 19 15
StructureStubClearingWatchpoint: 56 48 40
AdaptiveInferredPropertyValueWatchpointBase::StructureWatchpoint: 24 13 11
AdaptiveInferredPropertyValueWatchpointBase::PropertyWatchpoint: 24 13 11
FunctionRareData::AllocationProfileClearingWatchpoint: 32 19 15
ObjectToStringAdaptiveStructureWatchpoint: 56 48 40
LLIntPrototypeLoadAdaptiveStructureWatchpoint: 64 48 48
DFG::AdaptiveStructureWatchpoint: 56 48 40
While we will re-architect the mechanism of Watchpoint, anyway Packed<> mechanism and enum types will be used too.
* CMakeLists.txt:
* JavaScriptCore.xcodeproj/project.pbxproj:
* Sources.txt:
* bytecode/AdaptiveInferredPropertyValueWatchpointBase.h:
* bytecode/CodeBlockJettisoningWatchpoint.h:
* bytecode/CodeOrigin.h:
* bytecode/LLIntPrototypeLoadAdaptiveStructureWatchpoint.cpp:
(JSC::LLIntPrototypeLoadAdaptiveStructureWatchpoint::LLIntPrototypeLoadAdaptiveStructureWatchpoint):
(JSC::LLIntPrototypeLoadAdaptiveStructureWatchpoint::fireInternal):
* bytecode/LLIntPrototypeLoadAdaptiveStructureWatchpoint.h:
* bytecode/StructureStubClearingWatchpoint.cpp:
(JSC::StructureStubClearingWatchpoint::fireInternal):
* bytecode/StructureStubClearingWatchpoint.h:
* bytecode/Watchpoint.cpp:
(JSC::Watchpoint::fire):
* bytecode/Watchpoint.h:
(JSC::Watchpoint::Watchpoint):
* dfg/DFGAdaptiveStructureWatchpoint.cpp:
(JSC::DFG::AdaptiveStructureWatchpoint::AdaptiveStructureWatchpoint):
* dfg/DFGAdaptiveStructureWatchpoint.h:
* heap/PackedCellPtr.h: Added.
* runtime/FunctionRareData.h:
* runtime/ObjectToStringAdaptiveStructureWatchpoint.cpp: Added.
(JSC::ObjectToStringAdaptiveStructureWatchpoint::ObjectToStringAdaptiveStructureWatchpoint):
(JSC::ObjectToStringAdaptiveStructureWatchpoint::install):
(JSC::ObjectToStringAdaptiveStructureWatchpoint::fireInternal):
* runtime/ObjectToStringAdaptiveStructureWatchpoint.h: Added.
* runtime/StructureRareData.cpp:
(JSC::StructureRareData::clearObjectToStringValue):
(JSC::ObjectToStringAdaptiveStructureWatchpoint::ObjectToStringAdaptiveStructureWatchpoint): Deleted.
(JSC::ObjectToStringAdaptiveStructureWatchpoint::install): Deleted.
(JSC::ObjectToStringAdaptiveStructureWatchpoint::fireInternal): Deleted.
* runtime/StructureRareData.h:
Source/WTF:
This patch introduces a new data structures, WTF::Packed, WTF::PackedPtr, and WTF::PackedAlignedPtr.
- WTF::Packed
WTF::Packed is data storage. We can read and write trivial (in C++ term [1]) data to this storage. The difference to
the usual storage is that the alignment of this storage is always 1. We access the underlying data by using unalignedLoad/unalignedStore.
This class offers alignment = 1 data structure instead of missing the following characteristics.
1. Load / Store are non atomic even if the data size is within a pointer width. We should not use this for a member which can be accessed
in a racy way. (e.g. fields accessed optimistically from the concurrent compilers).
2. We cannot take reference / pointer to the underlying storage since they are unaligned.
3. Access to this storage is unaligned access. The code is using memcpy, and the compiler will convert to an appropriate unaligned access
in certain architectures (x86_64 / ARM64). It could be slow. So use it for non performance sensitive & memory sensitive places.
- WTF::PackedPtr
WTF::PackedPtr is a specialization of WTF::Packed<T*>. And it is basically WTF::PackedAlignedPtr with alignment = 1. We further compact
the pointer by leveraging the platform specific knowledge. In 64bit architectures, the effective width of pointers are less than 64 bit.
In x86_64, it is 48 bits. And Darwin ARM64 is further smaller, 36 bits. This information allows us to compact the pointer to 6 bytes in
x86_64 and 5 bytes in Darwin ARM64.
- WTF::PackedAlignedPtr
WTF::PackedAlignedPtr is the WTF::PackedPtr with alignment information of the T. If we use this alignment information, we could reduce the
size of packed pointer further in some cases. For example, since we guarantee that JSCells are 16 byte aligned, low 4 bits are empty. Leveraging
this information in Darwin ARM64 platform allows us to make packed JSCell pointer 4 bytes (36 - 4 bits). We do not use passed alignment
information if it is not profitable.
We also have PackedPtrTraits. This is new PtrTraits and use it for various data structures such as Bag<>.
[1]: https://en.cppreference.com/w/cpp/types/is_trivial
* WTF.xcodeproj/project.pbxproj:
* wtf/Bag.h:
(WTF::Bag::clear):
(WTF::Bag::iterator::operator++):
* wtf/CMakeLists.txt:
* wtf/DumbPtrTraits.h:
* wtf/DumbValueTraits.h:
* wtf/MathExtras.h:
(WTF::clzConstexpr):
(WTF::clz):
(WTF::ctzConstexpr):
(WTF::ctz):
(WTF::getLSBSetConstexpr):
(WTF::getMSBSetConstexpr):
* wtf/Packed.h: Added.
(WTF::Packed::Packed):
(WTF::Packed::get const):
(WTF::Packed::set):
(WTF::Packed::operator=):
(WTF::Packed::exchange):
(WTF::Packed::swap):
(WTF::alignof):
(WTF::PackedPtrTraits::exchange):
(WTF::PackedPtrTraits::swap):
(WTF::PackedPtrTraits::unwrap):
* wtf/Platform.h:
* wtf/SentinelLinkedList.h:
(WTF::BasicRawSentinelNode::BasicRawSentinelNode):
(WTF::BasicRawSentinelNode::prev):
(WTF::BasicRawSentinelNode::next):
(WTF::PtrTraits>::remove):
(WTF::PtrTraits>::prepend):
(WTF::PtrTraits>::append):
(WTF::RawNode>::SentinelLinkedList):
(WTF::RawNode>::remove):
(WTF::BasicRawSentinelNode<T>::remove): Deleted.
(WTF::BasicRawSentinelNode<T>::prepend): Deleted.
(WTF::BasicRawSentinelNode<T>::append): Deleted.
* wtf/StdLibExtras.h:
(WTF::roundUpToMultipleOfImpl):
(WTF::roundUpToMultipleOfImpl0): Deleted.
* wtf/UnalignedAccess.h:
(WTF::unalignedLoad):
(WTF::unalignedStore):
Tools:
* TestWebKitAPI/CMakeLists.txt:
* TestWebKitAPI/TestWebKitAPI.xcodeproj/project.pbxproj:
* TestWebKitAPI/Tests/WTF/MathExtras.cpp:
(TestWebKitAPI::TEST):
* TestWebKitAPI/Tests/WTF/Packed.cpp: Added.
(TestWebKitAPI::TEST):
Canonical link: https://commits.webkit.org/211952@main
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@245214 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2019-05-12 22:50:21 +00:00
|
|
|
/*
|
|
|
|
* Copyright (C) 2019 Apple Inc. All rights reserved.
|
|
|
|
*
|
|
|
|
* Redistribution and use in source and binary forms, with or without
|
|
|
|
* modification, are permitted provided that the following conditions
|
|
|
|
* are met:
|
|
|
|
* 1. Redistributions of source code must retain the above copyright
|
|
|
|
* notice, this list of conditions and the following disclaimer.
|
|
|
|
* 2. Redistributions in binary form must reproduce the above copyright
|
|
|
|
* notice, this list of conditions and the following disclaimer in the
|
|
|
|
* documentation and/or other materials provided with the distribution.
|
|
|
|
*
|
|
|
|
* THIS SOFTWARE IS PROVIDED BY APPLE INC. ``AS IS'' AND ANY
|
|
|
|
* EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
|
|
|
|
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
|
|
|
|
* PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR
|
|
|
|
* CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
|
|
|
|
* EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
|
|
|
|
* PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
|
|
|
|
* PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
|
|
|
|
* OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
|
|
|
|
* (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
|
|
|
|
* OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
|
|
|
|
*/
|
|
|
|
|
|
|
|
#pragma once
|
|
|
|
|
|
|
|
#include <array>
|
2020-01-21 22:05:43 +00:00
|
|
|
#include <wtf/Forward.h>
|
|
|
|
#include <wtf/GetPtr.h>
|
|
|
|
#include <wtf/HashFunctions.h>
|
[JSC] Compress Watchpoint size by using enum type and Packed<> data structure
https://bugs.webkit.org/show_bug.cgi?id=197730
Reviewed by Filip Pizlo.
Source/JavaScriptCore:
Watchpoint takes 5~ MB memory in Gmail (total memory starts with 400 - 500 MB), so 1~%. Since it is allocated massively,
reducing each size of Watchpoint reduces memory footprint significantly.
As a first step, this patch uses Packed<> and enum to reduce the size of Watchpoint.
1. Watchpoint should have enum type and should not use vtable. vtable takes one pointer, and it is too costly for such a
memory sensitive objects. We perform downcast and dispatch the method of the derived classes based on this enum. Since
the # of derived Watchpoint classes are limited (Only 8), we can list up them easily. One unfortunate thing is that
we cannot do this for destructor so long as we use "delete" for deleting objects. If we dispatch the destructor of derived
class in the destructor of the base class, we call the destructor of the base class multiple times. delete operator override
does not help since custom delete operator is called after the destructor is called. While we can fix this issue by always
using custom deleter, currently we do not since all the watchpoints do not have members which have non trivial destructor.
Once it is strongly required, we can start using custom deleter, but for now, we do not need to do this.
2. We use Packed<> to compact pointers in Watchpoint. Since Watchpoint is a node of doubly linked list, each one has two
pointers for prev and next. This is also too costly. PackedPtr reduces the size and makes alignment 1.S
3. We use PackedCellPtr<> for JSCells in Watchpoint. This leverages alignment information and makes pointers smaller in
Darwin ARM64. One important thing to note here is that since this pointer is packed, it cannot be found by conservative
GC scan. It is OK for watchpoint since they are allocated in the heap anyway.
We applied this change to Watchpoint and get the following memory reduction. The highlight is that CodeBlockJettisoningWatchpoint in
ARM64 only takes 2 pointers size.
ORIGINAL X86_64 ARM64
WatchpointSet: 40 32 28
CodeBlockJettisoningWatchpoint: 32 19 15
StructureStubClearingWatchpoint: 56 48 40
AdaptiveInferredPropertyValueWatchpointBase::StructureWatchpoint: 24 13 11
AdaptiveInferredPropertyValueWatchpointBase::PropertyWatchpoint: 24 13 11
FunctionRareData::AllocationProfileClearingWatchpoint: 32 19 15
ObjectToStringAdaptiveStructureWatchpoint: 56 48 40
LLIntPrototypeLoadAdaptiveStructureWatchpoint: 64 48 48
DFG::AdaptiveStructureWatchpoint: 56 48 40
While we will re-architect the mechanism of Watchpoint, anyway Packed<> mechanism and enum types will be used too.
* CMakeLists.txt:
* JavaScriptCore.xcodeproj/project.pbxproj:
* Sources.txt:
* bytecode/AdaptiveInferredPropertyValueWatchpointBase.h:
* bytecode/CodeBlockJettisoningWatchpoint.h:
* bytecode/CodeOrigin.h:
* bytecode/LLIntPrototypeLoadAdaptiveStructureWatchpoint.cpp:
(JSC::LLIntPrototypeLoadAdaptiveStructureWatchpoint::LLIntPrototypeLoadAdaptiveStructureWatchpoint):
(JSC::LLIntPrototypeLoadAdaptiveStructureWatchpoint::fireInternal):
* bytecode/LLIntPrototypeLoadAdaptiveStructureWatchpoint.h:
* bytecode/StructureStubClearingWatchpoint.cpp:
(JSC::StructureStubClearingWatchpoint::fireInternal):
* bytecode/StructureStubClearingWatchpoint.h:
* bytecode/Watchpoint.cpp:
(JSC::Watchpoint::fire):
* bytecode/Watchpoint.h:
(JSC::Watchpoint::Watchpoint):
* dfg/DFGAdaptiveStructureWatchpoint.cpp:
(JSC::DFG::AdaptiveStructureWatchpoint::AdaptiveStructureWatchpoint):
* dfg/DFGAdaptiveStructureWatchpoint.h:
* heap/PackedCellPtr.h: Added.
* runtime/FunctionRareData.h:
* runtime/ObjectToStringAdaptiveStructureWatchpoint.cpp: Added.
(JSC::ObjectToStringAdaptiveStructureWatchpoint::ObjectToStringAdaptiveStructureWatchpoint):
(JSC::ObjectToStringAdaptiveStructureWatchpoint::install):
(JSC::ObjectToStringAdaptiveStructureWatchpoint::fireInternal):
* runtime/ObjectToStringAdaptiveStructureWatchpoint.h: Added.
* runtime/StructureRareData.cpp:
(JSC::StructureRareData::clearObjectToStringValue):
(JSC::ObjectToStringAdaptiveStructureWatchpoint::ObjectToStringAdaptiveStructureWatchpoint): Deleted.
(JSC::ObjectToStringAdaptiveStructureWatchpoint::install): Deleted.
(JSC::ObjectToStringAdaptiveStructureWatchpoint::fireInternal): Deleted.
* runtime/StructureRareData.h:
Source/WTF:
This patch introduces a new data structures, WTF::Packed, WTF::PackedPtr, and WTF::PackedAlignedPtr.
- WTF::Packed
WTF::Packed is data storage. We can read and write trivial (in C++ term [1]) data to this storage. The difference to
the usual storage is that the alignment of this storage is always 1. We access the underlying data by using unalignedLoad/unalignedStore.
This class offers alignment = 1 data structure instead of missing the following characteristics.
1. Load / Store are non atomic even if the data size is within a pointer width. We should not use this for a member which can be accessed
in a racy way. (e.g. fields accessed optimistically from the concurrent compilers).
2. We cannot take reference / pointer to the underlying storage since they are unaligned.
3. Access to this storage is unaligned access. The code is using memcpy, and the compiler will convert to an appropriate unaligned access
in certain architectures (x86_64 / ARM64). It could be slow. So use it for non performance sensitive & memory sensitive places.
- WTF::PackedPtr
WTF::PackedPtr is a specialization of WTF::Packed<T*>. And it is basically WTF::PackedAlignedPtr with alignment = 1. We further compact
the pointer by leveraging the platform specific knowledge. In 64bit architectures, the effective width of pointers are less than 64 bit.
In x86_64, it is 48 bits. And Darwin ARM64 is further smaller, 36 bits. This information allows us to compact the pointer to 6 bytes in
x86_64 and 5 bytes in Darwin ARM64.
- WTF::PackedAlignedPtr
WTF::PackedAlignedPtr is the WTF::PackedPtr with alignment information of the T. If we use this alignment information, we could reduce the
size of packed pointer further in some cases. For example, since we guarantee that JSCells are 16 byte aligned, low 4 bits are empty. Leveraging
this information in Darwin ARM64 platform allows us to make packed JSCell pointer 4 bytes (36 - 4 bits). We do not use passed alignment
information if it is not profitable.
We also have PackedPtrTraits. This is new PtrTraits and use it for various data structures such as Bag<>.
[1]: https://en.cppreference.com/w/cpp/types/is_trivial
* WTF.xcodeproj/project.pbxproj:
* wtf/Bag.h:
(WTF::Bag::clear):
(WTF::Bag::iterator::operator++):
* wtf/CMakeLists.txt:
* wtf/DumbPtrTraits.h:
* wtf/DumbValueTraits.h:
* wtf/MathExtras.h:
(WTF::clzConstexpr):
(WTF::clz):
(WTF::ctzConstexpr):
(WTF::ctz):
(WTF::getLSBSetConstexpr):
(WTF::getMSBSetConstexpr):
* wtf/Packed.h: Added.
(WTF::Packed::Packed):
(WTF::Packed::get const):
(WTF::Packed::set):
(WTF::Packed::operator=):
(WTF::Packed::exchange):
(WTF::Packed::swap):
(WTF::alignof):
(WTF::PackedPtrTraits::exchange):
(WTF::PackedPtrTraits::swap):
(WTF::PackedPtrTraits::unwrap):
* wtf/Platform.h:
* wtf/SentinelLinkedList.h:
(WTF::BasicRawSentinelNode::BasicRawSentinelNode):
(WTF::BasicRawSentinelNode::prev):
(WTF::BasicRawSentinelNode::next):
(WTF::PtrTraits>::remove):
(WTF::PtrTraits>::prepend):
(WTF::PtrTraits>::append):
(WTF::RawNode>::SentinelLinkedList):
(WTF::RawNode>::remove):
(WTF::BasicRawSentinelNode<T>::remove): Deleted.
(WTF::BasicRawSentinelNode<T>::prepend): Deleted.
(WTF::BasicRawSentinelNode<T>::append): Deleted.
* wtf/StdLibExtras.h:
(WTF::roundUpToMultipleOfImpl):
(WTF::roundUpToMultipleOfImpl0): Deleted.
* wtf/UnalignedAccess.h:
(WTF::unalignedLoad):
(WTF::unalignedStore):
Tools:
* TestWebKitAPI/CMakeLists.txt:
* TestWebKitAPI/TestWebKitAPI.xcodeproj/project.pbxproj:
* TestWebKitAPI/Tests/WTF/MathExtras.cpp:
(TestWebKitAPI::TEST):
* TestWebKitAPI/Tests/WTF/Packed.cpp: Added.
(TestWebKitAPI::TEST):
Canonical link: https://commits.webkit.org/211952@main
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@245214 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2019-05-12 22:50:21 +00:00
|
|
|
#include <wtf/MathExtras.h>
|
|
|
|
#include <wtf/StdLibExtras.h>
|
|
|
|
#include <wtf/UnalignedAccess.h>
|
|
|
|
|
2021-06-18 01:44:42 +00:00
|
|
|
#if OS(DARWIN)
|
|
|
|
#include <mach/vm_param.h>
|
|
|
|
#endif
|
|
|
|
|
[JSC] Compress Watchpoint size by using enum type and Packed<> data structure
https://bugs.webkit.org/show_bug.cgi?id=197730
Reviewed by Filip Pizlo.
Source/JavaScriptCore:
Watchpoint takes 5~ MB memory in Gmail (total memory starts with 400 - 500 MB), so 1~%. Since it is allocated massively,
reducing each size of Watchpoint reduces memory footprint significantly.
As a first step, this patch uses Packed<> and enum to reduce the size of Watchpoint.
1. Watchpoint should have enum type and should not use vtable. vtable takes one pointer, and it is too costly for such a
memory sensitive objects. We perform downcast and dispatch the method of the derived classes based on this enum. Since
the # of derived Watchpoint classes are limited (Only 8), we can list up them easily. One unfortunate thing is that
we cannot do this for destructor so long as we use "delete" for deleting objects. If we dispatch the destructor of derived
class in the destructor of the base class, we call the destructor of the base class multiple times. delete operator override
does not help since custom delete operator is called after the destructor is called. While we can fix this issue by always
using custom deleter, currently we do not since all the watchpoints do not have members which have non trivial destructor.
Once it is strongly required, we can start using custom deleter, but for now, we do not need to do this.
2. We use Packed<> to compact pointers in Watchpoint. Since Watchpoint is a node of doubly linked list, each one has two
pointers for prev and next. This is also too costly. PackedPtr reduces the size and makes alignment 1.S
3. We use PackedCellPtr<> for JSCells in Watchpoint. This leverages alignment information and makes pointers smaller in
Darwin ARM64. One important thing to note here is that since this pointer is packed, it cannot be found by conservative
GC scan. It is OK for watchpoint since they are allocated in the heap anyway.
We applied this change to Watchpoint and get the following memory reduction. The highlight is that CodeBlockJettisoningWatchpoint in
ARM64 only takes 2 pointers size.
ORIGINAL X86_64 ARM64
WatchpointSet: 40 32 28
CodeBlockJettisoningWatchpoint: 32 19 15
StructureStubClearingWatchpoint: 56 48 40
AdaptiveInferredPropertyValueWatchpointBase::StructureWatchpoint: 24 13 11
AdaptiveInferredPropertyValueWatchpointBase::PropertyWatchpoint: 24 13 11
FunctionRareData::AllocationProfileClearingWatchpoint: 32 19 15
ObjectToStringAdaptiveStructureWatchpoint: 56 48 40
LLIntPrototypeLoadAdaptiveStructureWatchpoint: 64 48 48
DFG::AdaptiveStructureWatchpoint: 56 48 40
While we will re-architect the mechanism of Watchpoint, anyway Packed<> mechanism and enum types will be used too.
* CMakeLists.txt:
* JavaScriptCore.xcodeproj/project.pbxproj:
* Sources.txt:
* bytecode/AdaptiveInferredPropertyValueWatchpointBase.h:
* bytecode/CodeBlockJettisoningWatchpoint.h:
* bytecode/CodeOrigin.h:
* bytecode/LLIntPrototypeLoadAdaptiveStructureWatchpoint.cpp:
(JSC::LLIntPrototypeLoadAdaptiveStructureWatchpoint::LLIntPrototypeLoadAdaptiveStructureWatchpoint):
(JSC::LLIntPrototypeLoadAdaptiveStructureWatchpoint::fireInternal):
* bytecode/LLIntPrototypeLoadAdaptiveStructureWatchpoint.h:
* bytecode/StructureStubClearingWatchpoint.cpp:
(JSC::StructureStubClearingWatchpoint::fireInternal):
* bytecode/StructureStubClearingWatchpoint.h:
* bytecode/Watchpoint.cpp:
(JSC::Watchpoint::fire):
* bytecode/Watchpoint.h:
(JSC::Watchpoint::Watchpoint):
* dfg/DFGAdaptiveStructureWatchpoint.cpp:
(JSC::DFG::AdaptiveStructureWatchpoint::AdaptiveStructureWatchpoint):
* dfg/DFGAdaptiveStructureWatchpoint.h:
* heap/PackedCellPtr.h: Added.
* runtime/FunctionRareData.h:
* runtime/ObjectToStringAdaptiveStructureWatchpoint.cpp: Added.
(JSC::ObjectToStringAdaptiveStructureWatchpoint::ObjectToStringAdaptiveStructureWatchpoint):
(JSC::ObjectToStringAdaptiveStructureWatchpoint::install):
(JSC::ObjectToStringAdaptiveStructureWatchpoint::fireInternal):
* runtime/ObjectToStringAdaptiveStructureWatchpoint.h: Added.
* runtime/StructureRareData.cpp:
(JSC::StructureRareData::clearObjectToStringValue):
(JSC::ObjectToStringAdaptiveStructureWatchpoint::ObjectToStringAdaptiveStructureWatchpoint): Deleted.
(JSC::ObjectToStringAdaptiveStructureWatchpoint::install): Deleted.
(JSC::ObjectToStringAdaptiveStructureWatchpoint::fireInternal): Deleted.
* runtime/StructureRareData.h:
Source/WTF:
This patch introduces a new data structures, WTF::Packed, WTF::PackedPtr, and WTF::PackedAlignedPtr.
- WTF::Packed
WTF::Packed is data storage. We can read and write trivial (in C++ term [1]) data to this storage. The difference to
the usual storage is that the alignment of this storage is always 1. We access the underlying data by using unalignedLoad/unalignedStore.
This class offers alignment = 1 data structure instead of missing the following characteristics.
1. Load / Store are non atomic even if the data size is within a pointer width. We should not use this for a member which can be accessed
in a racy way. (e.g. fields accessed optimistically from the concurrent compilers).
2. We cannot take reference / pointer to the underlying storage since they are unaligned.
3. Access to this storage is unaligned access. The code is using memcpy, and the compiler will convert to an appropriate unaligned access
in certain architectures (x86_64 / ARM64). It could be slow. So use it for non performance sensitive & memory sensitive places.
- WTF::PackedPtr
WTF::PackedPtr is a specialization of WTF::Packed<T*>. And it is basically WTF::PackedAlignedPtr with alignment = 1. We further compact
the pointer by leveraging the platform specific knowledge. In 64bit architectures, the effective width of pointers are less than 64 bit.
In x86_64, it is 48 bits. And Darwin ARM64 is further smaller, 36 bits. This information allows us to compact the pointer to 6 bytes in
x86_64 and 5 bytes in Darwin ARM64.
- WTF::PackedAlignedPtr
WTF::PackedAlignedPtr is the WTF::PackedPtr with alignment information of the T. If we use this alignment information, we could reduce the
size of packed pointer further in some cases. For example, since we guarantee that JSCells are 16 byte aligned, low 4 bits are empty. Leveraging
this information in Darwin ARM64 platform allows us to make packed JSCell pointer 4 bytes (36 - 4 bits). We do not use passed alignment
information if it is not profitable.
We also have PackedPtrTraits. This is new PtrTraits and use it for various data structures such as Bag<>.
[1]: https://en.cppreference.com/w/cpp/types/is_trivial
* WTF.xcodeproj/project.pbxproj:
* wtf/Bag.h:
(WTF::Bag::clear):
(WTF::Bag::iterator::operator++):
* wtf/CMakeLists.txt:
* wtf/DumbPtrTraits.h:
* wtf/DumbValueTraits.h:
* wtf/MathExtras.h:
(WTF::clzConstexpr):
(WTF::clz):
(WTF::ctzConstexpr):
(WTF::ctz):
(WTF::getLSBSetConstexpr):
(WTF::getMSBSetConstexpr):
* wtf/Packed.h: Added.
(WTF::Packed::Packed):
(WTF::Packed::get const):
(WTF::Packed::set):
(WTF::Packed::operator=):
(WTF::Packed::exchange):
(WTF::Packed::swap):
(WTF::alignof):
(WTF::PackedPtrTraits::exchange):
(WTF::PackedPtrTraits::swap):
(WTF::PackedPtrTraits::unwrap):
* wtf/Platform.h:
* wtf/SentinelLinkedList.h:
(WTF::BasicRawSentinelNode::BasicRawSentinelNode):
(WTF::BasicRawSentinelNode::prev):
(WTF::BasicRawSentinelNode::next):
(WTF::PtrTraits>::remove):
(WTF::PtrTraits>::prepend):
(WTF::PtrTraits>::append):
(WTF::RawNode>::SentinelLinkedList):
(WTF::RawNode>::remove):
(WTF::BasicRawSentinelNode<T>::remove): Deleted.
(WTF::BasicRawSentinelNode<T>::prepend): Deleted.
(WTF::BasicRawSentinelNode<T>::append): Deleted.
* wtf/StdLibExtras.h:
(WTF::roundUpToMultipleOfImpl):
(WTF::roundUpToMultipleOfImpl0): Deleted.
* wtf/UnalignedAccess.h:
(WTF::unalignedLoad):
(WTF::unalignedStore):
Tools:
* TestWebKitAPI/CMakeLists.txt:
* TestWebKitAPI/TestWebKitAPI.xcodeproj/project.pbxproj:
* TestWebKitAPI/Tests/WTF/MathExtras.cpp:
(TestWebKitAPI::TEST):
* TestWebKitAPI/Tests/WTF/Packed.cpp: Added.
(TestWebKitAPI::TEST):
Canonical link: https://commits.webkit.org/211952@main
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@245214 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2019-05-12 22:50:21 +00:00
|
|
|
namespace WTF {
|
|
|
|
|
|
|
|
template<typename T>
|
|
|
|
class Packed {
|
2019-08-12 20:57:15 +00:00
|
|
|
WTF_MAKE_FAST_ALLOCATED;
|
[JSC] Compress Watchpoint size by using enum type and Packed<> data structure
https://bugs.webkit.org/show_bug.cgi?id=197730
Reviewed by Filip Pizlo.
Source/JavaScriptCore:
Watchpoint takes 5~ MB memory in Gmail (total memory starts with 400 - 500 MB), so 1~%. Since it is allocated massively,
reducing each size of Watchpoint reduces memory footprint significantly.
As a first step, this patch uses Packed<> and enum to reduce the size of Watchpoint.
1. Watchpoint should have enum type and should not use vtable. vtable takes one pointer, and it is too costly for such a
memory sensitive objects. We perform downcast and dispatch the method of the derived classes based on this enum. Since
the # of derived Watchpoint classes are limited (Only 8), we can list up them easily. One unfortunate thing is that
we cannot do this for destructor so long as we use "delete" for deleting objects. If we dispatch the destructor of derived
class in the destructor of the base class, we call the destructor of the base class multiple times. delete operator override
does not help since custom delete operator is called after the destructor is called. While we can fix this issue by always
using custom deleter, currently we do not since all the watchpoints do not have members which have non trivial destructor.
Once it is strongly required, we can start using custom deleter, but for now, we do not need to do this.
2. We use Packed<> to compact pointers in Watchpoint. Since Watchpoint is a node of doubly linked list, each one has two
pointers for prev and next. This is also too costly. PackedPtr reduces the size and makes alignment 1.S
3. We use PackedCellPtr<> for JSCells in Watchpoint. This leverages alignment information and makes pointers smaller in
Darwin ARM64. One important thing to note here is that since this pointer is packed, it cannot be found by conservative
GC scan. It is OK for watchpoint since they are allocated in the heap anyway.
We applied this change to Watchpoint and get the following memory reduction. The highlight is that CodeBlockJettisoningWatchpoint in
ARM64 only takes 2 pointers size.
ORIGINAL X86_64 ARM64
WatchpointSet: 40 32 28
CodeBlockJettisoningWatchpoint: 32 19 15
StructureStubClearingWatchpoint: 56 48 40
AdaptiveInferredPropertyValueWatchpointBase::StructureWatchpoint: 24 13 11
AdaptiveInferredPropertyValueWatchpointBase::PropertyWatchpoint: 24 13 11
FunctionRareData::AllocationProfileClearingWatchpoint: 32 19 15
ObjectToStringAdaptiveStructureWatchpoint: 56 48 40
LLIntPrototypeLoadAdaptiveStructureWatchpoint: 64 48 48
DFG::AdaptiveStructureWatchpoint: 56 48 40
While we will re-architect the mechanism of Watchpoint, anyway Packed<> mechanism and enum types will be used too.
* CMakeLists.txt:
* JavaScriptCore.xcodeproj/project.pbxproj:
* Sources.txt:
* bytecode/AdaptiveInferredPropertyValueWatchpointBase.h:
* bytecode/CodeBlockJettisoningWatchpoint.h:
* bytecode/CodeOrigin.h:
* bytecode/LLIntPrototypeLoadAdaptiveStructureWatchpoint.cpp:
(JSC::LLIntPrototypeLoadAdaptiveStructureWatchpoint::LLIntPrototypeLoadAdaptiveStructureWatchpoint):
(JSC::LLIntPrototypeLoadAdaptiveStructureWatchpoint::fireInternal):
* bytecode/LLIntPrototypeLoadAdaptiveStructureWatchpoint.h:
* bytecode/StructureStubClearingWatchpoint.cpp:
(JSC::StructureStubClearingWatchpoint::fireInternal):
* bytecode/StructureStubClearingWatchpoint.h:
* bytecode/Watchpoint.cpp:
(JSC::Watchpoint::fire):
* bytecode/Watchpoint.h:
(JSC::Watchpoint::Watchpoint):
* dfg/DFGAdaptiveStructureWatchpoint.cpp:
(JSC::DFG::AdaptiveStructureWatchpoint::AdaptiveStructureWatchpoint):
* dfg/DFGAdaptiveStructureWatchpoint.h:
* heap/PackedCellPtr.h: Added.
* runtime/FunctionRareData.h:
* runtime/ObjectToStringAdaptiveStructureWatchpoint.cpp: Added.
(JSC::ObjectToStringAdaptiveStructureWatchpoint::ObjectToStringAdaptiveStructureWatchpoint):
(JSC::ObjectToStringAdaptiveStructureWatchpoint::install):
(JSC::ObjectToStringAdaptiveStructureWatchpoint::fireInternal):
* runtime/ObjectToStringAdaptiveStructureWatchpoint.h: Added.
* runtime/StructureRareData.cpp:
(JSC::StructureRareData::clearObjectToStringValue):
(JSC::ObjectToStringAdaptiveStructureWatchpoint::ObjectToStringAdaptiveStructureWatchpoint): Deleted.
(JSC::ObjectToStringAdaptiveStructureWatchpoint::install): Deleted.
(JSC::ObjectToStringAdaptiveStructureWatchpoint::fireInternal): Deleted.
* runtime/StructureRareData.h:
Source/WTF:
This patch introduces a new data structures, WTF::Packed, WTF::PackedPtr, and WTF::PackedAlignedPtr.
- WTF::Packed
WTF::Packed is data storage. We can read and write trivial (in C++ term [1]) data to this storage. The difference to
the usual storage is that the alignment of this storage is always 1. We access the underlying data by using unalignedLoad/unalignedStore.
This class offers alignment = 1 data structure instead of missing the following characteristics.
1. Load / Store are non atomic even if the data size is within a pointer width. We should not use this for a member which can be accessed
in a racy way. (e.g. fields accessed optimistically from the concurrent compilers).
2. We cannot take reference / pointer to the underlying storage since they are unaligned.
3. Access to this storage is unaligned access. The code is using memcpy, and the compiler will convert to an appropriate unaligned access
in certain architectures (x86_64 / ARM64). It could be slow. So use it for non performance sensitive & memory sensitive places.
- WTF::PackedPtr
WTF::PackedPtr is a specialization of WTF::Packed<T*>. And it is basically WTF::PackedAlignedPtr with alignment = 1. We further compact
the pointer by leveraging the platform specific knowledge. In 64bit architectures, the effective width of pointers are less than 64 bit.
In x86_64, it is 48 bits. And Darwin ARM64 is further smaller, 36 bits. This information allows us to compact the pointer to 6 bytes in
x86_64 and 5 bytes in Darwin ARM64.
- WTF::PackedAlignedPtr
WTF::PackedAlignedPtr is the WTF::PackedPtr with alignment information of the T. If we use this alignment information, we could reduce the
size of packed pointer further in some cases. For example, since we guarantee that JSCells are 16 byte aligned, low 4 bits are empty. Leveraging
this information in Darwin ARM64 platform allows us to make packed JSCell pointer 4 bytes (36 - 4 bits). We do not use passed alignment
information if it is not profitable.
We also have PackedPtrTraits. This is new PtrTraits and use it for various data structures such as Bag<>.
[1]: https://en.cppreference.com/w/cpp/types/is_trivial
* WTF.xcodeproj/project.pbxproj:
* wtf/Bag.h:
(WTF::Bag::clear):
(WTF::Bag::iterator::operator++):
* wtf/CMakeLists.txt:
* wtf/DumbPtrTraits.h:
* wtf/DumbValueTraits.h:
* wtf/MathExtras.h:
(WTF::clzConstexpr):
(WTF::clz):
(WTF::ctzConstexpr):
(WTF::ctz):
(WTF::getLSBSetConstexpr):
(WTF::getMSBSetConstexpr):
* wtf/Packed.h: Added.
(WTF::Packed::Packed):
(WTF::Packed::get const):
(WTF::Packed::set):
(WTF::Packed::operator=):
(WTF::Packed::exchange):
(WTF::Packed::swap):
(WTF::alignof):
(WTF::PackedPtrTraits::exchange):
(WTF::PackedPtrTraits::swap):
(WTF::PackedPtrTraits::unwrap):
* wtf/Platform.h:
* wtf/SentinelLinkedList.h:
(WTF::BasicRawSentinelNode::BasicRawSentinelNode):
(WTF::BasicRawSentinelNode::prev):
(WTF::BasicRawSentinelNode::next):
(WTF::PtrTraits>::remove):
(WTF::PtrTraits>::prepend):
(WTF::PtrTraits>::append):
(WTF::RawNode>::SentinelLinkedList):
(WTF::RawNode>::remove):
(WTF::BasicRawSentinelNode<T>::remove): Deleted.
(WTF::BasicRawSentinelNode<T>::prepend): Deleted.
(WTF::BasicRawSentinelNode<T>::append): Deleted.
* wtf/StdLibExtras.h:
(WTF::roundUpToMultipleOfImpl):
(WTF::roundUpToMultipleOfImpl0): Deleted.
* wtf/UnalignedAccess.h:
(WTF::unalignedLoad):
(WTF::unalignedStore):
Tools:
* TestWebKitAPI/CMakeLists.txt:
* TestWebKitAPI/TestWebKitAPI.xcodeproj/project.pbxproj:
* TestWebKitAPI/Tests/WTF/MathExtras.cpp:
(TestWebKitAPI::TEST):
* TestWebKitAPI/Tests/WTF/Packed.cpp: Added.
(TestWebKitAPI::TEST):
Canonical link: https://commits.webkit.org/211952@main
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@245214 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2019-05-12 22:50:21 +00:00
|
|
|
public:
|
|
|
|
static constexpr bool isPackedType = true;
|
|
|
|
|
|
|
|
Packed()
|
|
|
|
: Packed(T { })
|
|
|
|
{
|
|
|
|
}
|
|
|
|
|
|
|
|
Packed(const T& value)
|
|
|
|
{
|
|
|
|
unalignedStore<T>(m_storage.data(), value);
|
|
|
|
}
|
|
|
|
|
|
|
|
T get() const
|
|
|
|
{
|
|
|
|
return unalignedLoad<T>(m_storage.data());
|
|
|
|
}
|
|
|
|
|
|
|
|
void set(const T& value)
|
|
|
|
{
|
|
|
|
unalignedStore<T>(m_storage.data(), value);
|
|
|
|
}
|
|
|
|
|
|
|
|
Packed<T>& operator=(const T& value)
|
|
|
|
{
|
|
|
|
set(value);
|
|
|
|
return *this;
|
|
|
|
}
|
|
|
|
|
|
|
|
template<class U>
|
|
|
|
T exchange(U&& newValue)
|
|
|
|
{
|
|
|
|
T oldValue = get();
|
|
|
|
set(std::forward<U>(newValue));
|
|
|
|
return oldValue;
|
|
|
|
}
|
|
|
|
|
|
|
|
void swap(Packed& other)
|
|
|
|
{
|
|
|
|
m_storage.swap(other.m_storage);
|
|
|
|
}
|
|
|
|
|
|
|
|
template<typename Other, typename = std::enable_if_t<Other::isPackedType>>
|
|
|
|
void swap(Other& other)
|
|
|
|
{
|
|
|
|
T t1 = get();
|
|
|
|
T t2 = other.get();
|
|
|
|
set(t2);
|
|
|
|
other.set(t1);
|
|
|
|
}
|
|
|
|
|
|
|
|
void swap(T& t2)
|
|
|
|
{
|
|
|
|
T t1 = get();
|
|
|
|
std::swap(t1, t2);
|
|
|
|
set(t1);
|
|
|
|
}
|
|
|
|
|
|
|
|
private:
|
|
|
|
std::array<uint8_t, sizeof(T)> m_storage;
|
|
|
|
};
|
|
|
|
|
|
|
|
// PackedAlignedPtr can take alignment parameter too. PackedAlignedPtr only uses this alignment information if it is profitable: we use
|
2020-02-25 10:37:36 +00:00
|
|
|
// alignment information only when we can reduce the size of the storage.
|
2020-01-21 22:05:43 +00:00
|
|
|
template<typename T, size_t passedAlignment>
|
[JSC] Compress Watchpoint size by using enum type and Packed<> data structure
https://bugs.webkit.org/show_bug.cgi?id=197730
Reviewed by Filip Pizlo.
Source/JavaScriptCore:
Watchpoint takes 5~ MB memory in Gmail (total memory starts with 400 - 500 MB), so 1~%. Since it is allocated massively,
reducing each size of Watchpoint reduces memory footprint significantly.
As a first step, this patch uses Packed<> and enum to reduce the size of Watchpoint.
1. Watchpoint should have enum type and should not use vtable. vtable takes one pointer, and it is too costly for such a
memory sensitive objects. We perform downcast and dispatch the method of the derived classes based on this enum. Since
the # of derived Watchpoint classes are limited (Only 8), we can list up them easily. One unfortunate thing is that
we cannot do this for destructor so long as we use "delete" for deleting objects. If we dispatch the destructor of derived
class in the destructor of the base class, we call the destructor of the base class multiple times. delete operator override
does not help since custom delete operator is called after the destructor is called. While we can fix this issue by always
using custom deleter, currently we do not since all the watchpoints do not have members which have non trivial destructor.
Once it is strongly required, we can start using custom deleter, but for now, we do not need to do this.
2. We use Packed<> to compact pointers in Watchpoint. Since Watchpoint is a node of doubly linked list, each one has two
pointers for prev and next. This is also too costly. PackedPtr reduces the size and makes alignment 1.S
3. We use PackedCellPtr<> for JSCells in Watchpoint. This leverages alignment information and makes pointers smaller in
Darwin ARM64. One important thing to note here is that since this pointer is packed, it cannot be found by conservative
GC scan. It is OK for watchpoint since they are allocated in the heap anyway.
We applied this change to Watchpoint and get the following memory reduction. The highlight is that CodeBlockJettisoningWatchpoint in
ARM64 only takes 2 pointers size.
ORIGINAL X86_64 ARM64
WatchpointSet: 40 32 28
CodeBlockJettisoningWatchpoint: 32 19 15
StructureStubClearingWatchpoint: 56 48 40
AdaptiveInferredPropertyValueWatchpointBase::StructureWatchpoint: 24 13 11
AdaptiveInferredPropertyValueWatchpointBase::PropertyWatchpoint: 24 13 11
FunctionRareData::AllocationProfileClearingWatchpoint: 32 19 15
ObjectToStringAdaptiveStructureWatchpoint: 56 48 40
LLIntPrototypeLoadAdaptiveStructureWatchpoint: 64 48 48
DFG::AdaptiveStructureWatchpoint: 56 48 40
While we will re-architect the mechanism of Watchpoint, anyway Packed<> mechanism and enum types will be used too.
* CMakeLists.txt:
* JavaScriptCore.xcodeproj/project.pbxproj:
* Sources.txt:
* bytecode/AdaptiveInferredPropertyValueWatchpointBase.h:
* bytecode/CodeBlockJettisoningWatchpoint.h:
* bytecode/CodeOrigin.h:
* bytecode/LLIntPrototypeLoadAdaptiveStructureWatchpoint.cpp:
(JSC::LLIntPrototypeLoadAdaptiveStructureWatchpoint::LLIntPrototypeLoadAdaptiveStructureWatchpoint):
(JSC::LLIntPrototypeLoadAdaptiveStructureWatchpoint::fireInternal):
* bytecode/LLIntPrototypeLoadAdaptiveStructureWatchpoint.h:
* bytecode/StructureStubClearingWatchpoint.cpp:
(JSC::StructureStubClearingWatchpoint::fireInternal):
* bytecode/StructureStubClearingWatchpoint.h:
* bytecode/Watchpoint.cpp:
(JSC::Watchpoint::fire):
* bytecode/Watchpoint.h:
(JSC::Watchpoint::Watchpoint):
* dfg/DFGAdaptiveStructureWatchpoint.cpp:
(JSC::DFG::AdaptiveStructureWatchpoint::AdaptiveStructureWatchpoint):
* dfg/DFGAdaptiveStructureWatchpoint.h:
* heap/PackedCellPtr.h: Added.
* runtime/FunctionRareData.h:
* runtime/ObjectToStringAdaptiveStructureWatchpoint.cpp: Added.
(JSC::ObjectToStringAdaptiveStructureWatchpoint::ObjectToStringAdaptiveStructureWatchpoint):
(JSC::ObjectToStringAdaptiveStructureWatchpoint::install):
(JSC::ObjectToStringAdaptiveStructureWatchpoint::fireInternal):
* runtime/ObjectToStringAdaptiveStructureWatchpoint.h: Added.
* runtime/StructureRareData.cpp:
(JSC::StructureRareData::clearObjectToStringValue):
(JSC::ObjectToStringAdaptiveStructureWatchpoint::ObjectToStringAdaptiveStructureWatchpoint): Deleted.
(JSC::ObjectToStringAdaptiveStructureWatchpoint::install): Deleted.
(JSC::ObjectToStringAdaptiveStructureWatchpoint::fireInternal): Deleted.
* runtime/StructureRareData.h:
Source/WTF:
This patch introduces a new data structures, WTF::Packed, WTF::PackedPtr, and WTF::PackedAlignedPtr.
- WTF::Packed
WTF::Packed is data storage. We can read and write trivial (in C++ term [1]) data to this storage. The difference to
the usual storage is that the alignment of this storage is always 1. We access the underlying data by using unalignedLoad/unalignedStore.
This class offers alignment = 1 data structure instead of missing the following characteristics.
1. Load / Store are non atomic even if the data size is within a pointer width. We should not use this for a member which can be accessed
in a racy way. (e.g. fields accessed optimistically from the concurrent compilers).
2. We cannot take reference / pointer to the underlying storage since they are unaligned.
3. Access to this storage is unaligned access. The code is using memcpy, and the compiler will convert to an appropriate unaligned access
in certain architectures (x86_64 / ARM64). It could be slow. So use it for non performance sensitive & memory sensitive places.
- WTF::PackedPtr
WTF::PackedPtr is a specialization of WTF::Packed<T*>. And it is basically WTF::PackedAlignedPtr with alignment = 1. We further compact
the pointer by leveraging the platform specific knowledge. In 64bit architectures, the effective width of pointers are less than 64 bit.
In x86_64, it is 48 bits. And Darwin ARM64 is further smaller, 36 bits. This information allows us to compact the pointer to 6 bytes in
x86_64 and 5 bytes in Darwin ARM64.
- WTF::PackedAlignedPtr
WTF::PackedAlignedPtr is the WTF::PackedPtr with alignment information of the T. If we use this alignment information, we could reduce the
size of packed pointer further in some cases. For example, since we guarantee that JSCells are 16 byte aligned, low 4 bits are empty. Leveraging
this information in Darwin ARM64 platform allows us to make packed JSCell pointer 4 bytes (36 - 4 bits). We do not use passed alignment
information if it is not profitable.
We also have PackedPtrTraits. This is new PtrTraits and use it for various data structures such as Bag<>.
[1]: https://en.cppreference.com/w/cpp/types/is_trivial
* WTF.xcodeproj/project.pbxproj:
* wtf/Bag.h:
(WTF::Bag::clear):
(WTF::Bag::iterator::operator++):
* wtf/CMakeLists.txt:
* wtf/DumbPtrTraits.h:
* wtf/DumbValueTraits.h:
* wtf/MathExtras.h:
(WTF::clzConstexpr):
(WTF::clz):
(WTF::ctzConstexpr):
(WTF::ctz):
(WTF::getLSBSetConstexpr):
(WTF::getMSBSetConstexpr):
* wtf/Packed.h: Added.
(WTF::Packed::Packed):
(WTF::Packed::get const):
(WTF::Packed::set):
(WTF::Packed::operator=):
(WTF::Packed::exchange):
(WTF::Packed::swap):
(WTF::alignof):
(WTF::PackedPtrTraits::exchange):
(WTF::PackedPtrTraits::swap):
(WTF::PackedPtrTraits::unwrap):
* wtf/Platform.h:
* wtf/SentinelLinkedList.h:
(WTF::BasicRawSentinelNode::BasicRawSentinelNode):
(WTF::BasicRawSentinelNode::prev):
(WTF::BasicRawSentinelNode::next):
(WTF::PtrTraits>::remove):
(WTF::PtrTraits>::prepend):
(WTF::PtrTraits>::append):
(WTF::RawNode>::SentinelLinkedList):
(WTF::RawNode>::remove):
(WTF::BasicRawSentinelNode<T>::remove): Deleted.
(WTF::BasicRawSentinelNode<T>::prepend): Deleted.
(WTF::BasicRawSentinelNode<T>::append): Deleted.
* wtf/StdLibExtras.h:
(WTF::roundUpToMultipleOfImpl):
(WTF::roundUpToMultipleOfImpl0): Deleted.
* wtf/UnalignedAccess.h:
(WTF::unalignedLoad):
(WTF::unalignedStore):
Tools:
* TestWebKitAPI/CMakeLists.txt:
* TestWebKitAPI/TestWebKitAPI.xcodeproj/project.pbxproj:
* TestWebKitAPI/Tests/WTF/MathExtras.cpp:
(TestWebKitAPI::TEST):
* TestWebKitAPI/Tests/WTF/Packed.cpp: Added.
(TestWebKitAPI::TEST):
Canonical link: https://commits.webkit.org/211952@main
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@245214 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2019-05-12 22:50:21 +00:00
|
|
|
class PackedAlignedPtr {
|
2019-08-12 20:57:15 +00:00
|
|
|
WTF_MAKE_FAST_ALLOCATED;
|
[JSC] Compress Watchpoint size by using enum type and Packed<> data structure
https://bugs.webkit.org/show_bug.cgi?id=197730
Reviewed by Filip Pizlo.
Source/JavaScriptCore:
Watchpoint takes 5~ MB memory in Gmail (total memory starts with 400 - 500 MB), so 1~%. Since it is allocated massively,
reducing each size of Watchpoint reduces memory footprint significantly.
As a first step, this patch uses Packed<> and enum to reduce the size of Watchpoint.
1. Watchpoint should have enum type and should not use vtable. vtable takes one pointer, and it is too costly for such a
memory sensitive objects. We perform downcast and dispatch the method of the derived classes based on this enum. Since
the # of derived Watchpoint classes are limited (Only 8), we can list up them easily. One unfortunate thing is that
we cannot do this for destructor so long as we use "delete" for deleting objects. If we dispatch the destructor of derived
class in the destructor of the base class, we call the destructor of the base class multiple times. delete operator override
does not help since custom delete operator is called after the destructor is called. While we can fix this issue by always
using custom deleter, currently we do not since all the watchpoints do not have members which have non trivial destructor.
Once it is strongly required, we can start using custom deleter, but for now, we do not need to do this.
2. We use Packed<> to compact pointers in Watchpoint. Since Watchpoint is a node of doubly linked list, each one has two
pointers for prev and next. This is also too costly. PackedPtr reduces the size and makes alignment 1.S
3. We use PackedCellPtr<> for JSCells in Watchpoint. This leverages alignment information and makes pointers smaller in
Darwin ARM64. One important thing to note here is that since this pointer is packed, it cannot be found by conservative
GC scan. It is OK for watchpoint since they are allocated in the heap anyway.
We applied this change to Watchpoint and get the following memory reduction. The highlight is that CodeBlockJettisoningWatchpoint in
ARM64 only takes 2 pointers size.
ORIGINAL X86_64 ARM64
WatchpointSet: 40 32 28
CodeBlockJettisoningWatchpoint: 32 19 15
StructureStubClearingWatchpoint: 56 48 40
AdaptiveInferredPropertyValueWatchpointBase::StructureWatchpoint: 24 13 11
AdaptiveInferredPropertyValueWatchpointBase::PropertyWatchpoint: 24 13 11
FunctionRareData::AllocationProfileClearingWatchpoint: 32 19 15
ObjectToStringAdaptiveStructureWatchpoint: 56 48 40
LLIntPrototypeLoadAdaptiveStructureWatchpoint: 64 48 48
DFG::AdaptiveStructureWatchpoint: 56 48 40
While we will re-architect the mechanism of Watchpoint, anyway Packed<> mechanism and enum types will be used too.
* CMakeLists.txt:
* JavaScriptCore.xcodeproj/project.pbxproj:
* Sources.txt:
* bytecode/AdaptiveInferredPropertyValueWatchpointBase.h:
* bytecode/CodeBlockJettisoningWatchpoint.h:
* bytecode/CodeOrigin.h:
* bytecode/LLIntPrototypeLoadAdaptiveStructureWatchpoint.cpp:
(JSC::LLIntPrototypeLoadAdaptiveStructureWatchpoint::LLIntPrototypeLoadAdaptiveStructureWatchpoint):
(JSC::LLIntPrototypeLoadAdaptiveStructureWatchpoint::fireInternal):
* bytecode/LLIntPrototypeLoadAdaptiveStructureWatchpoint.h:
* bytecode/StructureStubClearingWatchpoint.cpp:
(JSC::StructureStubClearingWatchpoint::fireInternal):
* bytecode/StructureStubClearingWatchpoint.h:
* bytecode/Watchpoint.cpp:
(JSC::Watchpoint::fire):
* bytecode/Watchpoint.h:
(JSC::Watchpoint::Watchpoint):
* dfg/DFGAdaptiveStructureWatchpoint.cpp:
(JSC::DFG::AdaptiveStructureWatchpoint::AdaptiveStructureWatchpoint):
* dfg/DFGAdaptiveStructureWatchpoint.h:
* heap/PackedCellPtr.h: Added.
* runtime/FunctionRareData.h:
* runtime/ObjectToStringAdaptiveStructureWatchpoint.cpp: Added.
(JSC::ObjectToStringAdaptiveStructureWatchpoint::ObjectToStringAdaptiveStructureWatchpoint):
(JSC::ObjectToStringAdaptiveStructureWatchpoint::install):
(JSC::ObjectToStringAdaptiveStructureWatchpoint::fireInternal):
* runtime/ObjectToStringAdaptiveStructureWatchpoint.h: Added.
* runtime/StructureRareData.cpp:
(JSC::StructureRareData::clearObjectToStringValue):
(JSC::ObjectToStringAdaptiveStructureWatchpoint::ObjectToStringAdaptiveStructureWatchpoint): Deleted.
(JSC::ObjectToStringAdaptiveStructureWatchpoint::install): Deleted.
(JSC::ObjectToStringAdaptiveStructureWatchpoint::fireInternal): Deleted.
* runtime/StructureRareData.h:
Source/WTF:
This patch introduces a new data structures, WTF::Packed, WTF::PackedPtr, and WTF::PackedAlignedPtr.
- WTF::Packed
WTF::Packed is data storage. We can read and write trivial (in C++ term [1]) data to this storage. The difference to
the usual storage is that the alignment of this storage is always 1. We access the underlying data by using unalignedLoad/unalignedStore.
This class offers alignment = 1 data structure instead of missing the following characteristics.
1. Load / Store are non atomic even if the data size is within a pointer width. We should not use this for a member which can be accessed
in a racy way. (e.g. fields accessed optimistically from the concurrent compilers).
2. We cannot take reference / pointer to the underlying storage since they are unaligned.
3. Access to this storage is unaligned access. The code is using memcpy, and the compiler will convert to an appropriate unaligned access
in certain architectures (x86_64 / ARM64). It could be slow. So use it for non performance sensitive & memory sensitive places.
- WTF::PackedPtr
WTF::PackedPtr is a specialization of WTF::Packed<T*>. And it is basically WTF::PackedAlignedPtr with alignment = 1. We further compact
the pointer by leveraging the platform specific knowledge. In 64bit architectures, the effective width of pointers are less than 64 bit.
In x86_64, it is 48 bits. And Darwin ARM64 is further smaller, 36 bits. This information allows us to compact the pointer to 6 bytes in
x86_64 and 5 bytes in Darwin ARM64.
- WTF::PackedAlignedPtr
WTF::PackedAlignedPtr is the WTF::PackedPtr with alignment information of the T. If we use this alignment information, we could reduce the
size of packed pointer further in some cases. For example, since we guarantee that JSCells are 16 byte aligned, low 4 bits are empty. Leveraging
this information in Darwin ARM64 platform allows us to make packed JSCell pointer 4 bytes (36 - 4 bits). We do not use passed alignment
information if it is not profitable.
We also have PackedPtrTraits. This is new PtrTraits and use it for various data structures such as Bag<>.
[1]: https://en.cppreference.com/w/cpp/types/is_trivial
* WTF.xcodeproj/project.pbxproj:
* wtf/Bag.h:
(WTF::Bag::clear):
(WTF::Bag::iterator::operator++):
* wtf/CMakeLists.txt:
* wtf/DumbPtrTraits.h:
* wtf/DumbValueTraits.h:
* wtf/MathExtras.h:
(WTF::clzConstexpr):
(WTF::clz):
(WTF::ctzConstexpr):
(WTF::ctz):
(WTF::getLSBSetConstexpr):
(WTF::getMSBSetConstexpr):
* wtf/Packed.h: Added.
(WTF::Packed::Packed):
(WTF::Packed::get const):
(WTF::Packed::set):
(WTF::Packed::operator=):
(WTF::Packed::exchange):
(WTF::Packed::swap):
(WTF::alignof):
(WTF::PackedPtrTraits::exchange):
(WTF::PackedPtrTraits::swap):
(WTF::PackedPtrTraits::unwrap):
* wtf/Platform.h:
* wtf/SentinelLinkedList.h:
(WTF::BasicRawSentinelNode::BasicRawSentinelNode):
(WTF::BasicRawSentinelNode::prev):
(WTF::BasicRawSentinelNode::next):
(WTF::PtrTraits>::remove):
(WTF::PtrTraits>::prepend):
(WTF::PtrTraits>::append):
(WTF::RawNode>::SentinelLinkedList):
(WTF::RawNode>::remove):
(WTF::BasicRawSentinelNode<T>::remove): Deleted.
(WTF::BasicRawSentinelNode<T>::prepend): Deleted.
(WTF::BasicRawSentinelNode<T>::append): Deleted.
* wtf/StdLibExtras.h:
(WTF::roundUpToMultipleOfImpl):
(WTF::roundUpToMultipleOfImpl0): Deleted.
* wtf/UnalignedAccess.h:
(WTF::unalignedLoad):
(WTF::unalignedStore):
Tools:
* TestWebKitAPI/CMakeLists.txt:
* TestWebKitAPI/TestWebKitAPI.xcodeproj/project.pbxproj:
* TestWebKitAPI/Tests/WTF/MathExtras.cpp:
(TestWebKitAPI::TEST):
* TestWebKitAPI/Tests/WTF/Packed.cpp: Added.
(TestWebKitAPI::TEST):
Canonical link: https://commits.webkit.org/211952@main
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@245214 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2019-05-12 22:50:21 +00:00
|
|
|
public:
|
2020-01-21 22:05:43 +00:00
|
|
|
static_assert(hasOneBitSet(passedAlignment), "Alignment needs to be power-of-two");
|
|
|
|
static constexpr size_t alignment = passedAlignment;
|
[JSC] Compress Watchpoint size by using enum type and Packed<> data structure
https://bugs.webkit.org/show_bug.cgi?id=197730
Reviewed by Filip Pizlo.
Source/JavaScriptCore:
Watchpoint takes 5~ MB memory in Gmail (total memory starts with 400 - 500 MB), so 1~%. Since it is allocated massively,
reducing each size of Watchpoint reduces memory footprint significantly.
As a first step, this patch uses Packed<> and enum to reduce the size of Watchpoint.
1. Watchpoint should have enum type and should not use vtable. vtable takes one pointer, and it is too costly for such a
memory sensitive objects. We perform downcast and dispatch the method of the derived classes based on this enum. Since
the # of derived Watchpoint classes are limited (Only 8), we can list up them easily. One unfortunate thing is that
we cannot do this for destructor so long as we use "delete" for deleting objects. If we dispatch the destructor of derived
class in the destructor of the base class, we call the destructor of the base class multiple times. delete operator override
does not help since custom delete operator is called after the destructor is called. While we can fix this issue by always
using custom deleter, currently we do not since all the watchpoints do not have members which have non trivial destructor.
Once it is strongly required, we can start using custom deleter, but for now, we do not need to do this.
2. We use Packed<> to compact pointers in Watchpoint. Since Watchpoint is a node of doubly linked list, each one has two
pointers for prev and next. This is also too costly. PackedPtr reduces the size and makes alignment 1.S
3. We use PackedCellPtr<> for JSCells in Watchpoint. This leverages alignment information and makes pointers smaller in
Darwin ARM64. One important thing to note here is that since this pointer is packed, it cannot be found by conservative
GC scan. It is OK for watchpoint since they are allocated in the heap anyway.
We applied this change to Watchpoint and get the following memory reduction. The highlight is that CodeBlockJettisoningWatchpoint in
ARM64 only takes 2 pointers size.
ORIGINAL X86_64 ARM64
WatchpointSet: 40 32 28
CodeBlockJettisoningWatchpoint: 32 19 15
StructureStubClearingWatchpoint: 56 48 40
AdaptiveInferredPropertyValueWatchpointBase::StructureWatchpoint: 24 13 11
AdaptiveInferredPropertyValueWatchpointBase::PropertyWatchpoint: 24 13 11
FunctionRareData::AllocationProfileClearingWatchpoint: 32 19 15
ObjectToStringAdaptiveStructureWatchpoint: 56 48 40
LLIntPrototypeLoadAdaptiveStructureWatchpoint: 64 48 48
DFG::AdaptiveStructureWatchpoint: 56 48 40
While we will re-architect the mechanism of Watchpoint, anyway Packed<> mechanism and enum types will be used too.
* CMakeLists.txt:
* JavaScriptCore.xcodeproj/project.pbxproj:
* Sources.txt:
* bytecode/AdaptiveInferredPropertyValueWatchpointBase.h:
* bytecode/CodeBlockJettisoningWatchpoint.h:
* bytecode/CodeOrigin.h:
* bytecode/LLIntPrototypeLoadAdaptiveStructureWatchpoint.cpp:
(JSC::LLIntPrototypeLoadAdaptiveStructureWatchpoint::LLIntPrototypeLoadAdaptiveStructureWatchpoint):
(JSC::LLIntPrototypeLoadAdaptiveStructureWatchpoint::fireInternal):
* bytecode/LLIntPrototypeLoadAdaptiveStructureWatchpoint.h:
* bytecode/StructureStubClearingWatchpoint.cpp:
(JSC::StructureStubClearingWatchpoint::fireInternal):
* bytecode/StructureStubClearingWatchpoint.h:
* bytecode/Watchpoint.cpp:
(JSC::Watchpoint::fire):
* bytecode/Watchpoint.h:
(JSC::Watchpoint::Watchpoint):
* dfg/DFGAdaptiveStructureWatchpoint.cpp:
(JSC::DFG::AdaptiveStructureWatchpoint::AdaptiveStructureWatchpoint):
* dfg/DFGAdaptiveStructureWatchpoint.h:
* heap/PackedCellPtr.h: Added.
* runtime/FunctionRareData.h:
* runtime/ObjectToStringAdaptiveStructureWatchpoint.cpp: Added.
(JSC::ObjectToStringAdaptiveStructureWatchpoint::ObjectToStringAdaptiveStructureWatchpoint):
(JSC::ObjectToStringAdaptiveStructureWatchpoint::install):
(JSC::ObjectToStringAdaptiveStructureWatchpoint::fireInternal):
* runtime/ObjectToStringAdaptiveStructureWatchpoint.h: Added.
* runtime/StructureRareData.cpp:
(JSC::StructureRareData::clearObjectToStringValue):
(JSC::ObjectToStringAdaptiveStructureWatchpoint::ObjectToStringAdaptiveStructureWatchpoint): Deleted.
(JSC::ObjectToStringAdaptiveStructureWatchpoint::install): Deleted.
(JSC::ObjectToStringAdaptiveStructureWatchpoint::fireInternal): Deleted.
* runtime/StructureRareData.h:
Source/WTF:
This patch introduces a new data structures, WTF::Packed, WTF::PackedPtr, and WTF::PackedAlignedPtr.
- WTF::Packed
WTF::Packed is data storage. We can read and write trivial (in C++ term [1]) data to this storage. The difference to
the usual storage is that the alignment of this storage is always 1. We access the underlying data by using unalignedLoad/unalignedStore.
This class offers alignment = 1 data structure instead of missing the following characteristics.
1. Load / Store are non atomic even if the data size is within a pointer width. We should not use this for a member which can be accessed
in a racy way. (e.g. fields accessed optimistically from the concurrent compilers).
2. We cannot take reference / pointer to the underlying storage since they are unaligned.
3. Access to this storage is unaligned access. The code is using memcpy, and the compiler will convert to an appropriate unaligned access
in certain architectures (x86_64 / ARM64). It could be slow. So use it for non performance sensitive & memory sensitive places.
- WTF::PackedPtr
WTF::PackedPtr is a specialization of WTF::Packed<T*>. And it is basically WTF::PackedAlignedPtr with alignment = 1. We further compact
the pointer by leveraging the platform specific knowledge. In 64bit architectures, the effective width of pointers are less than 64 bit.
In x86_64, it is 48 bits. And Darwin ARM64 is further smaller, 36 bits. This information allows us to compact the pointer to 6 bytes in
x86_64 and 5 bytes in Darwin ARM64.
- WTF::PackedAlignedPtr
WTF::PackedAlignedPtr is the WTF::PackedPtr with alignment information of the T. If we use this alignment information, we could reduce the
size of packed pointer further in some cases. For example, since we guarantee that JSCells are 16 byte aligned, low 4 bits are empty. Leveraging
this information in Darwin ARM64 platform allows us to make packed JSCell pointer 4 bytes (36 - 4 bits). We do not use passed alignment
information if it is not profitable.
We also have PackedPtrTraits. This is new PtrTraits and use it for various data structures such as Bag<>.
[1]: https://en.cppreference.com/w/cpp/types/is_trivial
* WTF.xcodeproj/project.pbxproj:
* wtf/Bag.h:
(WTF::Bag::clear):
(WTF::Bag::iterator::operator++):
* wtf/CMakeLists.txt:
* wtf/DumbPtrTraits.h:
* wtf/DumbValueTraits.h:
* wtf/MathExtras.h:
(WTF::clzConstexpr):
(WTF::clz):
(WTF::ctzConstexpr):
(WTF::ctz):
(WTF::getLSBSetConstexpr):
(WTF::getMSBSetConstexpr):
* wtf/Packed.h: Added.
(WTF::Packed::Packed):
(WTF::Packed::get const):
(WTF::Packed::set):
(WTF::Packed::operator=):
(WTF::Packed::exchange):
(WTF::Packed::swap):
(WTF::alignof):
(WTF::PackedPtrTraits::exchange):
(WTF::PackedPtrTraits::swap):
(WTF::PackedPtrTraits::unwrap):
* wtf/Platform.h:
* wtf/SentinelLinkedList.h:
(WTF::BasicRawSentinelNode::BasicRawSentinelNode):
(WTF::BasicRawSentinelNode::prev):
(WTF::BasicRawSentinelNode::next):
(WTF::PtrTraits>::remove):
(WTF::PtrTraits>::prepend):
(WTF::PtrTraits>::append):
(WTF::RawNode>::SentinelLinkedList):
(WTF::RawNode>::remove):
(WTF::BasicRawSentinelNode<T>::remove): Deleted.
(WTF::BasicRawSentinelNode<T>::prepend): Deleted.
(WTF::BasicRawSentinelNode<T>::append): Deleted.
* wtf/StdLibExtras.h:
(WTF::roundUpToMultipleOfImpl):
(WTF::roundUpToMultipleOfImpl0): Deleted.
* wtf/UnalignedAccess.h:
(WTF::unalignedLoad):
(WTF::unalignedStore):
Tools:
* TestWebKitAPI/CMakeLists.txt:
* TestWebKitAPI/TestWebKitAPI.xcodeproj/project.pbxproj:
* TestWebKitAPI/Tests/WTF/MathExtras.cpp:
(TestWebKitAPI::TEST):
* TestWebKitAPI/Tests/WTF/Packed.cpp: Added.
(TestWebKitAPI::TEST):
Canonical link: https://commits.webkit.org/211952@main
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@245214 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2019-05-12 22:50:21 +00:00
|
|
|
static constexpr bool isPackedType = true;
|
|
|
|
static constexpr unsigned alignmentShiftSizeIfProfitable = getLSBSetConstexpr(alignment);
|
2020-01-21 04:01:50 +00:00
|
|
|
static constexpr unsigned storageSizeWithoutAlignmentShift = roundUpToMultipleOf<8>(OS_CONSTANT(EFFECTIVE_ADDRESS_WIDTH)) / 8;
|
|
|
|
static constexpr unsigned storageSizeWithAlignmentShift = roundUpToMultipleOf<8>(OS_CONSTANT(EFFECTIVE_ADDRESS_WIDTH) - alignmentShiftSizeIfProfitable) / 8;
|
[JSC] Compress Watchpoint size by using enum type and Packed<> data structure
https://bugs.webkit.org/show_bug.cgi?id=197730
Reviewed by Filip Pizlo.
Source/JavaScriptCore:
Watchpoint takes 5~ MB memory in Gmail (total memory starts with 400 - 500 MB), so 1~%. Since it is allocated massively,
reducing each size of Watchpoint reduces memory footprint significantly.
As a first step, this patch uses Packed<> and enum to reduce the size of Watchpoint.
1. Watchpoint should have enum type and should not use vtable. vtable takes one pointer, and it is too costly for such a
memory sensitive objects. We perform downcast and dispatch the method of the derived classes based on this enum. Since
the # of derived Watchpoint classes are limited (Only 8), we can list up them easily. One unfortunate thing is that
we cannot do this for destructor so long as we use "delete" for deleting objects. If we dispatch the destructor of derived
class in the destructor of the base class, we call the destructor of the base class multiple times. delete operator override
does not help since custom delete operator is called after the destructor is called. While we can fix this issue by always
using custom deleter, currently we do not since all the watchpoints do not have members which have non trivial destructor.
Once it is strongly required, we can start using custom deleter, but for now, we do not need to do this.
2. We use Packed<> to compact pointers in Watchpoint. Since Watchpoint is a node of doubly linked list, each one has two
pointers for prev and next. This is also too costly. PackedPtr reduces the size and makes alignment 1.S
3. We use PackedCellPtr<> for JSCells in Watchpoint. This leverages alignment information and makes pointers smaller in
Darwin ARM64. One important thing to note here is that since this pointer is packed, it cannot be found by conservative
GC scan. It is OK for watchpoint since they are allocated in the heap anyway.
We applied this change to Watchpoint and get the following memory reduction. The highlight is that CodeBlockJettisoningWatchpoint in
ARM64 only takes 2 pointers size.
ORIGINAL X86_64 ARM64
WatchpointSet: 40 32 28
CodeBlockJettisoningWatchpoint: 32 19 15
StructureStubClearingWatchpoint: 56 48 40
AdaptiveInferredPropertyValueWatchpointBase::StructureWatchpoint: 24 13 11
AdaptiveInferredPropertyValueWatchpointBase::PropertyWatchpoint: 24 13 11
FunctionRareData::AllocationProfileClearingWatchpoint: 32 19 15
ObjectToStringAdaptiveStructureWatchpoint: 56 48 40
LLIntPrototypeLoadAdaptiveStructureWatchpoint: 64 48 48
DFG::AdaptiveStructureWatchpoint: 56 48 40
While we will re-architect the mechanism of Watchpoint, anyway Packed<> mechanism and enum types will be used too.
* CMakeLists.txt:
* JavaScriptCore.xcodeproj/project.pbxproj:
* Sources.txt:
* bytecode/AdaptiveInferredPropertyValueWatchpointBase.h:
* bytecode/CodeBlockJettisoningWatchpoint.h:
* bytecode/CodeOrigin.h:
* bytecode/LLIntPrototypeLoadAdaptiveStructureWatchpoint.cpp:
(JSC::LLIntPrototypeLoadAdaptiveStructureWatchpoint::LLIntPrototypeLoadAdaptiveStructureWatchpoint):
(JSC::LLIntPrototypeLoadAdaptiveStructureWatchpoint::fireInternal):
* bytecode/LLIntPrototypeLoadAdaptiveStructureWatchpoint.h:
* bytecode/StructureStubClearingWatchpoint.cpp:
(JSC::StructureStubClearingWatchpoint::fireInternal):
* bytecode/StructureStubClearingWatchpoint.h:
* bytecode/Watchpoint.cpp:
(JSC::Watchpoint::fire):
* bytecode/Watchpoint.h:
(JSC::Watchpoint::Watchpoint):
* dfg/DFGAdaptiveStructureWatchpoint.cpp:
(JSC::DFG::AdaptiveStructureWatchpoint::AdaptiveStructureWatchpoint):
* dfg/DFGAdaptiveStructureWatchpoint.h:
* heap/PackedCellPtr.h: Added.
* runtime/FunctionRareData.h:
* runtime/ObjectToStringAdaptiveStructureWatchpoint.cpp: Added.
(JSC::ObjectToStringAdaptiveStructureWatchpoint::ObjectToStringAdaptiveStructureWatchpoint):
(JSC::ObjectToStringAdaptiveStructureWatchpoint::install):
(JSC::ObjectToStringAdaptiveStructureWatchpoint::fireInternal):
* runtime/ObjectToStringAdaptiveStructureWatchpoint.h: Added.
* runtime/StructureRareData.cpp:
(JSC::StructureRareData::clearObjectToStringValue):
(JSC::ObjectToStringAdaptiveStructureWatchpoint::ObjectToStringAdaptiveStructureWatchpoint): Deleted.
(JSC::ObjectToStringAdaptiveStructureWatchpoint::install): Deleted.
(JSC::ObjectToStringAdaptiveStructureWatchpoint::fireInternal): Deleted.
* runtime/StructureRareData.h:
Source/WTF:
This patch introduces a new data structures, WTF::Packed, WTF::PackedPtr, and WTF::PackedAlignedPtr.
- WTF::Packed
WTF::Packed is data storage. We can read and write trivial (in C++ term [1]) data to this storage. The difference to
the usual storage is that the alignment of this storage is always 1. We access the underlying data by using unalignedLoad/unalignedStore.
This class offers alignment = 1 data structure instead of missing the following characteristics.
1. Load / Store are non atomic even if the data size is within a pointer width. We should not use this for a member which can be accessed
in a racy way. (e.g. fields accessed optimistically from the concurrent compilers).
2. We cannot take reference / pointer to the underlying storage since they are unaligned.
3. Access to this storage is unaligned access. The code is using memcpy, and the compiler will convert to an appropriate unaligned access
in certain architectures (x86_64 / ARM64). It could be slow. So use it for non performance sensitive & memory sensitive places.
- WTF::PackedPtr
WTF::PackedPtr is a specialization of WTF::Packed<T*>. And it is basically WTF::PackedAlignedPtr with alignment = 1. We further compact
the pointer by leveraging the platform specific knowledge. In 64bit architectures, the effective width of pointers are less than 64 bit.
In x86_64, it is 48 bits. And Darwin ARM64 is further smaller, 36 bits. This information allows us to compact the pointer to 6 bytes in
x86_64 and 5 bytes in Darwin ARM64.
- WTF::PackedAlignedPtr
WTF::PackedAlignedPtr is the WTF::PackedPtr with alignment information of the T. If we use this alignment information, we could reduce the
size of packed pointer further in some cases. For example, since we guarantee that JSCells are 16 byte aligned, low 4 bits are empty. Leveraging
this information in Darwin ARM64 platform allows us to make packed JSCell pointer 4 bytes (36 - 4 bits). We do not use passed alignment
information if it is not profitable.
We also have PackedPtrTraits. This is new PtrTraits and use it for various data structures such as Bag<>.
[1]: https://en.cppreference.com/w/cpp/types/is_trivial
* WTF.xcodeproj/project.pbxproj:
* wtf/Bag.h:
(WTF::Bag::clear):
(WTF::Bag::iterator::operator++):
* wtf/CMakeLists.txt:
* wtf/DumbPtrTraits.h:
* wtf/DumbValueTraits.h:
* wtf/MathExtras.h:
(WTF::clzConstexpr):
(WTF::clz):
(WTF::ctzConstexpr):
(WTF::ctz):
(WTF::getLSBSetConstexpr):
(WTF::getMSBSetConstexpr):
* wtf/Packed.h: Added.
(WTF::Packed::Packed):
(WTF::Packed::get const):
(WTF::Packed::set):
(WTF::Packed::operator=):
(WTF::Packed::exchange):
(WTF::Packed::swap):
(WTF::alignof):
(WTF::PackedPtrTraits::exchange):
(WTF::PackedPtrTraits::swap):
(WTF::PackedPtrTraits::unwrap):
* wtf/Platform.h:
* wtf/SentinelLinkedList.h:
(WTF::BasicRawSentinelNode::BasicRawSentinelNode):
(WTF::BasicRawSentinelNode::prev):
(WTF::BasicRawSentinelNode::next):
(WTF::PtrTraits>::remove):
(WTF::PtrTraits>::prepend):
(WTF::PtrTraits>::append):
(WTF::RawNode>::SentinelLinkedList):
(WTF::RawNode>::remove):
(WTF::BasicRawSentinelNode<T>::remove): Deleted.
(WTF::BasicRawSentinelNode<T>::prepend): Deleted.
(WTF::BasicRawSentinelNode<T>::append): Deleted.
* wtf/StdLibExtras.h:
(WTF::roundUpToMultipleOfImpl):
(WTF::roundUpToMultipleOfImpl0): Deleted.
* wtf/UnalignedAccess.h:
(WTF::unalignedLoad):
(WTF::unalignedStore):
Tools:
* TestWebKitAPI/CMakeLists.txt:
* TestWebKitAPI/TestWebKitAPI.xcodeproj/project.pbxproj:
* TestWebKitAPI/Tests/WTF/MathExtras.cpp:
(TestWebKitAPI::TEST):
* TestWebKitAPI/Tests/WTF/Packed.cpp: Added.
(TestWebKitAPI::TEST):
Canonical link: https://commits.webkit.org/211952@main
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@245214 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2019-05-12 22:50:21 +00:00
|
|
|
static constexpr bool isAlignmentShiftProfitable = storageSizeWithoutAlignmentShift > storageSizeWithAlignmentShift;
|
|
|
|
static constexpr unsigned alignmentShiftSize = isAlignmentShiftProfitable ? alignmentShiftSizeIfProfitable : 0;
|
|
|
|
static constexpr unsigned storageSize = storageSizeWithAlignmentShift;
|
2021-06-15 16:47:42 +00:00
|
|
|
static_assert(storageSize <= sizeof(uintptr_t));
|
[JSC] Compress Watchpoint size by using enum type and Packed<> data structure
https://bugs.webkit.org/show_bug.cgi?id=197730
Reviewed by Filip Pizlo.
Source/JavaScriptCore:
Watchpoint takes 5~ MB memory in Gmail (total memory starts with 400 - 500 MB), so 1~%. Since it is allocated massively,
reducing each size of Watchpoint reduces memory footprint significantly.
As a first step, this patch uses Packed<> and enum to reduce the size of Watchpoint.
1. Watchpoint should have enum type and should not use vtable. vtable takes one pointer, and it is too costly for such a
memory sensitive objects. We perform downcast and dispatch the method of the derived classes based on this enum. Since
the # of derived Watchpoint classes are limited (Only 8), we can list up them easily. One unfortunate thing is that
we cannot do this for destructor so long as we use "delete" for deleting objects. If we dispatch the destructor of derived
class in the destructor of the base class, we call the destructor of the base class multiple times. delete operator override
does not help since custom delete operator is called after the destructor is called. While we can fix this issue by always
using custom deleter, currently we do not since all the watchpoints do not have members which have non trivial destructor.
Once it is strongly required, we can start using custom deleter, but for now, we do not need to do this.
2. We use Packed<> to compact pointers in Watchpoint. Since Watchpoint is a node of doubly linked list, each one has two
pointers for prev and next. This is also too costly. PackedPtr reduces the size and makes alignment 1.S
3. We use PackedCellPtr<> for JSCells in Watchpoint. This leverages alignment information and makes pointers smaller in
Darwin ARM64. One important thing to note here is that since this pointer is packed, it cannot be found by conservative
GC scan. It is OK for watchpoint since they are allocated in the heap anyway.
We applied this change to Watchpoint and get the following memory reduction. The highlight is that CodeBlockJettisoningWatchpoint in
ARM64 only takes 2 pointers size.
ORIGINAL X86_64 ARM64
WatchpointSet: 40 32 28
CodeBlockJettisoningWatchpoint: 32 19 15
StructureStubClearingWatchpoint: 56 48 40
AdaptiveInferredPropertyValueWatchpointBase::StructureWatchpoint: 24 13 11
AdaptiveInferredPropertyValueWatchpointBase::PropertyWatchpoint: 24 13 11
FunctionRareData::AllocationProfileClearingWatchpoint: 32 19 15
ObjectToStringAdaptiveStructureWatchpoint: 56 48 40
LLIntPrototypeLoadAdaptiveStructureWatchpoint: 64 48 48
DFG::AdaptiveStructureWatchpoint: 56 48 40
While we will re-architect the mechanism of Watchpoint, anyway Packed<> mechanism and enum types will be used too.
* CMakeLists.txt:
* JavaScriptCore.xcodeproj/project.pbxproj:
* Sources.txt:
* bytecode/AdaptiveInferredPropertyValueWatchpointBase.h:
* bytecode/CodeBlockJettisoningWatchpoint.h:
* bytecode/CodeOrigin.h:
* bytecode/LLIntPrototypeLoadAdaptiveStructureWatchpoint.cpp:
(JSC::LLIntPrototypeLoadAdaptiveStructureWatchpoint::LLIntPrototypeLoadAdaptiveStructureWatchpoint):
(JSC::LLIntPrototypeLoadAdaptiveStructureWatchpoint::fireInternal):
* bytecode/LLIntPrototypeLoadAdaptiveStructureWatchpoint.h:
* bytecode/StructureStubClearingWatchpoint.cpp:
(JSC::StructureStubClearingWatchpoint::fireInternal):
* bytecode/StructureStubClearingWatchpoint.h:
* bytecode/Watchpoint.cpp:
(JSC::Watchpoint::fire):
* bytecode/Watchpoint.h:
(JSC::Watchpoint::Watchpoint):
* dfg/DFGAdaptiveStructureWatchpoint.cpp:
(JSC::DFG::AdaptiveStructureWatchpoint::AdaptiveStructureWatchpoint):
* dfg/DFGAdaptiveStructureWatchpoint.h:
* heap/PackedCellPtr.h: Added.
* runtime/FunctionRareData.h:
* runtime/ObjectToStringAdaptiveStructureWatchpoint.cpp: Added.
(JSC::ObjectToStringAdaptiveStructureWatchpoint::ObjectToStringAdaptiveStructureWatchpoint):
(JSC::ObjectToStringAdaptiveStructureWatchpoint::install):
(JSC::ObjectToStringAdaptiveStructureWatchpoint::fireInternal):
* runtime/ObjectToStringAdaptiveStructureWatchpoint.h: Added.
* runtime/StructureRareData.cpp:
(JSC::StructureRareData::clearObjectToStringValue):
(JSC::ObjectToStringAdaptiveStructureWatchpoint::ObjectToStringAdaptiveStructureWatchpoint): Deleted.
(JSC::ObjectToStringAdaptiveStructureWatchpoint::install): Deleted.
(JSC::ObjectToStringAdaptiveStructureWatchpoint::fireInternal): Deleted.
* runtime/StructureRareData.h:
Source/WTF:
This patch introduces a new data structures, WTF::Packed, WTF::PackedPtr, and WTF::PackedAlignedPtr.
- WTF::Packed
WTF::Packed is data storage. We can read and write trivial (in C++ term [1]) data to this storage. The difference to
the usual storage is that the alignment of this storage is always 1. We access the underlying data by using unalignedLoad/unalignedStore.
This class offers alignment = 1 data structure instead of missing the following characteristics.
1. Load / Store are non atomic even if the data size is within a pointer width. We should not use this for a member which can be accessed
in a racy way. (e.g. fields accessed optimistically from the concurrent compilers).
2. We cannot take reference / pointer to the underlying storage since they are unaligned.
3. Access to this storage is unaligned access. The code is using memcpy, and the compiler will convert to an appropriate unaligned access
in certain architectures (x86_64 / ARM64). It could be slow. So use it for non performance sensitive & memory sensitive places.
- WTF::PackedPtr
WTF::PackedPtr is a specialization of WTF::Packed<T*>. And it is basically WTF::PackedAlignedPtr with alignment = 1. We further compact
the pointer by leveraging the platform specific knowledge. In 64bit architectures, the effective width of pointers are less than 64 bit.
In x86_64, it is 48 bits. And Darwin ARM64 is further smaller, 36 bits. This information allows us to compact the pointer to 6 bytes in
x86_64 and 5 bytes in Darwin ARM64.
- WTF::PackedAlignedPtr
WTF::PackedAlignedPtr is the WTF::PackedPtr with alignment information of the T. If we use this alignment information, we could reduce the
size of packed pointer further in some cases. For example, since we guarantee that JSCells are 16 byte aligned, low 4 bits are empty. Leveraging
this information in Darwin ARM64 platform allows us to make packed JSCell pointer 4 bytes (36 - 4 bits). We do not use passed alignment
information if it is not profitable.
We also have PackedPtrTraits. This is new PtrTraits and use it for various data structures such as Bag<>.
[1]: https://en.cppreference.com/w/cpp/types/is_trivial
* WTF.xcodeproj/project.pbxproj:
* wtf/Bag.h:
(WTF::Bag::clear):
(WTF::Bag::iterator::operator++):
* wtf/CMakeLists.txt:
* wtf/DumbPtrTraits.h:
* wtf/DumbValueTraits.h:
* wtf/MathExtras.h:
(WTF::clzConstexpr):
(WTF::clz):
(WTF::ctzConstexpr):
(WTF::ctz):
(WTF::getLSBSetConstexpr):
(WTF::getMSBSetConstexpr):
* wtf/Packed.h: Added.
(WTF::Packed::Packed):
(WTF::Packed::get const):
(WTF::Packed::set):
(WTF::Packed::operator=):
(WTF::Packed::exchange):
(WTF::Packed::swap):
(WTF::alignof):
(WTF::PackedPtrTraits::exchange):
(WTF::PackedPtrTraits::swap):
(WTF::PackedPtrTraits::unwrap):
* wtf/Platform.h:
* wtf/SentinelLinkedList.h:
(WTF::BasicRawSentinelNode::BasicRawSentinelNode):
(WTF::BasicRawSentinelNode::prev):
(WTF::BasicRawSentinelNode::next):
(WTF::PtrTraits>::remove):
(WTF::PtrTraits>::prepend):
(WTF::PtrTraits>::append):
(WTF::RawNode>::SentinelLinkedList):
(WTF::RawNode>::remove):
(WTF::BasicRawSentinelNode<T>::remove): Deleted.
(WTF::BasicRawSentinelNode<T>::prepend): Deleted.
(WTF::BasicRawSentinelNode<T>::append): Deleted.
* wtf/StdLibExtras.h:
(WTF::roundUpToMultipleOfImpl):
(WTF::roundUpToMultipleOfImpl0): Deleted.
* wtf/UnalignedAccess.h:
(WTF::unalignedLoad):
(WTF::unalignedStore):
Tools:
* TestWebKitAPI/CMakeLists.txt:
* TestWebKitAPI/TestWebKitAPI.xcodeproj/project.pbxproj:
* TestWebKitAPI/Tests/WTF/MathExtras.cpp:
(TestWebKitAPI::TEST):
* TestWebKitAPI/Tests/WTF/Packed.cpp: Added.
(TestWebKitAPI::TEST):
Canonical link: https://commits.webkit.org/211952@main
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@245214 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2019-05-12 22:50:21 +00:00
|
|
|
|
|
|
|
constexpr PackedAlignedPtr()
|
|
|
|
: m_storage()
|
|
|
|
{
|
|
|
|
}
|
|
|
|
|
|
|
|
constexpr PackedAlignedPtr(std::nullptr_t)
|
|
|
|
: m_storage()
|
|
|
|
{
|
|
|
|
}
|
|
|
|
|
|
|
|
PackedAlignedPtr(T* value)
|
|
|
|
{
|
|
|
|
set(value);
|
|
|
|
}
|
|
|
|
|
|
|
|
T* get() const
|
|
|
|
{
|
|
|
|
// FIXME: PackedPtr<> can load memory with one mov by checking page boundary.
|
|
|
|
// https://bugs.webkit.org/show_bug.cgi?id=197754
|
|
|
|
uintptr_t value = 0;
|
2021-06-15 16:47:42 +00:00
|
|
|
|
[JSC] Compress Watchpoint size by using enum type and Packed<> data structure
https://bugs.webkit.org/show_bug.cgi?id=197730
Reviewed by Filip Pizlo.
Source/JavaScriptCore:
Watchpoint takes 5~ MB memory in Gmail (total memory starts with 400 - 500 MB), so 1~%. Since it is allocated massively,
reducing each size of Watchpoint reduces memory footprint significantly.
As a first step, this patch uses Packed<> and enum to reduce the size of Watchpoint.
1. Watchpoint should have enum type and should not use vtable. vtable takes one pointer, and it is too costly for such a
memory sensitive objects. We perform downcast and dispatch the method of the derived classes based on this enum. Since
the # of derived Watchpoint classes are limited (Only 8), we can list up them easily. One unfortunate thing is that
we cannot do this for destructor so long as we use "delete" for deleting objects. If we dispatch the destructor of derived
class in the destructor of the base class, we call the destructor of the base class multiple times. delete operator override
does not help since custom delete operator is called after the destructor is called. While we can fix this issue by always
using custom deleter, currently we do not since all the watchpoints do not have members which have non trivial destructor.
Once it is strongly required, we can start using custom deleter, but for now, we do not need to do this.
2. We use Packed<> to compact pointers in Watchpoint. Since Watchpoint is a node of doubly linked list, each one has two
pointers for prev and next. This is also too costly. PackedPtr reduces the size and makes alignment 1.S
3. We use PackedCellPtr<> for JSCells in Watchpoint. This leverages alignment information and makes pointers smaller in
Darwin ARM64. One important thing to note here is that since this pointer is packed, it cannot be found by conservative
GC scan. It is OK for watchpoint since they are allocated in the heap anyway.
We applied this change to Watchpoint and get the following memory reduction. The highlight is that CodeBlockJettisoningWatchpoint in
ARM64 only takes 2 pointers size.
ORIGINAL X86_64 ARM64
WatchpointSet: 40 32 28
CodeBlockJettisoningWatchpoint: 32 19 15
StructureStubClearingWatchpoint: 56 48 40
AdaptiveInferredPropertyValueWatchpointBase::StructureWatchpoint: 24 13 11
AdaptiveInferredPropertyValueWatchpointBase::PropertyWatchpoint: 24 13 11
FunctionRareData::AllocationProfileClearingWatchpoint: 32 19 15
ObjectToStringAdaptiveStructureWatchpoint: 56 48 40
LLIntPrototypeLoadAdaptiveStructureWatchpoint: 64 48 48
DFG::AdaptiveStructureWatchpoint: 56 48 40
While we will re-architect the mechanism of Watchpoint, anyway Packed<> mechanism and enum types will be used too.
* CMakeLists.txt:
* JavaScriptCore.xcodeproj/project.pbxproj:
* Sources.txt:
* bytecode/AdaptiveInferredPropertyValueWatchpointBase.h:
* bytecode/CodeBlockJettisoningWatchpoint.h:
* bytecode/CodeOrigin.h:
* bytecode/LLIntPrototypeLoadAdaptiveStructureWatchpoint.cpp:
(JSC::LLIntPrototypeLoadAdaptiveStructureWatchpoint::LLIntPrototypeLoadAdaptiveStructureWatchpoint):
(JSC::LLIntPrototypeLoadAdaptiveStructureWatchpoint::fireInternal):
* bytecode/LLIntPrototypeLoadAdaptiveStructureWatchpoint.h:
* bytecode/StructureStubClearingWatchpoint.cpp:
(JSC::StructureStubClearingWatchpoint::fireInternal):
* bytecode/StructureStubClearingWatchpoint.h:
* bytecode/Watchpoint.cpp:
(JSC::Watchpoint::fire):
* bytecode/Watchpoint.h:
(JSC::Watchpoint::Watchpoint):
* dfg/DFGAdaptiveStructureWatchpoint.cpp:
(JSC::DFG::AdaptiveStructureWatchpoint::AdaptiveStructureWatchpoint):
* dfg/DFGAdaptiveStructureWatchpoint.h:
* heap/PackedCellPtr.h: Added.
* runtime/FunctionRareData.h:
* runtime/ObjectToStringAdaptiveStructureWatchpoint.cpp: Added.
(JSC::ObjectToStringAdaptiveStructureWatchpoint::ObjectToStringAdaptiveStructureWatchpoint):
(JSC::ObjectToStringAdaptiveStructureWatchpoint::install):
(JSC::ObjectToStringAdaptiveStructureWatchpoint::fireInternal):
* runtime/ObjectToStringAdaptiveStructureWatchpoint.h: Added.
* runtime/StructureRareData.cpp:
(JSC::StructureRareData::clearObjectToStringValue):
(JSC::ObjectToStringAdaptiveStructureWatchpoint::ObjectToStringAdaptiveStructureWatchpoint): Deleted.
(JSC::ObjectToStringAdaptiveStructureWatchpoint::install): Deleted.
(JSC::ObjectToStringAdaptiveStructureWatchpoint::fireInternal): Deleted.
* runtime/StructureRareData.h:
Source/WTF:
This patch introduces a new data structures, WTF::Packed, WTF::PackedPtr, and WTF::PackedAlignedPtr.
- WTF::Packed
WTF::Packed is data storage. We can read and write trivial (in C++ term [1]) data to this storage. The difference to
the usual storage is that the alignment of this storage is always 1. We access the underlying data by using unalignedLoad/unalignedStore.
This class offers alignment = 1 data structure instead of missing the following characteristics.
1. Load / Store are non atomic even if the data size is within a pointer width. We should not use this for a member which can be accessed
in a racy way. (e.g. fields accessed optimistically from the concurrent compilers).
2. We cannot take reference / pointer to the underlying storage since they are unaligned.
3. Access to this storage is unaligned access. The code is using memcpy, and the compiler will convert to an appropriate unaligned access
in certain architectures (x86_64 / ARM64). It could be slow. So use it for non performance sensitive & memory sensitive places.
- WTF::PackedPtr
WTF::PackedPtr is a specialization of WTF::Packed<T*>. And it is basically WTF::PackedAlignedPtr with alignment = 1. We further compact
the pointer by leveraging the platform specific knowledge. In 64bit architectures, the effective width of pointers are less than 64 bit.
In x86_64, it is 48 bits. And Darwin ARM64 is further smaller, 36 bits. This information allows us to compact the pointer to 6 bytes in
x86_64 and 5 bytes in Darwin ARM64.
- WTF::PackedAlignedPtr
WTF::PackedAlignedPtr is the WTF::PackedPtr with alignment information of the T. If we use this alignment information, we could reduce the
size of packed pointer further in some cases. For example, since we guarantee that JSCells are 16 byte aligned, low 4 bits are empty. Leveraging
this information in Darwin ARM64 platform allows us to make packed JSCell pointer 4 bytes (36 - 4 bits). We do not use passed alignment
information if it is not profitable.
We also have PackedPtrTraits. This is new PtrTraits and use it for various data structures such as Bag<>.
[1]: https://en.cppreference.com/w/cpp/types/is_trivial
* WTF.xcodeproj/project.pbxproj:
* wtf/Bag.h:
(WTF::Bag::clear):
(WTF::Bag::iterator::operator++):
* wtf/CMakeLists.txt:
* wtf/DumbPtrTraits.h:
* wtf/DumbValueTraits.h:
* wtf/MathExtras.h:
(WTF::clzConstexpr):
(WTF::clz):
(WTF::ctzConstexpr):
(WTF::ctz):
(WTF::getLSBSetConstexpr):
(WTF::getMSBSetConstexpr):
* wtf/Packed.h: Added.
(WTF::Packed::Packed):
(WTF::Packed::get const):
(WTF::Packed::set):
(WTF::Packed::operator=):
(WTF::Packed::exchange):
(WTF::Packed::swap):
(WTF::alignof):
(WTF::PackedPtrTraits::exchange):
(WTF::PackedPtrTraits::swap):
(WTF::PackedPtrTraits::unwrap):
* wtf/Platform.h:
* wtf/SentinelLinkedList.h:
(WTF::BasicRawSentinelNode::BasicRawSentinelNode):
(WTF::BasicRawSentinelNode::prev):
(WTF::BasicRawSentinelNode::next):
(WTF::PtrTraits>::remove):
(WTF::PtrTraits>::prepend):
(WTF::PtrTraits>::append):
(WTF::RawNode>::SentinelLinkedList):
(WTF::RawNode>::remove):
(WTF::BasicRawSentinelNode<T>::remove): Deleted.
(WTF::BasicRawSentinelNode<T>::prepend): Deleted.
(WTF::BasicRawSentinelNode<T>::append): Deleted.
* wtf/StdLibExtras.h:
(WTF::roundUpToMultipleOfImpl):
(WTF::roundUpToMultipleOfImpl0): Deleted.
* wtf/UnalignedAccess.h:
(WTF::unalignedLoad):
(WTF::unalignedStore):
Tools:
* TestWebKitAPI/CMakeLists.txt:
* TestWebKitAPI/TestWebKitAPI.xcodeproj/project.pbxproj:
* TestWebKitAPI/Tests/WTF/MathExtras.cpp:
(TestWebKitAPI::TEST):
* TestWebKitAPI/Tests/WTF/Packed.cpp: Added.
(TestWebKitAPI::TEST):
Canonical link: https://commits.webkit.org/211952@main
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@245214 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2019-05-12 22:50:21 +00:00
|
|
|
#if CPU(LITTLE_ENDIAN)
|
|
|
|
memcpy(&value, m_storage.data(), storageSize);
|
|
|
|
#else
|
|
|
|
memcpy(bitwise_cast<uint8_t*>(&value) + (sizeof(void*) - storageSize), m_storage.data(), storageSize);
|
|
|
|
#endif
|
2021-06-15 16:47:42 +00:00
|
|
|
|
[JSC] Compress Watchpoint size by using enum type and Packed<> data structure
https://bugs.webkit.org/show_bug.cgi?id=197730
Reviewed by Filip Pizlo.
Source/JavaScriptCore:
Watchpoint takes 5~ MB memory in Gmail (total memory starts with 400 - 500 MB), so 1~%. Since it is allocated massively,
reducing each size of Watchpoint reduces memory footprint significantly.
As a first step, this patch uses Packed<> and enum to reduce the size of Watchpoint.
1. Watchpoint should have enum type and should not use vtable. vtable takes one pointer, and it is too costly for such a
memory sensitive objects. We perform downcast and dispatch the method of the derived classes based on this enum. Since
the # of derived Watchpoint classes are limited (Only 8), we can list up them easily. One unfortunate thing is that
we cannot do this for destructor so long as we use "delete" for deleting objects. If we dispatch the destructor of derived
class in the destructor of the base class, we call the destructor of the base class multiple times. delete operator override
does not help since custom delete operator is called after the destructor is called. While we can fix this issue by always
using custom deleter, currently we do not since all the watchpoints do not have members which have non trivial destructor.
Once it is strongly required, we can start using custom deleter, but for now, we do not need to do this.
2. We use Packed<> to compact pointers in Watchpoint. Since Watchpoint is a node of doubly linked list, each one has two
pointers for prev and next. This is also too costly. PackedPtr reduces the size and makes alignment 1.S
3. We use PackedCellPtr<> for JSCells in Watchpoint. This leverages alignment information and makes pointers smaller in
Darwin ARM64. One important thing to note here is that since this pointer is packed, it cannot be found by conservative
GC scan. It is OK for watchpoint since they are allocated in the heap anyway.
We applied this change to Watchpoint and get the following memory reduction. The highlight is that CodeBlockJettisoningWatchpoint in
ARM64 only takes 2 pointers size.
ORIGINAL X86_64 ARM64
WatchpointSet: 40 32 28
CodeBlockJettisoningWatchpoint: 32 19 15
StructureStubClearingWatchpoint: 56 48 40
AdaptiveInferredPropertyValueWatchpointBase::StructureWatchpoint: 24 13 11
AdaptiveInferredPropertyValueWatchpointBase::PropertyWatchpoint: 24 13 11
FunctionRareData::AllocationProfileClearingWatchpoint: 32 19 15
ObjectToStringAdaptiveStructureWatchpoint: 56 48 40
LLIntPrototypeLoadAdaptiveStructureWatchpoint: 64 48 48
DFG::AdaptiveStructureWatchpoint: 56 48 40
While we will re-architect the mechanism of Watchpoint, anyway Packed<> mechanism and enum types will be used too.
* CMakeLists.txt:
* JavaScriptCore.xcodeproj/project.pbxproj:
* Sources.txt:
* bytecode/AdaptiveInferredPropertyValueWatchpointBase.h:
* bytecode/CodeBlockJettisoningWatchpoint.h:
* bytecode/CodeOrigin.h:
* bytecode/LLIntPrototypeLoadAdaptiveStructureWatchpoint.cpp:
(JSC::LLIntPrototypeLoadAdaptiveStructureWatchpoint::LLIntPrototypeLoadAdaptiveStructureWatchpoint):
(JSC::LLIntPrototypeLoadAdaptiveStructureWatchpoint::fireInternal):
* bytecode/LLIntPrototypeLoadAdaptiveStructureWatchpoint.h:
* bytecode/StructureStubClearingWatchpoint.cpp:
(JSC::StructureStubClearingWatchpoint::fireInternal):
* bytecode/StructureStubClearingWatchpoint.h:
* bytecode/Watchpoint.cpp:
(JSC::Watchpoint::fire):
* bytecode/Watchpoint.h:
(JSC::Watchpoint::Watchpoint):
* dfg/DFGAdaptiveStructureWatchpoint.cpp:
(JSC::DFG::AdaptiveStructureWatchpoint::AdaptiveStructureWatchpoint):
* dfg/DFGAdaptiveStructureWatchpoint.h:
* heap/PackedCellPtr.h: Added.
* runtime/FunctionRareData.h:
* runtime/ObjectToStringAdaptiveStructureWatchpoint.cpp: Added.
(JSC::ObjectToStringAdaptiveStructureWatchpoint::ObjectToStringAdaptiveStructureWatchpoint):
(JSC::ObjectToStringAdaptiveStructureWatchpoint::install):
(JSC::ObjectToStringAdaptiveStructureWatchpoint::fireInternal):
* runtime/ObjectToStringAdaptiveStructureWatchpoint.h: Added.
* runtime/StructureRareData.cpp:
(JSC::StructureRareData::clearObjectToStringValue):
(JSC::ObjectToStringAdaptiveStructureWatchpoint::ObjectToStringAdaptiveStructureWatchpoint): Deleted.
(JSC::ObjectToStringAdaptiveStructureWatchpoint::install): Deleted.
(JSC::ObjectToStringAdaptiveStructureWatchpoint::fireInternal): Deleted.
* runtime/StructureRareData.h:
Source/WTF:
This patch introduces a new data structures, WTF::Packed, WTF::PackedPtr, and WTF::PackedAlignedPtr.
- WTF::Packed
WTF::Packed is data storage. We can read and write trivial (in C++ term [1]) data to this storage. The difference to
the usual storage is that the alignment of this storage is always 1. We access the underlying data by using unalignedLoad/unalignedStore.
This class offers alignment = 1 data structure instead of missing the following characteristics.
1. Load / Store are non atomic even if the data size is within a pointer width. We should not use this for a member which can be accessed
in a racy way. (e.g. fields accessed optimistically from the concurrent compilers).
2. We cannot take reference / pointer to the underlying storage since they are unaligned.
3. Access to this storage is unaligned access. The code is using memcpy, and the compiler will convert to an appropriate unaligned access
in certain architectures (x86_64 / ARM64). It could be slow. So use it for non performance sensitive & memory sensitive places.
- WTF::PackedPtr
WTF::PackedPtr is a specialization of WTF::Packed<T*>. And it is basically WTF::PackedAlignedPtr with alignment = 1. We further compact
the pointer by leveraging the platform specific knowledge. In 64bit architectures, the effective width of pointers are less than 64 bit.
In x86_64, it is 48 bits. And Darwin ARM64 is further smaller, 36 bits. This information allows us to compact the pointer to 6 bytes in
x86_64 and 5 bytes in Darwin ARM64.
- WTF::PackedAlignedPtr
WTF::PackedAlignedPtr is the WTF::PackedPtr with alignment information of the T. If we use this alignment information, we could reduce the
size of packed pointer further in some cases. For example, since we guarantee that JSCells are 16 byte aligned, low 4 bits are empty. Leveraging
this information in Darwin ARM64 platform allows us to make packed JSCell pointer 4 bytes (36 - 4 bits). We do not use passed alignment
information if it is not profitable.
We also have PackedPtrTraits. This is new PtrTraits and use it for various data structures such as Bag<>.
[1]: https://en.cppreference.com/w/cpp/types/is_trivial
* WTF.xcodeproj/project.pbxproj:
* wtf/Bag.h:
(WTF::Bag::clear):
(WTF::Bag::iterator::operator++):
* wtf/CMakeLists.txt:
* wtf/DumbPtrTraits.h:
* wtf/DumbValueTraits.h:
* wtf/MathExtras.h:
(WTF::clzConstexpr):
(WTF::clz):
(WTF::ctzConstexpr):
(WTF::ctz):
(WTF::getLSBSetConstexpr):
(WTF::getMSBSetConstexpr):
* wtf/Packed.h: Added.
(WTF::Packed::Packed):
(WTF::Packed::get const):
(WTF::Packed::set):
(WTF::Packed::operator=):
(WTF::Packed::exchange):
(WTF::Packed::swap):
(WTF::alignof):
(WTF::PackedPtrTraits::exchange):
(WTF::PackedPtrTraits::swap):
(WTF::PackedPtrTraits::unwrap):
* wtf/Platform.h:
* wtf/SentinelLinkedList.h:
(WTF::BasicRawSentinelNode::BasicRawSentinelNode):
(WTF::BasicRawSentinelNode::prev):
(WTF::BasicRawSentinelNode::next):
(WTF::PtrTraits>::remove):
(WTF::PtrTraits>::prepend):
(WTF::PtrTraits>::append):
(WTF::RawNode>::SentinelLinkedList):
(WTF::RawNode>::remove):
(WTF::BasicRawSentinelNode<T>::remove): Deleted.
(WTF::BasicRawSentinelNode<T>::prepend): Deleted.
(WTF::BasicRawSentinelNode<T>::append): Deleted.
* wtf/StdLibExtras.h:
(WTF::roundUpToMultipleOfImpl):
(WTF::roundUpToMultipleOfImpl0): Deleted.
* wtf/UnalignedAccess.h:
(WTF::unalignedLoad):
(WTF::unalignedStore):
Tools:
* TestWebKitAPI/CMakeLists.txt:
* TestWebKitAPI/TestWebKitAPI.xcodeproj/project.pbxproj:
* TestWebKitAPI/Tests/WTF/MathExtras.cpp:
(TestWebKitAPI::TEST):
* TestWebKitAPI/Tests/WTF/Packed.cpp: Added.
(TestWebKitAPI::TEST):
Canonical link: https://commits.webkit.org/211952@main
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@245214 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2019-05-12 22:50:21 +00:00
|
|
|
if (isAlignmentShiftProfitable)
|
|
|
|
value <<= alignmentShiftSize;
|
2020-07-15 16:32:01 +00:00
|
|
|
|
|
|
|
#if CPU(X86_64) && !(OS(DARWIN) || OS(LINUX) || OS(WINDOWS))
|
|
|
|
// The AMD specification requires that the most significant 16
|
|
|
|
// bits of any virtual address, bits 48 through 63, must be
|
|
|
|
// copies of bit 47 (in a manner akin to sign extension).
|
|
|
|
//
|
|
|
|
// The above-named OSes will never allocate user space addresses
|
|
|
|
// with bit 47 set, thus are already in canonical form.
|
|
|
|
//
|
|
|
|
// Reference: https://en.wikipedia.org/wiki/X86-64#Virtual_address_space_details
|
|
|
|
constexpr unsigned shiftBits = countOfBits<uintptr_t> - OS_CONSTANT(EFFECTIVE_ADDRESS_WIDTH);
|
|
|
|
value = (bitwise_cast<intptr_t>(value) << shiftBits) >> shiftBits;
|
|
|
|
#endif
|
|
|
|
|
[JSC] Compress Watchpoint size by using enum type and Packed<> data structure
https://bugs.webkit.org/show_bug.cgi?id=197730
Reviewed by Filip Pizlo.
Source/JavaScriptCore:
Watchpoint takes 5~ MB memory in Gmail (total memory starts with 400 - 500 MB), so 1~%. Since it is allocated massively,
reducing each size of Watchpoint reduces memory footprint significantly.
As a first step, this patch uses Packed<> and enum to reduce the size of Watchpoint.
1. Watchpoint should have enum type and should not use vtable. vtable takes one pointer, and it is too costly for such a
memory sensitive objects. We perform downcast and dispatch the method of the derived classes based on this enum. Since
the # of derived Watchpoint classes are limited (Only 8), we can list up them easily. One unfortunate thing is that
we cannot do this for destructor so long as we use "delete" for deleting objects. If we dispatch the destructor of derived
class in the destructor of the base class, we call the destructor of the base class multiple times. delete operator override
does not help since custom delete operator is called after the destructor is called. While we can fix this issue by always
using custom deleter, currently we do not since all the watchpoints do not have members which have non trivial destructor.
Once it is strongly required, we can start using custom deleter, but for now, we do not need to do this.
2. We use Packed<> to compact pointers in Watchpoint. Since Watchpoint is a node of doubly linked list, each one has two
pointers for prev and next. This is also too costly. PackedPtr reduces the size and makes alignment 1.S
3. We use PackedCellPtr<> for JSCells in Watchpoint. This leverages alignment information and makes pointers smaller in
Darwin ARM64. One important thing to note here is that since this pointer is packed, it cannot be found by conservative
GC scan. It is OK for watchpoint since they are allocated in the heap anyway.
We applied this change to Watchpoint and get the following memory reduction. The highlight is that CodeBlockJettisoningWatchpoint in
ARM64 only takes 2 pointers size.
ORIGINAL X86_64 ARM64
WatchpointSet: 40 32 28
CodeBlockJettisoningWatchpoint: 32 19 15
StructureStubClearingWatchpoint: 56 48 40
AdaptiveInferredPropertyValueWatchpointBase::StructureWatchpoint: 24 13 11
AdaptiveInferredPropertyValueWatchpointBase::PropertyWatchpoint: 24 13 11
FunctionRareData::AllocationProfileClearingWatchpoint: 32 19 15
ObjectToStringAdaptiveStructureWatchpoint: 56 48 40
LLIntPrototypeLoadAdaptiveStructureWatchpoint: 64 48 48
DFG::AdaptiveStructureWatchpoint: 56 48 40
While we will re-architect the mechanism of Watchpoint, anyway Packed<> mechanism and enum types will be used too.
* CMakeLists.txt:
* JavaScriptCore.xcodeproj/project.pbxproj:
* Sources.txt:
* bytecode/AdaptiveInferredPropertyValueWatchpointBase.h:
* bytecode/CodeBlockJettisoningWatchpoint.h:
* bytecode/CodeOrigin.h:
* bytecode/LLIntPrototypeLoadAdaptiveStructureWatchpoint.cpp:
(JSC::LLIntPrototypeLoadAdaptiveStructureWatchpoint::LLIntPrototypeLoadAdaptiveStructureWatchpoint):
(JSC::LLIntPrototypeLoadAdaptiveStructureWatchpoint::fireInternal):
* bytecode/LLIntPrototypeLoadAdaptiveStructureWatchpoint.h:
* bytecode/StructureStubClearingWatchpoint.cpp:
(JSC::StructureStubClearingWatchpoint::fireInternal):
* bytecode/StructureStubClearingWatchpoint.h:
* bytecode/Watchpoint.cpp:
(JSC::Watchpoint::fire):
* bytecode/Watchpoint.h:
(JSC::Watchpoint::Watchpoint):
* dfg/DFGAdaptiveStructureWatchpoint.cpp:
(JSC::DFG::AdaptiveStructureWatchpoint::AdaptiveStructureWatchpoint):
* dfg/DFGAdaptiveStructureWatchpoint.h:
* heap/PackedCellPtr.h: Added.
* runtime/FunctionRareData.h:
* runtime/ObjectToStringAdaptiveStructureWatchpoint.cpp: Added.
(JSC::ObjectToStringAdaptiveStructureWatchpoint::ObjectToStringAdaptiveStructureWatchpoint):
(JSC::ObjectToStringAdaptiveStructureWatchpoint::install):
(JSC::ObjectToStringAdaptiveStructureWatchpoint::fireInternal):
* runtime/ObjectToStringAdaptiveStructureWatchpoint.h: Added.
* runtime/StructureRareData.cpp:
(JSC::StructureRareData::clearObjectToStringValue):
(JSC::ObjectToStringAdaptiveStructureWatchpoint::ObjectToStringAdaptiveStructureWatchpoint): Deleted.
(JSC::ObjectToStringAdaptiveStructureWatchpoint::install): Deleted.
(JSC::ObjectToStringAdaptiveStructureWatchpoint::fireInternal): Deleted.
* runtime/StructureRareData.h:
Source/WTF:
This patch introduces a new data structures, WTF::Packed, WTF::PackedPtr, and WTF::PackedAlignedPtr.
- WTF::Packed
WTF::Packed is data storage. We can read and write trivial (in C++ term [1]) data to this storage. The difference to
the usual storage is that the alignment of this storage is always 1. We access the underlying data by using unalignedLoad/unalignedStore.
This class offers alignment = 1 data structure instead of missing the following characteristics.
1. Load / Store are non atomic even if the data size is within a pointer width. We should not use this for a member which can be accessed
in a racy way. (e.g. fields accessed optimistically from the concurrent compilers).
2. We cannot take reference / pointer to the underlying storage since they are unaligned.
3. Access to this storage is unaligned access. The code is using memcpy, and the compiler will convert to an appropriate unaligned access
in certain architectures (x86_64 / ARM64). It could be slow. So use it for non performance sensitive & memory sensitive places.
- WTF::PackedPtr
WTF::PackedPtr is a specialization of WTF::Packed<T*>. And it is basically WTF::PackedAlignedPtr with alignment = 1. We further compact
the pointer by leveraging the platform specific knowledge. In 64bit architectures, the effective width of pointers are less than 64 bit.
In x86_64, it is 48 bits. And Darwin ARM64 is further smaller, 36 bits. This information allows us to compact the pointer to 6 bytes in
x86_64 and 5 bytes in Darwin ARM64.
- WTF::PackedAlignedPtr
WTF::PackedAlignedPtr is the WTF::PackedPtr with alignment information of the T. If we use this alignment information, we could reduce the
size of packed pointer further in some cases. For example, since we guarantee that JSCells are 16 byte aligned, low 4 bits are empty. Leveraging
this information in Darwin ARM64 platform allows us to make packed JSCell pointer 4 bytes (36 - 4 bits). We do not use passed alignment
information if it is not profitable.
We also have PackedPtrTraits. This is new PtrTraits and use it for various data structures such as Bag<>.
[1]: https://en.cppreference.com/w/cpp/types/is_trivial
* WTF.xcodeproj/project.pbxproj:
* wtf/Bag.h:
(WTF::Bag::clear):
(WTF::Bag::iterator::operator++):
* wtf/CMakeLists.txt:
* wtf/DumbPtrTraits.h:
* wtf/DumbValueTraits.h:
* wtf/MathExtras.h:
(WTF::clzConstexpr):
(WTF::clz):
(WTF::ctzConstexpr):
(WTF::ctz):
(WTF::getLSBSetConstexpr):
(WTF::getMSBSetConstexpr):
* wtf/Packed.h: Added.
(WTF::Packed::Packed):
(WTF::Packed::get const):
(WTF::Packed::set):
(WTF::Packed::operator=):
(WTF::Packed::exchange):
(WTF::Packed::swap):
(WTF::alignof):
(WTF::PackedPtrTraits::exchange):
(WTF::PackedPtrTraits::swap):
(WTF::PackedPtrTraits::unwrap):
* wtf/Platform.h:
* wtf/SentinelLinkedList.h:
(WTF::BasicRawSentinelNode::BasicRawSentinelNode):
(WTF::BasicRawSentinelNode::prev):
(WTF::BasicRawSentinelNode::next):
(WTF::PtrTraits>::remove):
(WTF::PtrTraits>::prepend):
(WTF::PtrTraits>::append):
(WTF::RawNode>::SentinelLinkedList):
(WTF::RawNode>::remove):
(WTF::BasicRawSentinelNode<T>::remove): Deleted.
(WTF::BasicRawSentinelNode<T>::prepend): Deleted.
(WTF::BasicRawSentinelNode<T>::append): Deleted.
* wtf/StdLibExtras.h:
(WTF::roundUpToMultipleOfImpl):
(WTF::roundUpToMultipleOfImpl0): Deleted.
* wtf/UnalignedAccess.h:
(WTF::unalignedLoad):
(WTF::unalignedStore):
Tools:
* TestWebKitAPI/CMakeLists.txt:
* TestWebKitAPI/TestWebKitAPI.xcodeproj/project.pbxproj:
* TestWebKitAPI/Tests/WTF/MathExtras.cpp:
(TestWebKitAPI::TEST):
* TestWebKitAPI/Tests/WTF/Packed.cpp: Added.
(TestWebKitAPI::TEST):
Canonical link: https://commits.webkit.org/211952@main
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@245214 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2019-05-12 22:50:21 +00:00
|
|
|
return bitwise_cast<T*>(value);
|
|
|
|
}
|
|
|
|
|
|
|
|
void set(T* passedValue)
|
|
|
|
{
|
|
|
|
uintptr_t value = bitwise_cast<uintptr_t>(passedValue);
|
|
|
|
if (isAlignmentShiftProfitable)
|
|
|
|
value >>= alignmentShiftSize;
|
|
|
|
#if CPU(LITTLE_ENDIAN)
|
|
|
|
memcpy(m_storage.data(), &value, storageSize);
|
|
|
|
#else
|
2019-05-16 13:38:10 +00:00
|
|
|
memcpy(m_storage.data(), bitwise_cast<uint8_t*>(&value) + (sizeof(void*) - storageSize), storageSize);
|
[JSC] Compress Watchpoint size by using enum type and Packed<> data structure
https://bugs.webkit.org/show_bug.cgi?id=197730
Reviewed by Filip Pizlo.
Source/JavaScriptCore:
Watchpoint takes 5~ MB memory in Gmail (total memory starts with 400 - 500 MB), so 1~%. Since it is allocated massively,
reducing each size of Watchpoint reduces memory footprint significantly.
As a first step, this patch uses Packed<> and enum to reduce the size of Watchpoint.
1. Watchpoint should have enum type and should not use vtable. vtable takes one pointer, and it is too costly for such a
memory sensitive objects. We perform downcast and dispatch the method of the derived classes based on this enum. Since
the # of derived Watchpoint classes are limited (Only 8), we can list up them easily. One unfortunate thing is that
we cannot do this for destructor so long as we use "delete" for deleting objects. If we dispatch the destructor of derived
class in the destructor of the base class, we call the destructor of the base class multiple times. delete operator override
does not help since custom delete operator is called after the destructor is called. While we can fix this issue by always
using custom deleter, currently we do not since all the watchpoints do not have members which have non trivial destructor.
Once it is strongly required, we can start using custom deleter, but for now, we do not need to do this.
2. We use Packed<> to compact pointers in Watchpoint. Since Watchpoint is a node of doubly linked list, each one has two
pointers for prev and next. This is also too costly. PackedPtr reduces the size and makes alignment 1.S
3. We use PackedCellPtr<> for JSCells in Watchpoint. This leverages alignment information and makes pointers smaller in
Darwin ARM64. One important thing to note here is that since this pointer is packed, it cannot be found by conservative
GC scan. It is OK for watchpoint since they are allocated in the heap anyway.
We applied this change to Watchpoint and get the following memory reduction. The highlight is that CodeBlockJettisoningWatchpoint in
ARM64 only takes 2 pointers size.
ORIGINAL X86_64 ARM64
WatchpointSet: 40 32 28
CodeBlockJettisoningWatchpoint: 32 19 15
StructureStubClearingWatchpoint: 56 48 40
AdaptiveInferredPropertyValueWatchpointBase::StructureWatchpoint: 24 13 11
AdaptiveInferredPropertyValueWatchpointBase::PropertyWatchpoint: 24 13 11
FunctionRareData::AllocationProfileClearingWatchpoint: 32 19 15
ObjectToStringAdaptiveStructureWatchpoint: 56 48 40
LLIntPrototypeLoadAdaptiveStructureWatchpoint: 64 48 48
DFG::AdaptiveStructureWatchpoint: 56 48 40
While we will re-architect the mechanism of Watchpoint, anyway Packed<> mechanism and enum types will be used too.
* CMakeLists.txt:
* JavaScriptCore.xcodeproj/project.pbxproj:
* Sources.txt:
* bytecode/AdaptiveInferredPropertyValueWatchpointBase.h:
* bytecode/CodeBlockJettisoningWatchpoint.h:
* bytecode/CodeOrigin.h:
* bytecode/LLIntPrototypeLoadAdaptiveStructureWatchpoint.cpp:
(JSC::LLIntPrototypeLoadAdaptiveStructureWatchpoint::LLIntPrototypeLoadAdaptiveStructureWatchpoint):
(JSC::LLIntPrototypeLoadAdaptiveStructureWatchpoint::fireInternal):
* bytecode/LLIntPrototypeLoadAdaptiveStructureWatchpoint.h:
* bytecode/StructureStubClearingWatchpoint.cpp:
(JSC::StructureStubClearingWatchpoint::fireInternal):
* bytecode/StructureStubClearingWatchpoint.h:
* bytecode/Watchpoint.cpp:
(JSC::Watchpoint::fire):
* bytecode/Watchpoint.h:
(JSC::Watchpoint::Watchpoint):
* dfg/DFGAdaptiveStructureWatchpoint.cpp:
(JSC::DFG::AdaptiveStructureWatchpoint::AdaptiveStructureWatchpoint):
* dfg/DFGAdaptiveStructureWatchpoint.h:
* heap/PackedCellPtr.h: Added.
* runtime/FunctionRareData.h:
* runtime/ObjectToStringAdaptiveStructureWatchpoint.cpp: Added.
(JSC::ObjectToStringAdaptiveStructureWatchpoint::ObjectToStringAdaptiveStructureWatchpoint):
(JSC::ObjectToStringAdaptiveStructureWatchpoint::install):
(JSC::ObjectToStringAdaptiveStructureWatchpoint::fireInternal):
* runtime/ObjectToStringAdaptiveStructureWatchpoint.h: Added.
* runtime/StructureRareData.cpp:
(JSC::StructureRareData::clearObjectToStringValue):
(JSC::ObjectToStringAdaptiveStructureWatchpoint::ObjectToStringAdaptiveStructureWatchpoint): Deleted.
(JSC::ObjectToStringAdaptiveStructureWatchpoint::install): Deleted.
(JSC::ObjectToStringAdaptiveStructureWatchpoint::fireInternal): Deleted.
* runtime/StructureRareData.h:
Source/WTF:
This patch introduces a new data structures, WTF::Packed, WTF::PackedPtr, and WTF::PackedAlignedPtr.
- WTF::Packed
WTF::Packed is data storage. We can read and write trivial (in C++ term [1]) data to this storage. The difference to
the usual storage is that the alignment of this storage is always 1. We access the underlying data by using unalignedLoad/unalignedStore.
This class offers alignment = 1 data structure instead of missing the following characteristics.
1. Load / Store are non atomic even if the data size is within a pointer width. We should not use this for a member which can be accessed
in a racy way. (e.g. fields accessed optimistically from the concurrent compilers).
2. We cannot take reference / pointer to the underlying storage since they are unaligned.
3. Access to this storage is unaligned access. The code is using memcpy, and the compiler will convert to an appropriate unaligned access
in certain architectures (x86_64 / ARM64). It could be slow. So use it for non performance sensitive & memory sensitive places.
- WTF::PackedPtr
WTF::PackedPtr is a specialization of WTF::Packed<T*>. And it is basically WTF::PackedAlignedPtr with alignment = 1. We further compact
the pointer by leveraging the platform specific knowledge. In 64bit architectures, the effective width of pointers are less than 64 bit.
In x86_64, it is 48 bits. And Darwin ARM64 is further smaller, 36 bits. This information allows us to compact the pointer to 6 bytes in
x86_64 and 5 bytes in Darwin ARM64.
- WTF::PackedAlignedPtr
WTF::PackedAlignedPtr is the WTF::PackedPtr with alignment information of the T. If we use this alignment information, we could reduce the
size of packed pointer further in some cases. For example, since we guarantee that JSCells are 16 byte aligned, low 4 bits are empty. Leveraging
this information in Darwin ARM64 platform allows us to make packed JSCell pointer 4 bytes (36 - 4 bits). We do not use passed alignment
information if it is not profitable.
We also have PackedPtrTraits. This is new PtrTraits and use it for various data structures such as Bag<>.
[1]: https://en.cppreference.com/w/cpp/types/is_trivial
* WTF.xcodeproj/project.pbxproj:
* wtf/Bag.h:
(WTF::Bag::clear):
(WTF::Bag::iterator::operator++):
* wtf/CMakeLists.txt:
* wtf/DumbPtrTraits.h:
* wtf/DumbValueTraits.h:
* wtf/MathExtras.h:
(WTF::clzConstexpr):
(WTF::clz):
(WTF::ctzConstexpr):
(WTF::ctz):
(WTF::getLSBSetConstexpr):
(WTF::getMSBSetConstexpr):
* wtf/Packed.h: Added.
(WTF::Packed::Packed):
(WTF::Packed::get const):
(WTF::Packed::set):
(WTF::Packed::operator=):
(WTF::Packed::exchange):
(WTF::Packed::swap):
(WTF::alignof):
(WTF::PackedPtrTraits::exchange):
(WTF::PackedPtrTraits::swap):
(WTF::PackedPtrTraits::unwrap):
* wtf/Platform.h:
* wtf/SentinelLinkedList.h:
(WTF::BasicRawSentinelNode::BasicRawSentinelNode):
(WTF::BasicRawSentinelNode::prev):
(WTF::BasicRawSentinelNode::next):
(WTF::PtrTraits>::remove):
(WTF::PtrTraits>::prepend):
(WTF::PtrTraits>::append):
(WTF::RawNode>::SentinelLinkedList):
(WTF::RawNode>::remove):
(WTF::BasicRawSentinelNode<T>::remove): Deleted.
(WTF::BasicRawSentinelNode<T>::prepend): Deleted.
(WTF::BasicRawSentinelNode<T>::append): Deleted.
* wtf/StdLibExtras.h:
(WTF::roundUpToMultipleOfImpl):
(WTF::roundUpToMultipleOfImpl0): Deleted.
* wtf/UnalignedAccess.h:
(WTF::unalignedLoad):
(WTF::unalignedStore):
Tools:
* TestWebKitAPI/CMakeLists.txt:
* TestWebKitAPI/TestWebKitAPI.xcodeproj/project.pbxproj:
* TestWebKitAPI/Tests/WTF/MathExtras.cpp:
(TestWebKitAPI::TEST):
* TestWebKitAPI/Tests/WTF/Packed.cpp: Added.
(TestWebKitAPI::TEST):
Canonical link: https://commits.webkit.org/211952@main
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@245214 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2019-05-12 22:50:21 +00:00
|
|
|
#endif
|
2020-07-15 16:32:01 +00:00
|
|
|
ASSERT(bitwise_cast<uintptr_t>(get()) == value);
|
[JSC] Compress Watchpoint size by using enum type and Packed<> data structure
https://bugs.webkit.org/show_bug.cgi?id=197730
Reviewed by Filip Pizlo.
Source/JavaScriptCore:
Watchpoint takes 5~ MB memory in Gmail (total memory starts with 400 - 500 MB), so 1~%. Since it is allocated massively,
reducing each size of Watchpoint reduces memory footprint significantly.
As a first step, this patch uses Packed<> and enum to reduce the size of Watchpoint.
1. Watchpoint should have enum type and should not use vtable. vtable takes one pointer, and it is too costly for such a
memory sensitive objects. We perform downcast and dispatch the method of the derived classes based on this enum. Since
the # of derived Watchpoint classes are limited (Only 8), we can list up them easily. One unfortunate thing is that
we cannot do this for destructor so long as we use "delete" for deleting objects. If we dispatch the destructor of derived
class in the destructor of the base class, we call the destructor of the base class multiple times. delete operator override
does not help since custom delete operator is called after the destructor is called. While we can fix this issue by always
using custom deleter, currently we do not since all the watchpoints do not have members which have non trivial destructor.
Once it is strongly required, we can start using custom deleter, but for now, we do not need to do this.
2. We use Packed<> to compact pointers in Watchpoint. Since Watchpoint is a node of doubly linked list, each one has two
pointers for prev and next. This is also too costly. PackedPtr reduces the size and makes alignment 1.S
3. We use PackedCellPtr<> for JSCells in Watchpoint. This leverages alignment information and makes pointers smaller in
Darwin ARM64. One important thing to note here is that since this pointer is packed, it cannot be found by conservative
GC scan. It is OK for watchpoint since they are allocated in the heap anyway.
We applied this change to Watchpoint and get the following memory reduction. The highlight is that CodeBlockJettisoningWatchpoint in
ARM64 only takes 2 pointers size.
ORIGINAL X86_64 ARM64
WatchpointSet: 40 32 28
CodeBlockJettisoningWatchpoint: 32 19 15
StructureStubClearingWatchpoint: 56 48 40
AdaptiveInferredPropertyValueWatchpointBase::StructureWatchpoint: 24 13 11
AdaptiveInferredPropertyValueWatchpointBase::PropertyWatchpoint: 24 13 11
FunctionRareData::AllocationProfileClearingWatchpoint: 32 19 15
ObjectToStringAdaptiveStructureWatchpoint: 56 48 40
LLIntPrototypeLoadAdaptiveStructureWatchpoint: 64 48 48
DFG::AdaptiveStructureWatchpoint: 56 48 40
While we will re-architect the mechanism of Watchpoint, anyway Packed<> mechanism and enum types will be used too.
* CMakeLists.txt:
* JavaScriptCore.xcodeproj/project.pbxproj:
* Sources.txt:
* bytecode/AdaptiveInferredPropertyValueWatchpointBase.h:
* bytecode/CodeBlockJettisoningWatchpoint.h:
* bytecode/CodeOrigin.h:
* bytecode/LLIntPrototypeLoadAdaptiveStructureWatchpoint.cpp:
(JSC::LLIntPrototypeLoadAdaptiveStructureWatchpoint::LLIntPrototypeLoadAdaptiveStructureWatchpoint):
(JSC::LLIntPrototypeLoadAdaptiveStructureWatchpoint::fireInternal):
* bytecode/LLIntPrototypeLoadAdaptiveStructureWatchpoint.h:
* bytecode/StructureStubClearingWatchpoint.cpp:
(JSC::StructureStubClearingWatchpoint::fireInternal):
* bytecode/StructureStubClearingWatchpoint.h:
* bytecode/Watchpoint.cpp:
(JSC::Watchpoint::fire):
* bytecode/Watchpoint.h:
(JSC::Watchpoint::Watchpoint):
* dfg/DFGAdaptiveStructureWatchpoint.cpp:
(JSC::DFG::AdaptiveStructureWatchpoint::AdaptiveStructureWatchpoint):
* dfg/DFGAdaptiveStructureWatchpoint.h:
* heap/PackedCellPtr.h: Added.
* runtime/FunctionRareData.h:
* runtime/ObjectToStringAdaptiveStructureWatchpoint.cpp: Added.
(JSC::ObjectToStringAdaptiveStructureWatchpoint::ObjectToStringAdaptiveStructureWatchpoint):
(JSC::ObjectToStringAdaptiveStructureWatchpoint::install):
(JSC::ObjectToStringAdaptiveStructureWatchpoint::fireInternal):
* runtime/ObjectToStringAdaptiveStructureWatchpoint.h: Added.
* runtime/StructureRareData.cpp:
(JSC::StructureRareData::clearObjectToStringValue):
(JSC::ObjectToStringAdaptiveStructureWatchpoint::ObjectToStringAdaptiveStructureWatchpoint): Deleted.
(JSC::ObjectToStringAdaptiveStructureWatchpoint::install): Deleted.
(JSC::ObjectToStringAdaptiveStructureWatchpoint::fireInternal): Deleted.
* runtime/StructureRareData.h:
Source/WTF:
This patch introduces a new data structures, WTF::Packed, WTF::PackedPtr, and WTF::PackedAlignedPtr.
- WTF::Packed
WTF::Packed is data storage. We can read and write trivial (in C++ term [1]) data to this storage. The difference to
the usual storage is that the alignment of this storage is always 1. We access the underlying data by using unalignedLoad/unalignedStore.
This class offers alignment = 1 data structure instead of missing the following characteristics.
1. Load / Store are non atomic even if the data size is within a pointer width. We should not use this for a member which can be accessed
in a racy way. (e.g. fields accessed optimistically from the concurrent compilers).
2. We cannot take reference / pointer to the underlying storage since they are unaligned.
3. Access to this storage is unaligned access. The code is using memcpy, and the compiler will convert to an appropriate unaligned access
in certain architectures (x86_64 / ARM64). It could be slow. So use it for non performance sensitive & memory sensitive places.
- WTF::PackedPtr
WTF::PackedPtr is a specialization of WTF::Packed<T*>. And it is basically WTF::PackedAlignedPtr with alignment = 1. We further compact
the pointer by leveraging the platform specific knowledge. In 64bit architectures, the effective width of pointers are less than 64 bit.
In x86_64, it is 48 bits. And Darwin ARM64 is further smaller, 36 bits. This information allows us to compact the pointer to 6 bytes in
x86_64 and 5 bytes in Darwin ARM64.
- WTF::PackedAlignedPtr
WTF::PackedAlignedPtr is the WTF::PackedPtr with alignment information of the T. If we use this alignment information, we could reduce the
size of packed pointer further in some cases. For example, since we guarantee that JSCells are 16 byte aligned, low 4 bits are empty. Leveraging
this information in Darwin ARM64 platform allows us to make packed JSCell pointer 4 bytes (36 - 4 bits). We do not use passed alignment
information if it is not profitable.
We also have PackedPtrTraits. This is new PtrTraits and use it for various data structures such as Bag<>.
[1]: https://en.cppreference.com/w/cpp/types/is_trivial
* WTF.xcodeproj/project.pbxproj:
* wtf/Bag.h:
(WTF::Bag::clear):
(WTF::Bag::iterator::operator++):
* wtf/CMakeLists.txt:
* wtf/DumbPtrTraits.h:
* wtf/DumbValueTraits.h:
* wtf/MathExtras.h:
(WTF::clzConstexpr):
(WTF::clz):
(WTF::ctzConstexpr):
(WTF::ctz):
(WTF::getLSBSetConstexpr):
(WTF::getMSBSetConstexpr):
* wtf/Packed.h: Added.
(WTF::Packed::Packed):
(WTF::Packed::get const):
(WTF::Packed::set):
(WTF::Packed::operator=):
(WTF::Packed::exchange):
(WTF::Packed::swap):
(WTF::alignof):
(WTF::PackedPtrTraits::exchange):
(WTF::PackedPtrTraits::swap):
(WTF::PackedPtrTraits::unwrap):
* wtf/Platform.h:
* wtf/SentinelLinkedList.h:
(WTF::BasicRawSentinelNode::BasicRawSentinelNode):
(WTF::BasicRawSentinelNode::prev):
(WTF::BasicRawSentinelNode::next):
(WTF::PtrTraits>::remove):
(WTF::PtrTraits>::prepend):
(WTF::PtrTraits>::append):
(WTF::RawNode>::SentinelLinkedList):
(WTF::RawNode>::remove):
(WTF::BasicRawSentinelNode<T>::remove): Deleted.
(WTF::BasicRawSentinelNode<T>::prepend): Deleted.
(WTF::BasicRawSentinelNode<T>::append): Deleted.
* wtf/StdLibExtras.h:
(WTF::roundUpToMultipleOfImpl):
(WTF::roundUpToMultipleOfImpl0): Deleted.
* wtf/UnalignedAccess.h:
(WTF::unalignedLoad):
(WTF::unalignedStore):
Tools:
* TestWebKitAPI/CMakeLists.txt:
* TestWebKitAPI/TestWebKitAPI.xcodeproj/project.pbxproj:
* TestWebKitAPI/Tests/WTF/MathExtras.cpp:
(TestWebKitAPI::TEST):
* TestWebKitAPI/Tests/WTF/Packed.cpp: Added.
(TestWebKitAPI::TEST):
Canonical link: https://commits.webkit.org/211952@main
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@245214 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2019-05-12 22:50:21 +00:00
|
|
|
}
|
|
|
|
|
|
|
|
void clear()
|
|
|
|
{
|
|
|
|
set(nullptr);
|
|
|
|
}
|
|
|
|
|
|
|
|
T* operator->() const { return get(); }
|
|
|
|
T& operator*() const { return *get(); }
|
|
|
|
bool operator!() const { return !get(); }
|
2019-12-16 22:17:27 +00:00
|
|
|
|
|
|
|
// This conversion operator allows implicit conversion to bool but not to other integer types.
|
|
|
|
typedef T* (PackedAlignedPtr::*UnspecifiedBoolType);
|
|
|
|
operator UnspecifiedBoolType() const { return get() ? &PackedAlignedPtr::m_storage : nullptr; }
|
2019-05-13 17:32:31 +00:00
|
|
|
explicit operator bool() const { return get(); }
|
[JSC] Compress Watchpoint size by using enum type and Packed<> data structure
https://bugs.webkit.org/show_bug.cgi?id=197730
Reviewed by Filip Pizlo.
Source/JavaScriptCore:
Watchpoint takes 5~ MB memory in Gmail (total memory starts with 400 - 500 MB), so 1~%. Since it is allocated massively,
reducing each size of Watchpoint reduces memory footprint significantly.
As a first step, this patch uses Packed<> and enum to reduce the size of Watchpoint.
1. Watchpoint should have enum type and should not use vtable. vtable takes one pointer, and it is too costly for such a
memory sensitive objects. We perform downcast and dispatch the method of the derived classes based on this enum. Since
the # of derived Watchpoint classes are limited (Only 8), we can list up them easily. One unfortunate thing is that
we cannot do this for destructor so long as we use "delete" for deleting objects. If we dispatch the destructor of derived
class in the destructor of the base class, we call the destructor of the base class multiple times. delete operator override
does not help since custom delete operator is called after the destructor is called. While we can fix this issue by always
using custom deleter, currently we do not since all the watchpoints do not have members which have non trivial destructor.
Once it is strongly required, we can start using custom deleter, but for now, we do not need to do this.
2. We use Packed<> to compact pointers in Watchpoint. Since Watchpoint is a node of doubly linked list, each one has two
pointers for prev and next. This is also too costly. PackedPtr reduces the size and makes alignment 1.S
3. We use PackedCellPtr<> for JSCells in Watchpoint. This leverages alignment information and makes pointers smaller in
Darwin ARM64. One important thing to note here is that since this pointer is packed, it cannot be found by conservative
GC scan. It is OK for watchpoint since they are allocated in the heap anyway.
We applied this change to Watchpoint and get the following memory reduction. The highlight is that CodeBlockJettisoningWatchpoint in
ARM64 only takes 2 pointers size.
ORIGINAL X86_64 ARM64
WatchpointSet: 40 32 28
CodeBlockJettisoningWatchpoint: 32 19 15
StructureStubClearingWatchpoint: 56 48 40
AdaptiveInferredPropertyValueWatchpointBase::StructureWatchpoint: 24 13 11
AdaptiveInferredPropertyValueWatchpointBase::PropertyWatchpoint: 24 13 11
FunctionRareData::AllocationProfileClearingWatchpoint: 32 19 15
ObjectToStringAdaptiveStructureWatchpoint: 56 48 40
LLIntPrototypeLoadAdaptiveStructureWatchpoint: 64 48 48
DFG::AdaptiveStructureWatchpoint: 56 48 40
While we will re-architect the mechanism of Watchpoint, anyway Packed<> mechanism and enum types will be used too.
* CMakeLists.txt:
* JavaScriptCore.xcodeproj/project.pbxproj:
* Sources.txt:
* bytecode/AdaptiveInferredPropertyValueWatchpointBase.h:
* bytecode/CodeBlockJettisoningWatchpoint.h:
* bytecode/CodeOrigin.h:
* bytecode/LLIntPrototypeLoadAdaptiveStructureWatchpoint.cpp:
(JSC::LLIntPrototypeLoadAdaptiveStructureWatchpoint::LLIntPrototypeLoadAdaptiveStructureWatchpoint):
(JSC::LLIntPrototypeLoadAdaptiveStructureWatchpoint::fireInternal):
* bytecode/LLIntPrototypeLoadAdaptiveStructureWatchpoint.h:
* bytecode/StructureStubClearingWatchpoint.cpp:
(JSC::StructureStubClearingWatchpoint::fireInternal):
* bytecode/StructureStubClearingWatchpoint.h:
* bytecode/Watchpoint.cpp:
(JSC::Watchpoint::fire):
* bytecode/Watchpoint.h:
(JSC::Watchpoint::Watchpoint):
* dfg/DFGAdaptiveStructureWatchpoint.cpp:
(JSC::DFG::AdaptiveStructureWatchpoint::AdaptiveStructureWatchpoint):
* dfg/DFGAdaptiveStructureWatchpoint.h:
* heap/PackedCellPtr.h: Added.
* runtime/FunctionRareData.h:
* runtime/ObjectToStringAdaptiveStructureWatchpoint.cpp: Added.
(JSC::ObjectToStringAdaptiveStructureWatchpoint::ObjectToStringAdaptiveStructureWatchpoint):
(JSC::ObjectToStringAdaptiveStructureWatchpoint::install):
(JSC::ObjectToStringAdaptiveStructureWatchpoint::fireInternal):
* runtime/ObjectToStringAdaptiveStructureWatchpoint.h: Added.
* runtime/StructureRareData.cpp:
(JSC::StructureRareData::clearObjectToStringValue):
(JSC::ObjectToStringAdaptiveStructureWatchpoint::ObjectToStringAdaptiveStructureWatchpoint): Deleted.
(JSC::ObjectToStringAdaptiveStructureWatchpoint::install): Deleted.
(JSC::ObjectToStringAdaptiveStructureWatchpoint::fireInternal): Deleted.
* runtime/StructureRareData.h:
Source/WTF:
This patch introduces a new data structures, WTF::Packed, WTF::PackedPtr, and WTF::PackedAlignedPtr.
- WTF::Packed
WTF::Packed is data storage. We can read and write trivial (in C++ term [1]) data to this storage. The difference to
the usual storage is that the alignment of this storage is always 1. We access the underlying data by using unalignedLoad/unalignedStore.
This class offers alignment = 1 data structure instead of missing the following characteristics.
1. Load / Store are non atomic even if the data size is within a pointer width. We should not use this for a member which can be accessed
in a racy way. (e.g. fields accessed optimistically from the concurrent compilers).
2. We cannot take reference / pointer to the underlying storage since they are unaligned.
3. Access to this storage is unaligned access. The code is using memcpy, and the compiler will convert to an appropriate unaligned access
in certain architectures (x86_64 / ARM64). It could be slow. So use it for non performance sensitive & memory sensitive places.
- WTF::PackedPtr
WTF::PackedPtr is a specialization of WTF::Packed<T*>. And it is basically WTF::PackedAlignedPtr with alignment = 1. We further compact
the pointer by leveraging the platform specific knowledge. In 64bit architectures, the effective width of pointers are less than 64 bit.
In x86_64, it is 48 bits. And Darwin ARM64 is further smaller, 36 bits. This information allows us to compact the pointer to 6 bytes in
x86_64 and 5 bytes in Darwin ARM64.
- WTF::PackedAlignedPtr
WTF::PackedAlignedPtr is the WTF::PackedPtr with alignment information of the T. If we use this alignment information, we could reduce the
size of packed pointer further in some cases. For example, since we guarantee that JSCells are 16 byte aligned, low 4 bits are empty. Leveraging
this information in Darwin ARM64 platform allows us to make packed JSCell pointer 4 bytes (36 - 4 bits). We do not use passed alignment
information if it is not profitable.
We also have PackedPtrTraits. This is new PtrTraits and use it for various data structures such as Bag<>.
[1]: https://en.cppreference.com/w/cpp/types/is_trivial
* WTF.xcodeproj/project.pbxproj:
* wtf/Bag.h:
(WTF::Bag::clear):
(WTF::Bag::iterator::operator++):
* wtf/CMakeLists.txt:
* wtf/DumbPtrTraits.h:
* wtf/DumbValueTraits.h:
* wtf/MathExtras.h:
(WTF::clzConstexpr):
(WTF::clz):
(WTF::ctzConstexpr):
(WTF::ctz):
(WTF::getLSBSetConstexpr):
(WTF::getMSBSetConstexpr):
* wtf/Packed.h: Added.
(WTF::Packed::Packed):
(WTF::Packed::get const):
(WTF::Packed::set):
(WTF::Packed::operator=):
(WTF::Packed::exchange):
(WTF::Packed::swap):
(WTF::alignof):
(WTF::PackedPtrTraits::exchange):
(WTF::PackedPtrTraits::swap):
(WTF::PackedPtrTraits::unwrap):
* wtf/Platform.h:
* wtf/SentinelLinkedList.h:
(WTF::BasicRawSentinelNode::BasicRawSentinelNode):
(WTF::BasicRawSentinelNode::prev):
(WTF::BasicRawSentinelNode::next):
(WTF::PtrTraits>::remove):
(WTF::PtrTraits>::prepend):
(WTF::PtrTraits>::append):
(WTF::RawNode>::SentinelLinkedList):
(WTF::RawNode>::remove):
(WTF::BasicRawSentinelNode<T>::remove): Deleted.
(WTF::BasicRawSentinelNode<T>::prepend): Deleted.
(WTF::BasicRawSentinelNode<T>::append): Deleted.
* wtf/StdLibExtras.h:
(WTF::roundUpToMultipleOfImpl):
(WTF::roundUpToMultipleOfImpl0): Deleted.
* wtf/UnalignedAccess.h:
(WTF::unalignedLoad):
(WTF::unalignedStore):
Tools:
* TestWebKitAPI/CMakeLists.txt:
* TestWebKitAPI/TestWebKitAPI.xcodeproj/project.pbxproj:
* TestWebKitAPI/Tests/WTF/MathExtras.cpp:
(TestWebKitAPI::TEST):
* TestWebKitAPI/Tests/WTF/Packed.cpp: Added.
(TestWebKitAPI::TEST):
Canonical link: https://commits.webkit.org/211952@main
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@245214 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2019-05-12 22:50:21 +00:00
|
|
|
|
|
|
|
PackedAlignedPtr& operator=(T* value)
|
|
|
|
{
|
|
|
|
set(value);
|
|
|
|
return *this;
|
|
|
|
}
|
|
|
|
|
|
|
|
template<class U>
|
2019-12-16 22:17:27 +00:00
|
|
|
T* exchange(U&& newValue)
|
[JSC] Compress Watchpoint size by using enum type and Packed<> data structure
https://bugs.webkit.org/show_bug.cgi?id=197730
Reviewed by Filip Pizlo.
Source/JavaScriptCore:
Watchpoint takes 5~ MB memory in Gmail (total memory starts with 400 - 500 MB), so 1~%. Since it is allocated massively,
reducing each size of Watchpoint reduces memory footprint significantly.
As a first step, this patch uses Packed<> and enum to reduce the size of Watchpoint.
1. Watchpoint should have enum type and should not use vtable. vtable takes one pointer, and it is too costly for such a
memory sensitive objects. We perform downcast and dispatch the method of the derived classes based on this enum. Since
the # of derived Watchpoint classes are limited (Only 8), we can list up them easily. One unfortunate thing is that
we cannot do this for destructor so long as we use "delete" for deleting objects. If we dispatch the destructor of derived
class in the destructor of the base class, we call the destructor of the base class multiple times. delete operator override
does not help since custom delete operator is called after the destructor is called. While we can fix this issue by always
using custom deleter, currently we do not since all the watchpoints do not have members which have non trivial destructor.
Once it is strongly required, we can start using custom deleter, but for now, we do not need to do this.
2. We use Packed<> to compact pointers in Watchpoint. Since Watchpoint is a node of doubly linked list, each one has two
pointers for prev and next. This is also too costly. PackedPtr reduces the size and makes alignment 1.S
3. We use PackedCellPtr<> for JSCells in Watchpoint. This leverages alignment information and makes pointers smaller in
Darwin ARM64. One important thing to note here is that since this pointer is packed, it cannot be found by conservative
GC scan. It is OK for watchpoint since they are allocated in the heap anyway.
We applied this change to Watchpoint and get the following memory reduction. The highlight is that CodeBlockJettisoningWatchpoint in
ARM64 only takes 2 pointers size.
ORIGINAL X86_64 ARM64
WatchpointSet: 40 32 28
CodeBlockJettisoningWatchpoint: 32 19 15
StructureStubClearingWatchpoint: 56 48 40
AdaptiveInferredPropertyValueWatchpointBase::StructureWatchpoint: 24 13 11
AdaptiveInferredPropertyValueWatchpointBase::PropertyWatchpoint: 24 13 11
FunctionRareData::AllocationProfileClearingWatchpoint: 32 19 15
ObjectToStringAdaptiveStructureWatchpoint: 56 48 40
LLIntPrototypeLoadAdaptiveStructureWatchpoint: 64 48 48
DFG::AdaptiveStructureWatchpoint: 56 48 40
While we will re-architect the mechanism of Watchpoint, anyway Packed<> mechanism and enum types will be used too.
* CMakeLists.txt:
* JavaScriptCore.xcodeproj/project.pbxproj:
* Sources.txt:
* bytecode/AdaptiveInferredPropertyValueWatchpointBase.h:
* bytecode/CodeBlockJettisoningWatchpoint.h:
* bytecode/CodeOrigin.h:
* bytecode/LLIntPrototypeLoadAdaptiveStructureWatchpoint.cpp:
(JSC::LLIntPrototypeLoadAdaptiveStructureWatchpoint::LLIntPrototypeLoadAdaptiveStructureWatchpoint):
(JSC::LLIntPrototypeLoadAdaptiveStructureWatchpoint::fireInternal):
* bytecode/LLIntPrototypeLoadAdaptiveStructureWatchpoint.h:
* bytecode/StructureStubClearingWatchpoint.cpp:
(JSC::StructureStubClearingWatchpoint::fireInternal):
* bytecode/StructureStubClearingWatchpoint.h:
* bytecode/Watchpoint.cpp:
(JSC::Watchpoint::fire):
* bytecode/Watchpoint.h:
(JSC::Watchpoint::Watchpoint):
* dfg/DFGAdaptiveStructureWatchpoint.cpp:
(JSC::DFG::AdaptiveStructureWatchpoint::AdaptiveStructureWatchpoint):
* dfg/DFGAdaptiveStructureWatchpoint.h:
* heap/PackedCellPtr.h: Added.
* runtime/FunctionRareData.h:
* runtime/ObjectToStringAdaptiveStructureWatchpoint.cpp: Added.
(JSC::ObjectToStringAdaptiveStructureWatchpoint::ObjectToStringAdaptiveStructureWatchpoint):
(JSC::ObjectToStringAdaptiveStructureWatchpoint::install):
(JSC::ObjectToStringAdaptiveStructureWatchpoint::fireInternal):
* runtime/ObjectToStringAdaptiveStructureWatchpoint.h: Added.
* runtime/StructureRareData.cpp:
(JSC::StructureRareData::clearObjectToStringValue):
(JSC::ObjectToStringAdaptiveStructureWatchpoint::ObjectToStringAdaptiveStructureWatchpoint): Deleted.
(JSC::ObjectToStringAdaptiveStructureWatchpoint::install): Deleted.
(JSC::ObjectToStringAdaptiveStructureWatchpoint::fireInternal): Deleted.
* runtime/StructureRareData.h:
Source/WTF:
This patch introduces a new data structures, WTF::Packed, WTF::PackedPtr, and WTF::PackedAlignedPtr.
- WTF::Packed
WTF::Packed is data storage. We can read and write trivial (in C++ term [1]) data to this storage. The difference to
the usual storage is that the alignment of this storage is always 1. We access the underlying data by using unalignedLoad/unalignedStore.
This class offers alignment = 1 data structure instead of missing the following characteristics.
1. Load / Store are non atomic even if the data size is within a pointer width. We should not use this for a member which can be accessed
in a racy way. (e.g. fields accessed optimistically from the concurrent compilers).
2. We cannot take reference / pointer to the underlying storage since they are unaligned.
3. Access to this storage is unaligned access. The code is using memcpy, and the compiler will convert to an appropriate unaligned access
in certain architectures (x86_64 / ARM64). It could be slow. So use it for non performance sensitive & memory sensitive places.
- WTF::PackedPtr
WTF::PackedPtr is a specialization of WTF::Packed<T*>. And it is basically WTF::PackedAlignedPtr with alignment = 1. We further compact
the pointer by leveraging the platform specific knowledge. In 64bit architectures, the effective width of pointers are less than 64 bit.
In x86_64, it is 48 bits. And Darwin ARM64 is further smaller, 36 bits. This information allows us to compact the pointer to 6 bytes in
x86_64 and 5 bytes in Darwin ARM64.
- WTF::PackedAlignedPtr
WTF::PackedAlignedPtr is the WTF::PackedPtr with alignment information of the T. If we use this alignment information, we could reduce the
size of packed pointer further in some cases. For example, since we guarantee that JSCells are 16 byte aligned, low 4 bits are empty. Leveraging
this information in Darwin ARM64 platform allows us to make packed JSCell pointer 4 bytes (36 - 4 bits). We do not use passed alignment
information if it is not profitable.
We also have PackedPtrTraits. This is new PtrTraits and use it for various data structures such as Bag<>.
[1]: https://en.cppreference.com/w/cpp/types/is_trivial
* WTF.xcodeproj/project.pbxproj:
* wtf/Bag.h:
(WTF::Bag::clear):
(WTF::Bag::iterator::operator++):
* wtf/CMakeLists.txt:
* wtf/DumbPtrTraits.h:
* wtf/DumbValueTraits.h:
* wtf/MathExtras.h:
(WTF::clzConstexpr):
(WTF::clz):
(WTF::ctzConstexpr):
(WTF::ctz):
(WTF::getLSBSetConstexpr):
(WTF::getMSBSetConstexpr):
* wtf/Packed.h: Added.
(WTF::Packed::Packed):
(WTF::Packed::get const):
(WTF::Packed::set):
(WTF::Packed::operator=):
(WTF::Packed::exchange):
(WTF::Packed::swap):
(WTF::alignof):
(WTF::PackedPtrTraits::exchange):
(WTF::PackedPtrTraits::swap):
(WTF::PackedPtrTraits::unwrap):
* wtf/Platform.h:
* wtf/SentinelLinkedList.h:
(WTF::BasicRawSentinelNode::BasicRawSentinelNode):
(WTF::BasicRawSentinelNode::prev):
(WTF::BasicRawSentinelNode::next):
(WTF::PtrTraits>::remove):
(WTF::PtrTraits>::prepend):
(WTF::PtrTraits>::append):
(WTF::RawNode>::SentinelLinkedList):
(WTF::RawNode>::remove):
(WTF::BasicRawSentinelNode<T>::remove): Deleted.
(WTF::BasicRawSentinelNode<T>::prepend): Deleted.
(WTF::BasicRawSentinelNode<T>::append): Deleted.
* wtf/StdLibExtras.h:
(WTF::roundUpToMultipleOfImpl):
(WTF::roundUpToMultipleOfImpl0): Deleted.
* wtf/UnalignedAccess.h:
(WTF::unalignedLoad):
(WTF::unalignedStore):
Tools:
* TestWebKitAPI/CMakeLists.txt:
* TestWebKitAPI/TestWebKitAPI.xcodeproj/project.pbxproj:
* TestWebKitAPI/Tests/WTF/MathExtras.cpp:
(TestWebKitAPI::TEST):
* TestWebKitAPI/Tests/WTF/Packed.cpp: Added.
(TestWebKitAPI::TEST):
Canonical link: https://commits.webkit.org/211952@main
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@245214 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2019-05-12 22:50:21 +00:00
|
|
|
{
|
2019-12-16 22:17:27 +00:00
|
|
|
T* oldValue = get();
|
[JSC] Compress Watchpoint size by using enum type and Packed<> data structure
https://bugs.webkit.org/show_bug.cgi?id=197730
Reviewed by Filip Pizlo.
Source/JavaScriptCore:
Watchpoint takes 5~ MB memory in Gmail (total memory starts with 400 - 500 MB), so 1~%. Since it is allocated massively,
reducing each size of Watchpoint reduces memory footprint significantly.
As a first step, this patch uses Packed<> and enum to reduce the size of Watchpoint.
1. Watchpoint should have enum type and should not use vtable. vtable takes one pointer, and it is too costly for such a
memory sensitive objects. We perform downcast and dispatch the method of the derived classes based on this enum. Since
the # of derived Watchpoint classes are limited (Only 8), we can list up them easily. One unfortunate thing is that
we cannot do this for destructor so long as we use "delete" for deleting objects. If we dispatch the destructor of derived
class in the destructor of the base class, we call the destructor of the base class multiple times. delete operator override
does not help since custom delete operator is called after the destructor is called. While we can fix this issue by always
using custom deleter, currently we do not since all the watchpoints do not have members which have non trivial destructor.
Once it is strongly required, we can start using custom deleter, but for now, we do not need to do this.
2. We use Packed<> to compact pointers in Watchpoint. Since Watchpoint is a node of doubly linked list, each one has two
pointers for prev and next. This is also too costly. PackedPtr reduces the size and makes alignment 1.S
3. We use PackedCellPtr<> for JSCells in Watchpoint. This leverages alignment information and makes pointers smaller in
Darwin ARM64. One important thing to note here is that since this pointer is packed, it cannot be found by conservative
GC scan. It is OK for watchpoint since they are allocated in the heap anyway.
We applied this change to Watchpoint and get the following memory reduction. The highlight is that CodeBlockJettisoningWatchpoint in
ARM64 only takes 2 pointers size.
ORIGINAL X86_64 ARM64
WatchpointSet: 40 32 28
CodeBlockJettisoningWatchpoint: 32 19 15
StructureStubClearingWatchpoint: 56 48 40
AdaptiveInferredPropertyValueWatchpointBase::StructureWatchpoint: 24 13 11
AdaptiveInferredPropertyValueWatchpointBase::PropertyWatchpoint: 24 13 11
FunctionRareData::AllocationProfileClearingWatchpoint: 32 19 15
ObjectToStringAdaptiveStructureWatchpoint: 56 48 40
LLIntPrototypeLoadAdaptiveStructureWatchpoint: 64 48 48
DFG::AdaptiveStructureWatchpoint: 56 48 40
While we will re-architect the mechanism of Watchpoint, anyway Packed<> mechanism and enum types will be used too.
* CMakeLists.txt:
* JavaScriptCore.xcodeproj/project.pbxproj:
* Sources.txt:
* bytecode/AdaptiveInferredPropertyValueWatchpointBase.h:
* bytecode/CodeBlockJettisoningWatchpoint.h:
* bytecode/CodeOrigin.h:
* bytecode/LLIntPrototypeLoadAdaptiveStructureWatchpoint.cpp:
(JSC::LLIntPrototypeLoadAdaptiveStructureWatchpoint::LLIntPrototypeLoadAdaptiveStructureWatchpoint):
(JSC::LLIntPrototypeLoadAdaptiveStructureWatchpoint::fireInternal):
* bytecode/LLIntPrototypeLoadAdaptiveStructureWatchpoint.h:
* bytecode/StructureStubClearingWatchpoint.cpp:
(JSC::StructureStubClearingWatchpoint::fireInternal):
* bytecode/StructureStubClearingWatchpoint.h:
* bytecode/Watchpoint.cpp:
(JSC::Watchpoint::fire):
* bytecode/Watchpoint.h:
(JSC::Watchpoint::Watchpoint):
* dfg/DFGAdaptiveStructureWatchpoint.cpp:
(JSC::DFG::AdaptiveStructureWatchpoint::AdaptiveStructureWatchpoint):
* dfg/DFGAdaptiveStructureWatchpoint.h:
* heap/PackedCellPtr.h: Added.
* runtime/FunctionRareData.h:
* runtime/ObjectToStringAdaptiveStructureWatchpoint.cpp: Added.
(JSC::ObjectToStringAdaptiveStructureWatchpoint::ObjectToStringAdaptiveStructureWatchpoint):
(JSC::ObjectToStringAdaptiveStructureWatchpoint::install):
(JSC::ObjectToStringAdaptiveStructureWatchpoint::fireInternal):
* runtime/ObjectToStringAdaptiveStructureWatchpoint.h: Added.
* runtime/StructureRareData.cpp:
(JSC::StructureRareData::clearObjectToStringValue):
(JSC::ObjectToStringAdaptiveStructureWatchpoint::ObjectToStringAdaptiveStructureWatchpoint): Deleted.
(JSC::ObjectToStringAdaptiveStructureWatchpoint::install): Deleted.
(JSC::ObjectToStringAdaptiveStructureWatchpoint::fireInternal): Deleted.
* runtime/StructureRareData.h:
Source/WTF:
This patch introduces a new data structures, WTF::Packed, WTF::PackedPtr, and WTF::PackedAlignedPtr.
- WTF::Packed
WTF::Packed is data storage. We can read and write trivial (in C++ term [1]) data to this storage. The difference to
the usual storage is that the alignment of this storage is always 1. We access the underlying data by using unalignedLoad/unalignedStore.
This class offers alignment = 1 data structure instead of missing the following characteristics.
1. Load / Store are non atomic even if the data size is within a pointer width. We should not use this for a member which can be accessed
in a racy way. (e.g. fields accessed optimistically from the concurrent compilers).
2. We cannot take reference / pointer to the underlying storage since they are unaligned.
3. Access to this storage is unaligned access. The code is using memcpy, and the compiler will convert to an appropriate unaligned access
in certain architectures (x86_64 / ARM64). It could be slow. So use it for non performance sensitive & memory sensitive places.
- WTF::PackedPtr
WTF::PackedPtr is a specialization of WTF::Packed<T*>. And it is basically WTF::PackedAlignedPtr with alignment = 1. We further compact
the pointer by leveraging the platform specific knowledge. In 64bit architectures, the effective width of pointers are less than 64 bit.
In x86_64, it is 48 bits. And Darwin ARM64 is further smaller, 36 bits. This information allows us to compact the pointer to 6 bytes in
x86_64 and 5 bytes in Darwin ARM64.
- WTF::PackedAlignedPtr
WTF::PackedAlignedPtr is the WTF::PackedPtr with alignment information of the T. If we use this alignment information, we could reduce the
size of packed pointer further in some cases. For example, since we guarantee that JSCells are 16 byte aligned, low 4 bits are empty. Leveraging
this information in Darwin ARM64 platform allows us to make packed JSCell pointer 4 bytes (36 - 4 bits). We do not use passed alignment
information if it is not profitable.
We also have PackedPtrTraits. This is new PtrTraits and use it for various data structures such as Bag<>.
[1]: https://en.cppreference.com/w/cpp/types/is_trivial
* WTF.xcodeproj/project.pbxproj:
* wtf/Bag.h:
(WTF::Bag::clear):
(WTF::Bag::iterator::operator++):
* wtf/CMakeLists.txt:
* wtf/DumbPtrTraits.h:
* wtf/DumbValueTraits.h:
* wtf/MathExtras.h:
(WTF::clzConstexpr):
(WTF::clz):
(WTF::ctzConstexpr):
(WTF::ctz):
(WTF::getLSBSetConstexpr):
(WTF::getMSBSetConstexpr):
* wtf/Packed.h: Added.
(WTF::Packed::Packed):
(WTF::Packed::get const):
(WTF::Packed::set):
(WTF::Packed::operator=):
(WTF::Packed::exchange):
(WTF::Packed::swap):
(WTF::alignof):
(WTF::PackedPtrTraits::exchange):
(WTF::PackedPtrTraits::swap):
(WTF::PackedPtrTraits::unwrap):
* wtf/Platform.h:
* wtf/SentinelLinkedList.h:
(WTF::BasicRawSentinelNode::BasicRawSentinelNode):
(WTF::BasicRawSentinelNode::prev):
(WTF::BasicRawSentinelNode::next):
(WTF::PtrTraits>::remove):
(WTF::PtrTraits>::prepend):
(WTF::PtrTraits>::append):
(WTF::RawNode>::SentinelLinkedList):
(WTF::RawNode>::remove):
(WTF::BasicRawSentinelNode<T>::remove): Deleted.
(WTF::BasicRawSentinelNode<T>::prepend): Deleted.
(WTF::BasicRawSentinelNode<T>::append): Deleted.
* wtf/StdLibExtras.h:
(WTF::roundUpToMultipleOfImpl):
(WTF::roundUpToMultipleOfImpl0): Deleted.
* wtf/UnalignedAccess.h:
(WTF::unalignedLoad):
(WTF::unalignedStore):
Tools:
* TestWebKitAPI/CMakeLists.txt:
* TestWebKitAPI/TestWebKitAPI.xcodeproj/project.pbxproj:
* TestWebKitAPI/Tests/WTF/MathExtras.cpp:
(TestWebKitAPI::TEST):
* TestWebKitAPI/Tests/WTF/Packed.cpp: Added.
(TestWebKitAPI::TEST):
Canonical link: https://commits.webkit.org/211952@main
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@245214 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2019-05-12 22:50:21 +00:00
|
|
|
set(std::forward<U>(newValue));
|
|
|
|
return oldValue;
|
|
|
|
}
|
|
|
|
|
|
|
|
void swap(std::nullptr_t) { clear(); }
|
|
|
|
|
|
|
|
void swap(PackedAlignedPtr& other)
|
|
|
|
{
|
|
|
|
m_storage.swap(other.m_storage);
|
|
|
|
}
|
|
|
|
|
|
|
|
template<typename Other, typename = std::enable_if_t<Other::isPackedType>>
|
|
|
|
void swap(Other& other)
|
|
|
|
{
|
2019-12-16 22:17:27 +00:00
|
|
|
T* t1 = get();
|
|
|
|
T* t2 = other.get();
|
[JSC] Compress Watchpoint size by using enum type and Packed<> data structure
https://bugs.webkit.org/show_bug.cgi?id=197730
Reviewed by Filip Pizlo.
Source/JavaScriptCore:
Watchpoint takes 5~ MB memory in Gmail (total memory starts with 400 - 500 MB), so 1~%. Since it is allocated massively,
reducing each size of Watchpoint reduces memory footprint significantly.
As a first step, this patch uses Packed<> and enum to reduce the size of Watchpoint.
1. Watchpoint should have enum type and should not use vtable. vtable takes one pointer, and it is too costly for such a
memory sensitive objects. We perform downcast and dispatch the method of the derived classes based on this enum. Since
the # of derived Watchpoint classes are limited (Only 8), we can list up them easily. One unfortunate thing is that
we cannot do this for destructor so long as we use "delete" for deleting objects. If we dispatch the destructor of derived
class in the destructor of the base class, we call the destructor of the base class multiple times. delete operator override
does not help since custom delete operator is called after the destructor is called. While we can fix this issue by always
using custom deleter, currently we do not since all the watchpoints do not have members which have non trivial destructor.
Once it is strongly required, we can start using custom deleter, but for now, we do not need to do this.
2. We use Packed<> to compact pointers in Watchpoint. Since Watchpoint is a node of doubly linked list, each one has two
pointers for prev and next. This is also too costly. PackedPtr reduces the size and makes alignment 1.S
3. We use PackedCellPtr<> for JSCells in Watchpoint. This leverages alignment information and makes pointers smaller in
Darwin ARM64. One important thing to note here is that since this pointer is packed, it cannot be found by conservative
GC scan. It is OK for watchpoint since they are allocated in the heap anyway.
We applied this change to Watchpoint and get the following memory reduction. The highlight is that CodeBlockJettisoningWatchpoint in
ARM64 only takes 2 pointers size.
ORIGINAL X86_64 ARM64
WatchpointSet: 40 32 28
CodeBlockJettisoningWatchpoint: 32 19 15
StructureStubClearingWatchpoint: 56 48 40
AdaptiveInferredPropertyValueWatchpointBase::StructureWatchpoint: 24 13 11
AdaptiveInferredPropertyValueWatchpointBase::PropertyWatchpoint: 24 13 11
FunctionRareData::AllocationProfileClearingWatchpoint: 32 19 15
ObjectToStringAdaptiveStructureWatchpoint: 56 48 40
LLIntPrototypeLoadAdaptiveStructureWatchpoint: 64 48 48
DFG::AdaptiveStructureWatchpoint: 56 48 40
While we will re-architect the mechanism of Watchpoint, anyway Packed<> mechanism and enum types will be used too.
* CMakeLists.txt:
* JavaScriptCore.xcodeproj/project.pbxproj:
* Sources.txt:
* bytecode/AdaptiveInferredPropertyValueWatchpointBase.h:
* bytecode/CodeBlockJettisoningWatchpoint.h:
* bytecode/CodeOrigin.h:
* bytecode/LLIntPrototypeLoadAdaptiveStructureWatchpoint.cpp:
(JSC::LLIntPrototypeLoadAdaptiveStructureWatchpoint::LLIntPrototypeLoadAdaptiveStructureWatchpoint):
(JSC::LLIntPrototypeLoadAdaptiveStructureWatchpoint::fireInternal):
* bytecode/LLIntPrototypeLoadAdaptiveStructureWatchpoint.h:
* bytecode/StructureStubClearingWatchpoint.cpp:
(JSC::StructureStubClearingWatchpoint::fireInternal):
* bytecode/StructureStubClearingWatchpoint.h:
* bytecode/Watchpoint.cpp:
(JSC::Watchpoint::fire):
* bytecode/Watchpoint.h:
(JSC::Watchpoint::Watchpoint):
* dfg/DFGAdaptiveStructureWatchpoint.cpp:
(JSC::DFG::AdaptiveStructureWatchpoint::AdaptiveStructureWatchpoint):
* dfg/DFGAdaptiveStructureWatchpoint.h:
* heap/PackedCellPtr.h: Added.
* runtime/FunctionRareData.h:
* runtime/ObjectToStringAdaptiveStructureWatchpoint.cpp: Added.
(JSC::ObjectToStringAdaptiveStructureWatchpoint::ObjectToStringAdaptiveStructureWatchpoint):
(JSC::ObjectToStringAdaptiveStructureWatchpoint::install):
(JSC::ObjectToStringAdaptiveStructureWatchpoint::fireInternal):
* runtime/ObjectToStringAdaptiveStructureWatchpoint.h: Added.
* runtime/StructureRareData.cpp:
(JSC::StructureRareData::clearObjectToStringValue):
(JSC::ObjectToStringAdaptiveStructureWatchpoint::ObjectToStringAdaptiveStructureWatchpoint): Deleted.
(JSC::ObjectToStringAdaptiveStructureWatchpoint::install): Deleted.
(JSC::ObjectToStringAdaptiveStructureWatchpoint::fireInternal): Deleted.
* runtime/StructureRareData.h:
Source/WTF:
This patch introduces a new data structures, WTF::Packed, WTF::PackedPtr, and WTF::PackedAlignedPtr.
- WTF::Packed
WTF::Packed is data storage. We can read and write trivial (in C++ term [1]) data to this storage. The difference to
the usual storage is that the alignment of this storage is always 1. We access the underlying data by using unalignedLoad/unalignedStore.
This class offers alignment = 1 data structure instead of missing the following characteristics.
1. Load / Store are non atomic even if the data size is within a pointer width. We should not use this for a member which can be accessed
in a racy way. (e.g. fields accessed optimistically from the concurrent compilers).
2. We cannot take reference / pointer to the underlying storage since they are unaligned.
3. Access to this storage is unaligned access. The code is using memcpy, and the compiler will convert to an appropriate unaligned access
in certain architectures (x86_64 / ARM64). It could be slow. So use it for non performance sensitive & memory sensitive places.
- WTF::PackedPtr
WTF::PackedPtr is a specialization of WTF::Packed<T*>. And it is basically WTF::PackedAlignedPtr with alignment = 1. We further compact
the pointer by leveraging the platform specific knowledge. In 64bit architectures, the effective width of pointers are less than 64 bit.
In x86_64, it is 48 bits. And Darwin ARM64 is further smaller, 36 bits. This information allows us to compact the pointer to 6 bytes in
x86_64 and 5 bytes in Darwin ARM64.
- WTF::PackedAlignedPtr
WTF::PackedAlignedPtr is the WTF::PackedPtr with alignment information of the T. If we use this alignment information, we could reduce the
size of packed pointer further in some cases. For example, since we guarantee that JSCells are 16 byte aligned, low 4 bits are empty. Leveraging
this information in Darwin ARM64 platform allows us to make packed JSCell pointer 4 bytes (36 - 4 bits). We do not use passed alignment
information if it is not profitable.
We also have PackedPtrTraits. This is new PtrTraits and use it for various data structures such as Bag<>.
[1]: https://en.cppreference.com/w/cpp/types/is_trivial
* WTF.xcodeproj/project.pbxproj:
* wtf/Bag.h:
(WTF::Bag::clear):
(WTF::Bag::iterator::operator++):
* wtf/CMakeLists.txt:
* wtf/DumbPtrTraits.h:
* wtf/DumbValueTraits.h:
* wtf/MathExtras.h:
(WTF::clzConstexpr):
(WTF::clz):
(WTF::ctzConstexpr):
(WTF::ctz):
(WTF::getLSBSetConstexpr):
(WTF::getMSBSetConstexpr):
* wtf/Packed.h: Added.
(WTF::Packed::Packed):
(WTF::Packed::get const):
(WTF::Packed::set):
(WTF::Packed::operator=):
(WTF::Packed::exchange):
(WTF::Packed::swap):
(WTF::alignof):
(WTF::PackedPtrTraits::exchange):
(WTF::PackedPtrTraits::swap):
(WTF::PackedPtrTraits::unwrap):
* wtf/Platform.h:
* wtf/SentinelLinkedList.h:
(WTF::BasicRawSentinelNode::BasicRawSentinelNode):
(WTF::BasicRawSentinelNode::prev):
(WTF::BasicRawSentinelNode::next):
(WTF::PtrTraits>::remove):
(WTF::PtrTraits>::prepend):
(WTF::PtrTraits>::append):
(WTF::RawNode>::SentinelLinkedList):
(WTF::RawNode>::remove):
(WTF::BasicRawSentinelNode<T>::remove): Deleted.
(WTF::BasicRawSentinelNode<T>::prepend): Deleted.
(WTF::BasicRawSentinelNode<T>::append): Deleted.
* wtf/StdLibExtras.h:
(WTF::roundUpToMultipleOfImpl):
(WTF::roundUpToMultipleOfImpl0): Deleted.
* wtf/UnalignedAccess.h:
(WTF::unalignedLoad):
(WTF::unalignedStore):
Tools:
* TestWebKitAPI/CMakeLists.txt:
* TestWebKitAPI/TestWebKitAPI.xcodeproj/project.pbxproj:
* TestWebKitAPI/Tests/WTF/MathExtras.cpp:
(TestWebKitAPI::TEST):
* TestWebKitAPI/Tests/WTF/Packed.cpp: Added.
(TestWebKitAPI::TEST):
Canonical link: https://commits.webkit.org/211952@main
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@245214 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2019-05-12 22:50:21 +00:00
|
|
|
set(t2);
|
|
|
|
other.set(t1);
|
|
|
|
}
|
|
|
|
|
2019-12-16 22:17:27 +00:00
|
|
|
void swap(T* t2)
|
[JSC] Compress Watchpoint size by using enum type and Packed<> data structure
https://bugs.webkit.org/show_bug.cgi?id=197730
Reviewed by Filip Pizlo.
Source/JavaScriptCore:
Watchpoint takes 5~ MB memory in Gmail (total memory starts with 400 - 500 MB), so 1~%. Since it is allocated massively,
reducing each size of Watchpoint reduces memory footprint significantly.
As a first step, this patch uses Packed<> and enum to reduce the size of Watchpoint.
1. Watchpoint should have enum type and should not use vtable. vtable takes one pointer, and it is too costly for such a
memory sensitive objects. We perform downcast and dispatch the method of the derived classes based on this enum. Since
the # of derived Watchpoint classes are limited (Only 8), we can list up them easily. One unfortunate thing is that
we cannot do this for destructor so long as we use "delete" for deleting objects. If we dispatch the destructor of derived
class in the destructor of the base class, we call the destructor of the base class multiple times. delete operator override
does not help since custom delete operator is called after the destructor is called. While we can fix this issue by always
using custom deleter, currently we do not since all the watchpoints do not have members which have non trivial destructor.
Once it is strongly required, we can start using custom deleter, but for now, we do not need to do this.
2. We use Packed<> to compact pointers in Watchpoint. Since Watchpoint is a node of doubly linked list, each one has two
pointers for prev and next. This is also too costly. PackedPtr reduces the size and makes alignment 1.S
3. We use PackedCellPtr<> for JSCells in Watchpoint. This leverages alignment information and makes pointers smaller in
Darwin ARM64. One important thing to note here is that since this pointer is packed, it cannot be found by conservative
GC scan. It is OK for watchpoint since they are allocated in the heap anyway.
We applied this change to Watchpoint and get the following memory reduction. The highlight is that CodeBlockJettisoningWatchpoint in
ARM64 only takes 2 pointers size.
ORIGINAL X86_64 ARM64
WatchpointSet: 40 32 28
CodeBlockJettisoningWatchpoint: 32 19 15
StructureStubClearingWatchpoint: 56 48 40
AdaptiveInferredPropertyValueWatchpointBase::StructureWatchpoint: 24 13 11
AdaptiveInferredPropertyValueWatchpointBase::PropertyWatchpoint: 24 13 11
FunctionRareData::AllocationProfileClearingWatchpoint: 32 19 15
ObjectToStringAdaptiveStructureWatchpoint: 56 48 40
LLIntPrototypeLoadAdaptiveStructureWatchpoint: 64 48 48
DFG::AdaptiveStructureWatchpoint: 56 48 40
While we will re-architect the mechanism of Watchpoint, anyway Packed<> mechanism and enum types will be used too.
* CMakeLists.txt:
* JavaScriptCore.xcodeproj/project.pbxproj:
* Sources.txt:
* bytecode/AdaptiveInferredPropertyValueWatchpointBase.h:
* bytecode/CodeBlockJettisoningWatchpoint.h:
* bytecode/CodeOrigin.h:
* bytecode/LLIntPrototypeLoadAdaptiveStructureWatchpoint.cpp:
(JSC::LLIntPrototypeLoadAdaptiveStructureWatchpoint::LLIntPrototypeLoadAdaptiveStructureWatchpoint):
(JSC::LLIntPrototypeLoadAdaptiveStructureWatchpoint::fireInternal):
* bytecode/LLIntPrototypeLoadAdaptiveStructureWatchpoint.h:
* bytecode/StructureStubClearingWatchpoint.cpp:
(JSC::StructureStubClearingWatchpoint::fireInternal):
* bytecode/StructureStubClearingWatchpoint.h:
* bytecode/Watchpoint.cpp:
(JSC::Watchpoint::fire):
* bytecode/Watchpoint.h:
(JSC::Watchpoint::Watchpoint):
* dfg/DFGAdaptiveStructureWatchpoint.cpp:
(JSC::DFG::AdaptiveStructureWatchpoint::AdaptiveStructureWatchpoint):
* dfg/DFGAdaptiveStructureWatchpoint.h:
* heap/PackedCellPtr.h: Added.
* runtime/FunctionRareData.h:
* runtime/ObjectToStringAdaptiveStructureWatchpoint.cpp: Added.
(JSC::ObjectToStringAdaptiveStructureWatchpoint::ObjectToStringAdaptiveStructureWatchpoint):
(JSC::ObjectToStringAdaptiveStructureWatchpoint::install):
(JSC::ObjectToStringAdaptiveStructureWatchpoint::fireInternal):
* runtime/ObjectToStringAdaptiveStructureWatchpoint.h: Added.
* runtime/StructureRareData.cpp:
(JSC::StructureRareData::clearObjectToStringValue):
(JSC::ObjectToStringAdaptiveStructureWatchpoint::ObjectToStringAdaptiveStructureWatchpoint): Deleted.
(JSC::ObjectToStringAdaptiveStructureWatchpoint::install): Deleted.
(JSC::ObjectToStringAdaptiveStructureWatchpoint::fireInternal): Deleted.
* runtime/StructureRareData.h:
Source/WTF:
This patch introduces a new data structures, WTF::Packed, WTF::PackedPtr, and WTF::PackedAlignedPtr.
- WTF::Packed
WTF::Packed is data storage. We can read and write trivial (in C++ term [1]) data to this storage. The difference to
the usual storage is that the alignment of this storage is always 1. We access the underlying data by using unalignedLoad/unalignedStore.
This class offers alignment = 1 data structure instead of missing the following characteristics.
1. Load / Store are non atomic even if the data size is within a pointer width. We should not use this for a member which can be accessed
in a racy way. (e.g. fields accessed optimistically from the concurrent compilers).
2. We cannot take reference / pointer to the underlying storage since they are unaligned.
3. Access to this storage is unaligned access. The code is using memcpy, and the compiler will convert to an appropriate unaligned access
in certain architectures (x86_64 / ARM64). It could be slow. So use it for non performance sensitive & memory sensitive places.
- WTF::PackedPtr
WTF::PackedPtr is a specialization of WTF::Packed<T*>. And it is basically WTF::PackedAlignedPtr with alignment = 1. We further compact
the pointer by leveraging the platform specific knowledge. In 64bit architectures, the effective width of pointers are less than 64 bit.
In x86_64, it is 48 bits. And Darwin ARM64 is further smaller, 36 bits. This information allows us to compact the pointer to 6 bytes in
x86_64 and 5 bytes in Darwin ARM64.
- WTF::PackedAlignedPtr
WTF::PackedAlignedPtr is the WTF::PackedPtr with alignment information of the T. If we use this alignment information, we could reduce the
size of packed pointer further in some cases. For example, since we guarantee that JSCells are 16 byte aligned, low 4 bits are empty. Leveraging
this information in Darwin ARM64 platform allows us to make packed JSCell pointer 4 bytes (36 - 4 bits). We do not use passed alignment
information if it is not profitable.
We also have PackedPtrTraits. This is new PtrTraits and use it for various data structures such as Bag<>.
[1]: https://en.cppreference.com/w/cpp/types/is_trivial
* WTF.xcodeproj/project.pbxproj:
* wtf/Bag.h:
(WTF::Bag::clear):
(WTF::Bag::iterator::operator++):
* wtf/CMakeLists.txt:
* wtf/DumbPtrTraits.h:
* wtf/DumbValueTraits.h:
* wtf/MathExtras.h:
(WTF::clzConstexpr):
(WTF::clz):
(WTF::ctzConstexpr):
(WTF::ctz):
(WTF::getLSBSetConstexpr):
(WTF::getMSBSetConstexpr):
* wtf/Packed.h: Added.
(WTF::Packed::Packed):
(WTF::Packed::get const):
(WTF::Packed::set):
(WTF::Packed::operator=):
(WTF::Packed::exchange):
(WTF::Packed::swap):
(WTF::alignof):
(WTF::PackedPtrTraits::exchange):
(WTF::PackedPtrTraits::swap):
(WTF::PackedPtrTraits::unwrap):
* wtf/Platform.h:
* wtf/SentinelLinkedList.h:
(WTF::BasicRawSentinelNode::BasicRawSentinelNode):
(WTF::BasicRawSentinelNode::prev):
(WTF::BasicRawSentinelNode::next):
(WTF::PtrTraits>::remove):
(WTF::PtrTraits>::prepend):
(WTF::PtrTraits>::append):
(WTF::RawNode>::SentinelLinkedList):
(WTF::RawNode>::remove):
(WTF::BasicRawSentinelNode<T>::remove): Deleted.
(WTF::BasicRawSentinelNode<T>::prepend): Deleted.
(WTF::BasicRawSentinelNode<T>::append): Deleted.
* wtf/StdLibExtras.h:
(WTF::roundUpToMultipleOfImpl):
(WTF::roundUpToMultipleOfImpl0): Deleted.
* wtf/UnalignedAccess.h:
(WTF::unalignedLoad):
(WTF::unalignedStore):
Tools:
* TestWebKitAPI/CMakeLists.txt:
* TestWebKitAPI/TestWebKitAPI.xcodeproj/project.pbxproj:
* TestWebKitAPI/Tests/WTF/MathExtras.cpp:
(TestWebKitAPI::TEST):
* TestWebKitAPI/Tests/WTF/Packed.cpp: Added.
(TestWebKitAPI::TEST):
Canonical link: https://commits.webkit.org/211952@main
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@245214 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2019-05-12 22:50:21 +00:00
|
|
|
{
|
2019-12-16 22:17:27 +00:00
|
|
|
T* t1 = get();
|
[JSC] Compress Watchpoint size by using enum type and Packed<> data structure
https://bugs.webkit.org/show_bug.cgi?id=197730
Reviewed by Filip Pizlo.
Source/JavaScriptCore:
Watchpoint takes 5~ MB memory in Gmail (total memory starts with 400 - 500 MB), so 1~%. Since it is allocated massively,
reducing each size of Watchpoint reduces memory footprint significantly.
As a first step, this patch uses Packed<> and enum to reduce the size of Watchpoint.
1. Watchpoint should have enum type and should not use vtable. vtable takes one pointer, and it is too costly for such a
memory sensitive objects. We perform downcast and dispatch the method of the derived classes based on this enum. Since
the # of derived Watchpoint classes are limited (Only 8), we can list up them easily. One unfortunate thing is that
we cannot do this for destructor so long as we use "delete" for deleting objects. If we dispatch the destructor of derived
class in the destructor of the base class, we call the destructor of the base class multiple times. delete operator override
does not help since custom delete operator is called after the destructor is called. While we can fix this issue by always
using custom deleter, currently we do not since all the watchpoints do not have members which have non trivial destructor.
Once it is strongly required, we can start using custom deleter, but for now, we do not need to do this.
2. We use Packed<> to compact pointers in Watchpoint. Since Watchpoint is a node of doubly linked list, each one has two
pointers for prev and next. This is also too costly. PackedPtr reduces the size and makes alignment 1.S
3. We use PackedCellPtr<> for JSCells in Watchpoint. This leverages alignment information and makes pointers smaller in
Darwin ARM64. One important thing to note here is that since this pointer is packed, it cannot be found by conservative
GC scan. It is OK for watchpoint since they are allocated in the heap anyway.
We applied this change to Watchpoint and get the following memory reduction. The highlight is that CodeBlockJettisoningWatchpoint in
ARM64 only takes 2 pointers size.
ORIGINAL X86_64 ARM64
WatchpointSet: 40 32 28
CodeBlockJettisoningWatchpoint: 32 19 15
StructureStubClearingWatchpoint: 56 48 40
AdaptiveInferredPropertyValueWatchpointBase::StructureWatchpoint: 24 13 11
AdaptiveInferredPropertyValueWatchpointBase::PropertyWatchpoint: 24 13 11
FunctionRareData::AllocationProfileClearingWatchpoint: 32 19 15
ObjectToStringAdaptiveStructureWatchpoint: 56 48 40
LLIntPrototypeLoadAdaptiveStructureWatchpoint: 64 48 48
DFG::AdaptiveStructureWatchpoint: 56 48 40
While we will re-architect the mechanism of Watchpoint, anyway Packed<> mechanism and enum types will be used too.
* CMakeLists.txt:
* JavaScriptCore.xcodeproj/project.pbxproj:
* Sources.txt:
* bytecode/AdaptiveInferredPropertyValueWatchpointBase.h:
* bytecode/CodeBlockJettisoningWatchpoint.h:
* bytecode/CodeOrigin.h:
* bytecode/LLIntPrototypeLoadAdaptiveStructureWatchpoint.cpp:
(JSC::LLIntPrototypeLoadAdaptiveStructureWatchpoint::LLIntPrototypeLoadAdaptiveStructureWatchpoint):
(JSC::LLIntPrototypeLoadAdaptiveStructureWatchpoint::fireInternal):
* bytecode/LLIntPrototypeLoadAdaptiveStructureWatchpoint.h:
* bytecode/StructureStubClearingWatchpoint.cpp:
(JSC::StructureStubClearingWatchpoint::fireInternal):
* bytecode/StructureStubClearingWatchpoint.h:
* bytecode/Watchpoint.cpp:
(JSC::Watchpoint::fire):
* bytecode/Watchpoint.h:
(JSC::Watchpoint::Watchpoint):
* dfg/DFGAdaptiveStructureWatchpoint.cpp:
(JSC::DFG::AdaptiveStructureWatchpoint::AdaptiveStructureWatchpoint):
* dfg/DFGAdaptiveStructureWatchpoint.h:
* heap/PackedCellPtr.h: Added.
* runtime/FunctionRareData.h:
* runtime/ObjectToStringAdaptiveStructureWatchpoint.cpp: Added.
(JSC::ObjectToStringAdaptiveStructureWatchpoint::ObjectToStringAdaptiveStructureWatchpoint):
(JSC::ObjectToStringAdaptiveStructureWatchpoint::install):
(JSC::ObjectToStringAdaptiveStructureWatchpoint::fireInternal):
* runtime/ObjectToStringAdaptiveStructureWatchpoint.h: Added.
* runtime/StructureRareData.cpp:
(JSC::StructureRareData::clearObjectToStringValue):
(JSC::ObjectToStringAdaptiveStructureWatchpoint::ObjectToStringAdaptiveStructureWatchpoint): Deleted.
(JSC::ObjectToStringAdaptiveStructureWatchpoint::install): Deleted.
(JSC::ObjectToStringAdaptiveStructureWatchpoint::fireInternal): Deleted.
* runtime/StructureRareData.h:
Source/WTF:
This patch introduces a new data structures, WTF::Packed, WTF::PackedPtr, and WTF::PackedAlignedPtr.
- WTF::Packed
WTF::Packed is data storage. We can read and write trivial (in C++ term [1]) data to this storage. The difference to
the usual storage is that the alignment of this storage is always 1. We access the underlying data by using unalignedLoad/unalignedStore.
This class offers alignment = 1 data structure instead of missing the following characteristics.
1. Load / Store are non atomic even if the data size is within a pointer width. We should not use this for a member which can be accessed
in a racy way. (e.g. fields accessed optimistically from the concurrent compilers).
2. We cannot take reference / pointer to the underlying storage since they are unaligned.
3. Access to this storage is unaligned access. The code is using memcpy, and the compiler will convert to an appropriate unaligned access
in certain architectures (x86_64 / ARM64). It could be slow. So use it for non performance sensitive & memory sensitive places.
- WTF::PackedPtr
WTF::PackedPtr is a specialization of WTF::Packed<T*>. And it is basically WTF::PackedAlignedPtr with alignment = 1. We further compact
the pointer by leveraging the platform specific knowledge. In 64bit architectures, the effective width of pointers are less than 64 bit.
In x86_64, it is 48 bits. And Darwin ARM64 is further smaller, 36 bits. This information allows us to compact the pointer to 6 bytes in
x86_64 and 5 bytes in Darwin ARM64.
- WTF::PackedAlignedPtr
WTF::PackedAlignedPtr is the WTF::PackedPtr with alignment information of the T. If we use this alignment information, we could reduce the
size of packed pointer further in some cases. For example, since we guarantee that JSCells are 16 byte aligned, low 4 bits are empty. Leveraging
this information in Darwin ARM64 platform allows us to make packed JSCell pointer 4 bytes (36 - 4 bits). We do not use passed alignment
information if it is not profitable.
We also have PackedPtrTraits. This is new PtrTraits and use it for various data structures such as Bag<>.
[1]: https://en.cppreference.com/w/cpp/types/is_trivial
* WTF.xcodeproj/project.pbxproj:
* wtf/Bag.h:
(WTF::Bag::clear):
(WTF::Bag::iterator::operator++):
* wtf/CMakeLists.txt:
* wtf/DumbPtrTraits.h:
* wtf/DumbValueTraits.h:
* wtf/MathExtras.h:
(WTF::clzConstexpr):
(WTF::clz):
(WTF::ctzConstexpr):
(WTF::ctz):
(WTF::getLSBSetConstexpr):
(WTF::getMSBSetConstexpr):
* wtf/Packed.h: Added.
(WTF::Packed::Packed):
(WTF::Packed::get const):
(WTF::Packed::set):
(WTF::Packed::operator=):
(WTF::Packed::exchange):
(WTF::Packed::swap):
(WTF::alignof):
(WTF::PackedPtrTraits::exchange):
(WTF::PackedPtrTraits::swap):
(WTF::PackedPtrTraits::unwrap):
* wtf/Platform.h:
* wtf/SentinelLinkedList.h:
(WTF::BasicRawSentinelNode::BasicRawSentinelNode):
(WTF::BasicRawSentinelNode::prev):
(WTF::BasicRawSentinelNode::next):
(WTF::PtrTraits>::remove):
(WTF::PtrTraits>::prepend):
(WTF::PtrTraits>::append):
(WTF::RawNode>::SentinelLinkedList):
(WTF::RawNode>::remove):
(WTF::BasicRawSentinelNode<T>::remove): Deleted.
(WTF::BasicRawSentinelNode<T>::prepend): Deleted.
(WTF::BasicRawSentinelNode<T>::append): Deleted.
* wtf/StdLibExtras.h:
(WTF::roundUpToMultipleOfImpl):
(WTF::roundUpToMultipleOfImpl0): Deleted.
* wtf/UnalignedAccess.h:
(WTF::unalignedLoad):
(WTF::unalignedStore):
Tools:
* TestWebKitAPI/CMakeLists.txt:
* TestWebKitAPI/TestWebKitAPI.xcodeproj/project.pbxproj:
* TestWebKitAPI/Tests/WTF/MathExtras.cpp:
(TestWebKitAPI::TEST):
* TestWebKitAPI/Tests/WTF/Packed.cpp: Added.
(TestWebKitAPI::TEST):
Canonical link: https://commits.webkit.org/211952@main
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@245214 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2019-05-12 22:50:21 +00:00
|
|
|
std::swap(t1, t2);
|
|
|
|
set(t1);
|
|
|
|
}
|
|
|
|
|
|
|
|
private:
|
|
|
|
std::array<uint8_t, storageSize> m_storage;
|
|
|
|
};
|
|
|
|
|
|
|
|
template<typename T>
|
|
|
|
class Packed<T*> : public PackedAlignedPtr<T, 1> {
|
|
|
|
public:
|
|
|
|
using Base = PackedAlignedPtr<T, 1>;
|
|
|
|
using Base::Base;
|
2020-01-21 22:05:43 +00:00
|
|
|
|
|
|
|
// Hash table deleted values, which are only constructed and never copied or destroyed.
|
|
|
|
Packed(HashTableDeletedValueType) : Base(bitwise_cast<T*>(static_cast<uintptr_t>(Base::alignment))) { }
|
|
|
|
bool isHashTableDeletedValue() const { return Base::get() == bitwise_cast<T*>(static_cast<uintptr_t>(Base::alignment)); }
|
[JSC] Compress Watchpoint size by using enum type and Packed<> data structure
https://bugs.webkit.org/show_bug.cgi?id=197730
Reviewed by Filip Pizlo.
Source/JavaScriptCore:
Watchpoint takes 5~ MB memory in Gmail (total memory starts with 400 - 500 MB), so 1~%. Since it is allocated massively,
reducing each size of Watchpoint reduces memory footprint significantly.
As a first step, this patch uses Packed<> and enum to reduce the size of Watchpoint.
1. Watchpoint should have enum type and should not use vtable. vtable takes one pointer, and it is too costly for such a
memory sensitive objects. We perform downcast and dispatch the method of the derived classes based on this enum. Since
the # of derived Watchpoint classes are limited (Only 8), we can list up them easily. One unfortunate thing is that
we cannot do this for destructor so long as we use "delete" for deleting objects. If we dispatch the destructor of derived
class in the destructor of the base class, we call the destructor of the base class multiple times. delete operator override
does not help since custom delete operator is called after the destructor is called. While we can fix this issue by always
using custom deleter, currently we do not since all the watchpoints do not have members which have non trivial destructor.
Once it is strongly required, we can start using custom deleter, but for now, we do not need to do this.
2. We use Packed<> to compact pointers in Watchpoint. Since Watchpoint is a node of doubly linked list, each one has two
pointers for prev and next. This is also too costly. PackedPtr reduces the size and makes alignment 1.S
3. We use PackedCellPtr<> for JSCells in Watchpoint. This leverages alignment information and makes pointers smaller in
Darwin ARM64. One important thing to note here is that since this pointer is packed, it cannot be found by conservative
GC scan. It is OK for watchpoint since they are allocated in the heap anyway.
We applied this change to Watchpoint and get the following memory reduction. The highlight is that CodeBlockJettisoningWatchpoint in
ARM64 only takes 2 pointers size.
ORIGINAL X86_64 ARM64
WatchpointSet: 40 32 28
CodeBlockJettisoningWatchpoint: 32 19 15
StructureStubClearingWatchpoint: 56 48 40
AdaptiveInferredPropertyValueWatchpointBase::StructureWatchpoint: 24 13 11
AdaptiveInferredPropertyValueWatchpointBase::PropertyWatchpoint: 24 13 11
FunctionRareData::AllocationProfileClearingWatchpoint: 32 19 15
ObjectToStringAdaptiveStructureWatchpoint: 56 48 40
LLIntPrototypeLoadAdaptiveStructureWatchpoint: 64 48 48
DFG::AdaptiveStructureWatchpoint: 56 48 40
While we will re-architect the mechanism of Watchpoint, anyway Packed<> mechanism and enum types will be used too.
* CMakeLists.txt:
* JavaScriptCore.xcodeproj/project.pbxproj:
* Sources.txt:
* bytecode/AdaptiveInferredPropertyValueWatchpointBase.h:
* bytecode/CodeBlockJettisoningWatchpoint.h:
* bytecode/CodeOrigin.h:
* bytecode/LLIntPrototypeLoadAdaptiveStructureWatchpoint.cpp:
(JSC::LLIntPrototypeLoadAdaptiveStructureWatchpoint::LLIntPrototypeLoadAdaptiveStructureWatchpoint):
(JSC::LLIntPrototypeLoadAdaptiveStructureWatchpoint::fireInternal):
* bytecode/LLIntPrototypeLoadAdaptiveStructureWatchpoint.h:
* bytecode/StructureStubClearingWatchpoint.cpp:
(JSC::StructureStubClearingWatchpoint::fireInternal):
* bytecode/StructureStubClearingWatchpoint.h:
* bytecode/Watchpoint.cpp:
(JSC::Watchpoint::fire):
* bytecode/Watchpoint.h:
(JSC::Watchpoint::Watchpoint):
* dfg/DFGAdaptiveStructureWatchpoint.cpp:
(JSC::DFG::AdaptiveStructureWatchpoint::AdaptiveStructureWatchpoint):
* dfg/DFGAdaptiveStructureWatchpoint.h:
* heap/PackedCellPtr.h: Added.
* runtime/FunctionRareData.h:
* runtime/ObjectToStringAdaptiveStructureWatchpoint.cpp: Added.
(JSC::ObjectToStringAdaptiveStructureWatchpoint::ObjectToStringAdaptiveStructureWatchpoint):
(JSC::ObjectToStringAdaptiveStructureWatchpoint::install):
(JSC::ObjectToStringAdaptiveStructureWatchpoint::fireInternal):
* runtime/ObjectToStringAdaptiveStructureWatchpoint.h: Added.
* runtime/StructureRareData.cpp:
(JSC::StructureRareData::clearObjectToStringValue):
(JSC::ObjectToStringAdaptiveStructureWatchpoint::ObjectToStringAdaptiveStructureWatchpoint): Deleted.
(JSC::ObjectToStringAdaptiveStructureWatchpoint::install): Deleted.
(JSC::ObjectToStringAdaptiveStructureWatchpoint::fireInternal): Deleted.
* runtime/StructureRareData.h:
Source/WTF:
This patch introduces a new data structures, WTF::Packed, WTF::PackedPtr, and WTF::PackedAlignedPtr.
- WTF::Packed
WTF::Packed is data storage. We can read and write trivial (in C++ term [1]) data to this storage. The difference to
the usual storage is that the alignment of this storage is always 1. We access the underlying data by using unalignedLoad/unalignedStore.
This class offers alignment = 1 data structure instead of missing the following characteristics.
1. Load / Store are non atomic even if the data size is within a pointer width. We should not use this for a member which can be accessed
in a racy way. (e.g. fields accessed optimistically from the concurrent compilers).
2. We cannot take reference / pointer to the underlying storage since they are unaligned.
3. Access to this storage is unaligned access. The code is using memcpy, and the compiler will convert to an appropriate unaligned access
in certain architectures (x86_64 / ARM64). It could be slow. So use it for non performance sensitive & memory sensitive places.
- WTF::PackedPtr
WTF::PackedPtr is a specialization of WTF::Packed<T*>. And it is basically WTF::PackedAlignedPtr with alignment = 1. We further compact
the pointer by leveraging the platform specific knowledge. In 64bit architectures, the effective width of pointers are less than 64 bit.
In x86_64, it is 48 bits. And Darwin ARM64 is further smaller, 36 bits. This information allows us to compact the pointer to 6 bytes in
x86_64 and 5 bytes in Darwin ARM64.
- WTF::PackedAlignedPtr
WTF::PackedAlignedPtr is the WTF::PackedPtr with alignment information of the T. If we use this alignment information, we could reduce the
size of packed pointer further in some cases. For example, since we guarantee that JSCells are 16 byte aligned, low 4 bits are empty. Leveraging
this information in Darwin ARM64 platform allows us to make packed JSCell pointer 4 bytes (36 - 4 bits). We do not use passed alignment
information if it is not profitable.
We also have PackedPtrTraits. This is new PtrTraits and use it for various data structures such as Bag<>.
[1]: https://en.cppreference.com/w/cpp/types/is_trivial
* WTF.xcodeproj/project.pbxproj:
* wtf/Bag.h:
(WTF::Bag::clear):
(WTF::Bag::iterator::operator++):
* wtf/CMakeLists.txt:
* wtf/DumbPtrTraits.h:
* wtf/DumbValueTraits.h:
* wtf/MathExtras.h:
(WTF::clzConstexpr):
(WTF::clz):
(WTF::ctzConstexpr):
(WTF::ctz):
(WTF::getLSBSetConstexpr):
(WTF::getMSBSetConstexpr):
* wtf/Packed.h: Added.
(WTF::Packed::Packed):
(WTF::Packed::get const):
(WTF::Packed::set):
(WTF::Packed::operator=):
(WTF::Packed::exchange):
(WTF::Packed::swap):
(WTF::alignof):
(WTF::PackedPtrTraits::exchange):
(WTF::PackedPtrTraits::swap):
(WTF::PackedPtrTraits::unwrap):
* wtf/Platform.h:
* wtf/SentinelLinkedList.h:
(WTF::BasicRawSentinelNode::BasicRawSentinelNode):
(WTF::BasicRawSentinelNode::prev):
(WTF::BasicRawSentinelNode::next):
(WTF::PtrTraits>::remove):
(WTF::PtrTraits>::prepend):
(WTF::PtrTraits>::append):
(WTF::RawNode>::SentinelLinkedList):
(WTF::RawNode>::remove):
(WTF::BasicRawSentinelNode<T>::remove): Deleted.
(WTF::BasicRawSentinelNode<T>::prepend): Deleted.
(WTF::BasicRawSentinelNode<T>::append): Deleted.
* wtf/StdLibExtras.h:
(WTF::roundUpToMultipleOfImpl):
(WTF::roundUpToMultipleOfImpl0): Deleted.
* wtf/UnalignedAccess.h:
(WTF::unalignedLoad):
(WTF::unalignedStore):
Tools:
* TestWebKitAPI/CMakeLists.txt:
* TestWebKitAPI/TestWebKitAPI.xcodeproj/project.pbxproj:
* TestWebKitAPI/Tests/WTF/MathExtras.cpp:
(TestWebKitAPI::TEST):
* TestWebKitAPI/Tests/WTF/Packed.cpp: Added.
(TestWebKitAPI::TEST):
Canonical link: https://commits.webkit.org/211952@main
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@245214 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2019-05-12 22:50:21 +00:00
|
|
|
};
|
|
|
|
|
|
|
|
template<typename T>
|
|
|
|
using PackedPtr = Packed<T*>;
|
|
|
|
|
2020-01-21 22:05:43 +00:00
|
|
|
template <typename T>
|
|
|
|
struct GetPtrHelper<PackedPtr<T>> {
|
|
|
|
using PtrType = T*;
|
|
|
|
static T* getPtr(const PackedPtr<T>& p) { return const_cast<T*>(p.get()); }
|
|
|
|
};
|
|
|
|
|
|
|
|
template <typename T>
|
|
|
|
struct IsSmartPtr<PackedPtr<T>> {
|
|
|
|
static constexpr bool value = true;
|
|
|
|
};
|
|
|
|
|
[JSC] Compress Watchpoint size by using enum type and Packed<> data structure
https://bugs.webkit.org/show_bug.cgi?id=197730
Reviewed by Filip Pizlo.
Source/JavaScriptCore:
Watchpoint takes 5~ MB memory in Gmail (total memory starts with 400 - 500 MB), so 1~%. Since it is allocated massively,
reducing each size of Watchpoint reduces memory footprint significantly.
As a first step, this patch uses Packed<> and enum to reduce the size of Watchpoint.
1. Watchpoint should have enum type and should not use vtable. vtable takes one pointer, and it is too costly for such a
memory sensitive objects. We perform downcast and dispatch the method of the derived classes based on this enum. Since
the # of derived Watchpoint classes are limited (Only 8), we can list up them easily. One unfortunate thing is that
we cannot do this for destructor so long as we use "delete" for deleting objects. If we dispatch the destructor of derived
class in the destructor of the base class, we call the destructor of the base class multiple times. delete operator override
does not help since custom delete operator is called after the destructor is called. While we can fix this issue by always
using custom deleter, currently we do not since all the watchpoints do not have members which have non trivial destructor.
Once it is strongly required, we can start using custom deleter, but for now, we do not need to do this.
2. We use Packed<> to compact pointers in Watchpoint. Since Watchpoint is a node of doubly linked list, each one has two
pointers for prev and next. This is also too costly. PackedPtr reduces the size and makes alignment 1.S
3. We use PackedCellPtr<> for JSCells in Watchpoint. This leverages alignment information and makes pointers smaller in
Darwin ARM64. One important thing to note here is that since this pointer is packed, it cannot be found by conservative
GC scan. It is OK for watchpoint since they are allocated in the heap anyway.
We applied this change to Watchpoint and get the following memory reduction. The highlight is that CodeBlockJettisoningWatchpoint in
ARM64 only takes 2 pointers size.
ORIGINAL X86_64 ARM64
WatchpointSet: 40 32 28
CodeBlockJettisoningWatchpoint: 32 19 15
StructureStubClearingWatchpoint: 56 48 40
AdaptiveInferredPropertyValueWatchpointBase::StructureWatchpoint: 24 13 11
AdaptiveInferredPropertyValueWatchpointBase::PropertyWatchpoint: 24 13 11
FunctionRareData::AllocationProfileClearingWatchpoint: 32 19 15
ObjectToStringAdaptiveStructureWatchpoint: 56 48 40
LLIntPrototypeLoadAdaptiveStructureWatchpoint: 64 48 48
DFG::AdaptiveStructureWatchpoint: 56 48 40
While we will re-architect the mechanism of Watchpoint, anyway Packed<> mechanism and enum types will be used too.
* CMakeLists.txt:
* JavaScriptCore.xcodeproj/project.pbxproj:
* Sources.txt:
* bytecode/AdaptiveInferredPropertyValueWatchpointBase.h:
* bytecode/CodeBlockJettisoningWatchpoint.h:
* bytecode/CodeOrigin.h:
* bytecode/LLIntPrototypeLoadAdaptiveStructureWatchpoint.cpp:
(JSC::LLIntPrototypeLoadAdaptiveStructureWatchpoint::LLIntPrototypeLoadAdaptiveStructureWatchpoint):
(JSC::LLIntPrototypeLoadAdaptiveStructureWatchpoint::fireInternal):
* bytecode/LLIntPrototypeLoadAdaptiveStructureWatchpoint.h:
* bytecode/StructureStubClearingWatchpoint.cpp:
(JSC::StructureStubClearingWatchpoint::fireInternal):
* bytecode/StructureStubClearingWatchpoint.h:
* bytecode/Watchpoint.cpp:
(JSC::Watchpoint::fire):
* bytecode/Watchpoint.h:
(JSC::Watchpoint::Watchpoint):
* dfg/DFGAdaptiveStructureWatchpoint.cpp:
(JSC::DFG::AdaptiveStructureWatchpoint::AdaptiveStructureWatchpoint):
* dfg/DFGAdaptiveStructureWatchpoint.h:
* heap/PackedCellPtr.h: Added.
* runtime/FunctionRareData.h:
* runtime/ObjectToStringAdaptiveStructureWatchpoint.cpp: Added.
(JSC::ObjectToStringAdaptiveStructureWatchpoint::ObjectToStringAdaptiveStructureWatchpoint):
(JSC::ObjectToStringAdaptiveStructureWatchpoint::install):
(JSC::ObjectToStringAdaptiveStructureWatchpoint::fireInternal):
* runtime/ObjectToStringAdaptiveStructureWatchpoint.h: Added.
* runtime/StructureRareData.cpp:
(JSC::StructureRareData::clearObjectToStringValue):
(JSC::ObjectToStringAdaptiveStructureWatchpoint::ObjectToStringAdaptiveStructureWatchpoint): Deleted.
(JSC::ObjectToStringAdaptiveStructureWatchpoint::install): Deleted.
(JSC::ObjectToStringAdaptiveStructureWatchpoint::fireInternal): Deleted.
* runtime/StructureRareData.h:
Source/WTF:
This patch introduces a new data structures, WTF::Packed, WTF::PackedPtr, and WTF::PackedAlignedPtr.
- WTF::Packed
WTF::Packed is data storage. We can read and write trivial (in C++ term [1]) data to this storage. The difference to
the usual storage is that the alignment of this storage is always 1. We access the underlying data by using unalignedLoad/unalignedStore.
This class offers alignment = 1 data structure instead of missing the following characteristics.
1. Load / Store are non atomic even if the data size is within a pointer width. We should not use this for a member which can be accessed
in a racy way. (e.g. fields accessed optimistically from the concurrent compilers).
2. We cannot take reference / pointer to the underlying storage since they are unaligned.
3. Access to this storage is unaligned access. The code is using memcpy, and the compiler will convert to an appropriate unaligned access
in certain architectures (x86_64 / ARM64). It could be slow. So use it for non performance sensitive & memory sensitive places.
- WTF::PackedPtr
WTF::PackedPtr is a specialization of WTF::Packed<T*>. And it is basically WTF::PackedAlignedPtr with alignment = 1. We further compact
the pointer by leveraging the platform specific knowledge. In 64bit architectures, the effective width of pointers are less than 64 bit.
In x86_64, it is 48 bits. And Darwin ARM64 is further smaller, 36 bits. This information allows us to compact the pointer to 6 bytes in
x86_64 and 5 bytes in Darwin ARM64.
- WTF::PackedAlignedPtr
WTF::PackedAlignedPtr is the WTF::PackedPtr with alignment information of the T. If we use this alignment information, we could reduce the
size of packed pointer further in some cases. For example, since we guarantee that JSCells are 16 byte aligned, low 4 bits are empty. Leveraging
this information in Darwin ARM64 platform allows us to make packed JSCell pointer 4 bytes (36 - 4 bits). We do not use passed alignment
information if it is not profitable.
We also have PackedPtrTraits. This is new PtrTraits and use it for various data structures such as Bag<>.
[1]: https://en.cppreference.com/w/cpp/types/is_trivial
* WTF.xcodeproj/project.pbxproj:
* wtf/Bag.h:
(WTF::Bag::clear):
(WTF::Bag::iterator::operator++):
* wtf/CMakeLists.txt:
* wtf/DumbPtrTraits.h:
* wtf/DumbValueTraits.h:
* wtf/MathExtras.h:
(WTF::clzConstexpr):
(WTF::clz):
(WTF::ctzConstexpr):
(WTF::ctz):
(WTF::getLSBSetConstexpr):
(WTF::getMSBSetConstexpr):
* wtf/Packed.h: Added.
(WTF::Packed::Packed):
(WTF::Packed::get const):
(WTF::Packed::set):
(WTF::Packed::operator=):
(WTF::Packed::exchange):
(WTF::Packed::swap):
(WTF::alignof):
(WTF::PackedPtrTraits::exchange):
(WTF::PackedPtrTraits::swap):
(WTF::PackedPtrTraits::unwrap):
* wtf/Platform.h:
* wtf/SentinelLinkedList.h:
(WTF::BasicRawSentinelNode::BasicRawSentinelNode):
(WTF::BasicRawSentinelNode::prev):
(WTF::BasicRawSentinelNode::next):
(WTF::PtrTraits>::remove):
(WTF::PtrTraits>::prepend):
(WTF::PtrTraits>::append):
(WTF::RawNode>::SentinelLinkedList):
(WTF::RawNode>::remove):
(WTF::BasicRawSentinelNode<T>::remove): Deleted.
(WTF::BasicRawSentinelNode<T>::prepend): Deleted.
(WTF::BasicRawSentinelNode<T>::append): Deleted.
* wtf/StdLibExtras.h:
(WTF::roundUpToMultipleOfImpl):
(WTF::roundUpToMultipleOfImpl0): Deleted.
* wtf/UnalignedAccess.h:
(WTF::unalignedLoad):
(WTF::unalignedStore):
Tools:
* TestWebKitAPI/CMakeLists.txt:
* TestWebKitAPI/TestWebKitAPI.xcodeproj/project.pbxproj:
* TestWebKitAPI/Tests/WTF/MathExtras.cpp:
(TestWebKitAPI::TEST):
* TestWebKitAPI/Tests/WTF/Packed.cpp: Added.
(TestWebKitAPI::TEST):
Canonical link: https://commits.webkit.org/211952@main
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@245214 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2019-05-12 22:50:21 +00:00
|
|
|
template<typename T>
|
|
|
|
struct PackedPtrTraits {
|
|
|
|
template<typename U> using RebindTraits = PackedPtrTraits<U>;
|
|
|
|
|
|
|
|
using StorageType = PackedPtr<T>;
|
|
|
|
|
|
|
|
template<class U> static ALWAYS_INLINE T* exchange(StorageType& ptr, U&& newValue) { return ptr.exchange(newValue); }
|
|
|
|
|
|
|
|
template<typename Other> static ALWAYS_INLINE void swap(PackedPtr<T>& a, Other& b) { a.swap(b); }
|
|
|
|
|
|
|
|
static ALWAYS_INLINE T* unwrap(const StorageType& ptr) { return ptr.get(); }
|
2020-01-21 22:05:43 +00:00
|
|
|
|
|
|
|
// We assume that,
|
|
|
|
// 1. The alignment is < 4KB. (It is tested by HashTraits).
|
|
|
|
// 2. The first page (including nullptr) is never mapped.
|
|
|
|
static StorageType hashTableDeletedValue() { return StorageType { bitwise_cast<T*>(static_cast<uintptr_t>(StorageType::alignment)) }; }
|
|
|
|
static ALWAYS_INLINE bool isHashTableDeletedValue(const StorageType& ptr) { return ptr.get() == bitwise_cast<T*>(static_cast<uintptr_t>(StorageType::alignment)); }
|
[JSC] Compress Watchpoint size by using enum type and Packed<> data structure
https://bugs.webkit.org/show_bug.cgi?id=197730
Reviewed by Filip Pizlo.
Source/JavaScriptCore:
Watchpoint takes 5~ MB memory in Gmail (total memory starts with 400 - 500 MB), so 1~%. Since it is allocated massively,
reducing each size of Watchpoint reduces memory footprint significantly.
As a first step, this patch uses Packed<> and enum to reduce the size of Watchpoint.
1. Watchpoint should have enum type and should not use vtable. vtable takes one pointer, and it is too costly for such a
memory sensitive objects. We perform downcast and dispatch the method of the derived classes based on this enum. Since
the # of derived Watchpoint classes are limited (Only 8), we can list up them easily. One unfortunate thing is that
we cannot do this for destructor so long as we use "delete" for deleting objects. If we dispatch the destructor of derived
class in the destructor of the base class, we call the destructor of the base class multiple times. delete operator override
does not help since custom delete operator is called after the destructor is called. While we can fix this issue by always
using custom deleter, currently we do not since all the watchpoints do not have members which have non trivial destructor.
Once it is strongly required, we can start using custom deleter, but for now, we do not need to do this.
2. We use Packed<> to compact pointers in Watchpoint. Since Watchpoint is a node of doubly linked list, each one has two
pointers for prev and next. This is also too costly. PackedPtr reduces the size and makes alignment 1.S
3. We use PackedCellPtr<> for JSCells in Watchpoint. This leverages alignment information and makes pointers smaller in
Darwin ARM64. One important thing to note here is that since this pointer is packed, it cannot be found by conservative
GC scan. It is OK for watchpoint since they are allocated in the heap anyway.
We applied this change to Watchpoint and get the following memory reduction. The highlight is that CodeBlockJettisoningWatchpoint in
ARM64 only takes 2 pointers size.
ORIGINAL X86_64 ARM64
WatchpointSet: 40 32 28
CodeBlockJettisoningWatchpoint: 32 19 15
StructureStubClearingWatchpoint: 56 48 40
AdaptiveInferredPropertyValueWatchpointBase::StructureWatchpoint: 24 13 11
AdaptiveInferredPropertyValueWatchpointBase::PropertyWatchpoint: 24 13 11
FunctionRareData::AllocationProfileClearingWatchpoint: 32 19 15
ObjectToStringAdaptiveStructureWatchpoint: 56 48 40
LLIntPrototypeLoadAdaptiveStructureWatchpoint: 64 48 48
DFG::AdaptiveStructureWatchpoint: 56 48 40
While we will re-architect the mechanism of Watchpoint, anyway Packed<> mechanism and enum types will be used too.
* CMakeLists.txt:
* JavaScriptCore.xcodeproj/project.pbxproj:
* Sources.txt:
* bytecode/AdaptiveInferredPropertyValueWatchpointBase.h:
* bytecode/CodeBlockJettisoningWatchpoint.h:
* bytecode/CodeOrigin.h:
* bytecode/LLIntPrototypeLoadAdaptiveStructureWatchpoint.cpp:
(JSC::LLIntPrototypeLoadAdaptiveStructureWatchpoint::LLIntPrototypeLoadAdaptiveStructureWatchpoint):
(JSC::LLIntPrototypeLoadAdaptiveStructureWatchpoint::fireInternal):
* bytecode/LLIntPrototypeLoadAdaptiveStructureWatchpoint.h:
* bytecode/StructureStubClearingWatchpoint.cpp:
(JSC::StructureStubClearingWatchpoint::fireInternal):
* bytecode/StructureStubClearingWatchpoint.h:
* bytecode/Watchpoint.cpp:
(JSC::Watchpoint::fire):
* bytecode/Watchpoint.h:
(JSC::Watchpoint::Watchpoint):
* dfg/DFGAdaptiveStructureWatchpoint.cpp:
(JSC::DFG::AdaptiveStructureWatchpoint::AdaptiveStructureWatchpoint):
* dfg/DFGAdaptiveStructureWatchpoint.h:
* heap/PackedCellPtr.h: Added.
* runtime/FunctionRareData.h:
* runtime/ObjectToStringAdaptiveStructureWatchpoint.cpp: Added.
(JSC::ObjectToStringAdaptiveStructureWatchpoint::ObjectToStringAdaptiveStructureWatchpoint):
(JSC::ObjectToStringAdaptiveStructureWatchpoint::install):
(JSC::ObjectToStringAdaptiveStructureWatchpoint::fireInternal):
* runtime/ObjectToStringAdaptiveStructureWatchpoint.h: Added.
* runtime/StructureRareData.cpp:
(JSC::StructureRareData::clearObjectToStringValue):
(JSC::ObjectToStringAdaptiveStructureWatchpoint::ObjectToStringAdaptiveStructureWatchpoint): Deleted.
(JSC::ObjectToStringAdaptiveStructureWatchpoint::install): Deleted.
(JSC::ObjectToStringAdaptiveStructureWatchpoint::fireInternal): Deleted.
* runtime/StructureRareData.h:
Source/WTF:
This patch introduces a new data structures, WTF::Packed, WTF::PackedPtr, and WTF::PackedAlignedPtr.
- WTF::Packed
WTF::Packed is data storage. We can read and write trivial (in C++ term [1]) data to this storage. The difference to
the usual storage is that the alignment of this storage is always 1. We access the underlying data by using unalignedLoad/unalignedStore.
This class offers alignment = 1 data structure instead of missing the following characteristics.
1. Load / Store are non atomic even if the data size is within a pointer width. We should not use this for a member which can be accessed
in a racy way. (e.g. fields accessed optimistically from the concurrent compilers).
2. We cannot take reference / pointer to the underlying storage since they are unaligned.
3. Access to this storage is unaligned access. The code is using memcpy, and the compiler will convert to an appropriate unaligned access
in certain architectures (x86_64 / ARM64). It could be slow. So use it for non performance sensitive & memory sensitive places.
- WTF::PackedPtr
WTF::PackedPtr is a specialization of WTF::Packed<T*>. And it is basically WTF::PackedAlignedPtr with alignment = 1. We further compact
the pointer by leveraging the platform specific knowledge. In 64bit architectures, the effective width of pointers are less than 64 bit.
In x86_64, it is 48 bits. And Darwin ARM64 is further smaller, 36 bits. This information allows us to compact the pointer to 6 bytes in
x86_64 and 5 bytes in Darwin ARM64.
- WTF::PackedAlignedPtr
WTF::PackedAlignedPtr is the WTF::PackedPtr with alignment information of the T. If we use this alignment information, we could reduce the
size of packed pointer further in some cases. For example, since we guarantee that JSCells are 16 byte aligned, low 4 bits are empty. Leveraging
this information in Darwin ARM64 platform allows us to make packed JSCell pointer 4 bytes (36 - 4 bits). We do not use passed alignment
information if it is not profitable.
We also have PackedPtrTraits. This is new PtrTraits and use it for various data structures such as Bag<>.
[1]: https://en.cppreference.com/w/cpp/types/is_trivial
* WTF.xcodeproj/project.pbxproj:
* wtf/Bag.h:
(WTF::Bag::clear):
(WTF::Bag::iterator::operator++):
* wtf/CMakeLists.txt:
* wtf/DumbPtrTraits.h:
* wtf/DumbValueTraits.h:
* wtf/MathExtras.h:
(WTF::clzConstexpr):
(WTF::clz):
(WTF::ctzConstexpr):
(WTF::ctz):
(WTF::getLSBSetConstexpr):
(WTF::getMSBSetConstexpr):
* wtf/Packed.h: Added.
(WTF::Packed::Packed):
(WTF::Packed::get const):
(WTF::Packed::set):
(WTF::Packed::operator=):
(WTF::Packed::exchange):
(WTF::Packed::swap):
(WTF::alignof):
(WTF::PackedPtrTraits::exchange):
(WTF::PackedPtrTraits::swap):
(WTF::PackedPtrTraits::unwrap):
* wtf/Platform.h:
* wtf/SentinelLinkedList.h:
(WTF::BasicRawSentinelNode::BasicRawSentinelNode):
(WTF::BasicRawSentinelNode::prev):
(WTF::BasicRawSentinelNode::next):
(WTF::PtrTraits>::remove):
(WTF::PtrTraits>::prepend):
(WTF::PtrTraits>::append):
(WTF::RawNode>::SentinelLinkedList):
(WTF::RawNode>::remove):
(WTF::BasicRawSentinelNode<T>::remove): Deleted.
(WTF::BasicRawSentinelNode<T>::prepend): Deleted.
(WTF::BasicRawSentinelNode<T>::append): Deleted.
* wtf/StdLibExtras.h:
(WTF::roundUpToMultipleOfImpl):
(WTF::roundUpToMultipleOfImpl0): Deleted.
* wtf/UnalignedAccess.h:
(WTF::unalignedLoad):
(WTF::unalignedStore):
Tools:
* TestWebKitAPI/CMakeLists.txt:
* TestWebKitAPI/TestWebKitAPI.xcodeproj/project.pbxproj:
* TestWebKitAPI/Tests/WTF/MathExtras.cpp:
(TestWebKitAPI::TEST):
* TestWebKitAPI/Tests/WTF/Packed.cpp: Added.
(TestWebKitAPI::TEST):
Canonical link: https://commits.webkit.org/211952@main
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@245214 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2019-05-12 22:50:21 +00:00
|
|
|
};
|
|
|
|
|
2020-07-17 00:33:37 +00:00
|
|
|
template<typename P> struct DefaultHash<PackedPtr<P>> : PtrHash<PackedPtr<P>> { };
|
2020-01-21 22:05:43 +00:00
|
|
|
|
[JSC] Compress Watchpoint size by using enum type and Packed<> data structure
https://bugs.webkit.org/show_bug.cgi?id=197730
Reviewed by Filip Pizlo.
Source/JavaScriptCore:
Watchpoint takes 5~ MB memory in Gmail (total memory starts with 400 - 500 MB), so 1~%. Since it is allocated massively,
reducing each size of Watchpoint reduces memory footprint significantly.
As a first step, this patch uses Packed<> and enum to reduce the size of Watchpoint.
1. Watchpoint should have enum type and should not use vtable. vtable takes one pointer, and it is too costly for such a
memory sensitive objects. We perform downcast and dispatch the method of the derived classes based on this enum. Since
the # of derived Watchpoint classes are limited (Only 8), we can list up them easily. One unfortunate thing is that
we cannot do this for destructor so long as we use "delete" for deleting objects. If we dispatch the destructor of derived
class in the destructor of the base class, we call the destructor of the base class multiple times. delete operator override
does not help since custom delete operator is called after the destructor is called. While we can fix this issue by always
using custom deleter, currently we do not since all the watchpoints do not have members which have non trivial destructor.
Once it is strongly required, we can start using custom deleter, but for now, we do not need to do this.
2. We use Packed<> to compact pointers in Watchpoint. Since Watchpoint is a node of doubly linked list, each one has two
pointers for prev and next. This is also too costly. PackedPtr reduces the size and makes alignment 1.S
3. We use PackedCellPtr<> for JSCells in Watchpoint. This leverages alignment information and makes pointers smaller in
Darwin ARM64. One important thing to note here is that since this pointer is packed, it cannot be found by conservative
GC scan. It is OK for watchpoint since they are allocated in the heap anyway.
We applied this change to Watchpoint and get the following memory reduction. The highlight is that CodeBlockJettisoningWatchpoint in
ARM64 only takes 2 pointers size.
ORIGINAL X86_64 ARM64
WatchpointSet: 40 32 28
CodeBlockJettisoningWatchpoint: 32 19 15
StructureStubClearingWatchpoint: 56 48 40
AdaptiveInferredPropertyValueWatchpointBase::StructureWatchpoint: 24 13 11
AdaptiveInferredPropertyValueWatchpointBase::PropertyWatchpoint: 24 13 11
FunctionRareData::AllocationProfileClearingWatchpoint: 32 19 15
ObjectToStringAdaptiveStructureWatchpoint: 56 48 40
LLIntPrototypeLoadAdaptiveStructureWatchpoint: 64 48 48
DFG::AdaptiveStructureWatchpoint: 56 48 40
While we will re-architect the mechanism of Watchpoint, anyway Packed<> mechanism and enum types will be used too.
* CMakeLists.txt:
* JavaScriptCore.xcodeproj/project.pbxproj:
* Sources.txt:
* bytecode/AdaptiveInferredPropertyValueWatchpointBase.h:
* bytecode/CodeBlockJettisoningWatchpoint.h:
* bytecode/CodeOrigin.h:
* bytecode/LLIntPrototypeLoadAdaptiveStructureWatchpoint.cpp:
(JSC::LLIntPrototypeLoadAdaptiveStructureWatchpoint::LLIntPrototypeLoadAdaptiveStructureWatchpoint):
(JSC::LLIntPrototypeLoadAdaptiveStructureWatchpoint::fireInternal):
* bytecode/LLIntPrototypeLoadAdaptiveStructureWatchpoint.h:
* bytecode/StructureStubClearingWatchpoint.cpp:
(JSC::StructureStubClearingWatchpoint::fireInternal):
* bytecode/StructureStubClearingWatchpoint.h:
* bytecode/Watchpoint.cpp:
(JSC::Watchpoint::fire):
* bytecode/Watchpoint.h:
(JSC::Watchpoint::Watchpoint):
* dfg/DFGAdaptiveStructureWatchpoint.cpp:
(JSC::DFG::AdaptiveStructureWatchpoint::AdaptiveStructureWatchpoint):
* dfg/DFGAdaptiveStructureWatchpoint.h:
* heap/PackedCellPtr.h: Added.
* runtime/FunctionRareData.h:
* runtime/ObjectToStringAdaptiveStructureWatchpoint.cpp: Added.
(JSC::ObjectToStringAdaptiveStructureWatchpoint::ObjectToStringAdaptiveStructureWatchpoint):
(JSC::ObjectToStringAdaptiveStructureWatchpoint::install):
(JSC::ObjectToStringAdaptiveStructureWatchpoint::fireInternal):
* runtime/ObjectToStringAdaptiveStructureWatchpoint.h: Added.
* runtime/StructureRareData.cpp:
(JSC::StructureRareData::clearObjectToStringValue):
(JSC::ObjectToStringAdaptiveStructureWatchpoint::ObjectToStringAdaptiveStructureWatchpoint): Deleted.
(JSC::ObjectToStringAdaptiveStructureWatchpoint::install): Deleted.
(JSC::ObjectToStringAdaptiveStructureWatchpoint::fireInternal): Deleted.
* runtime/StructureRareData.h:
Source/WTF:
This patch introduces a new data structures, WTF::Packed, WTF::PackedPtr, and WTF::PackedAlignedPtr.
- WTF::Packed
WTF::Packed is data storage. We can read and write trivial (in C++ term [1]) data to this storage. The difference to
the usual storage is that the alignment of this storage is always 1. We access the underlying data by using unalignedLoad/unalignedStore.
This class offers alignment = 1 data structure instead of missing the following characteristics.
1. Load / Store are non atomic even if the data size is within a pointer width. We should not use this for a member which can be accessed
in a racy way. (e.g. fields accessed optimistically from the concurrent compilers).
2. We cannot take reference / pointer to the underlying storage since they are unaligned.
3. Access to this storage is unaligned access. The code is using memcpy, and the compiler will convert to an appropriate unaligned access
in certain architectures (x86_64 / ARM64). It could be slow. So use it for non performance sensitive & memory sensitive places.
- WTF::PackedPtr
WTF::PackedPtr is a specialization of WTF::Packed<T*>. And it is basically WTF::PackedAlignedPtr with alignment = 1. We further compact
the pointer by leveraging the platform specific knowledge. In 64bit architectures, the effective width of pointers are less than 64 bit.
In x86_64, it is 48 bits. And Darwin ARM64 is further smaller, 36 bits. This information allows us to compact the pointer to 6 bytes in
x86_64 and 5 bytes in Darwin ARM64.
- WTF::PackedAlignedPtr
WTF::PackedAlignedPtr is the WTF::PackedPtr with alignment information of the T. If we use this alignment information, we could reduce the
size of packed pointer further in some cases. For example, since we guarantee that JSCells are 16 byte aligned, low 4 bits are empty. Leveraging
this information in Darwin ARM64 platform allows us to make packed JSCell pointer 4 bytes (36 - 4 bits). We do not use passed alignment
information if it is not profitable.
We also have PackedPtrTraits. This is new PtrTraits and use it for various data structures such as Bag<>.
[1]: https://en.cppreference.com/w/cpp/types/is_trivial
* WTF.xcodeproj/project.pbxproj:
* wtf/Bag.h:
(WTF::Bag::clear):
(WTF::Bag::iterator::operator++):
* wtf/CMakeLists.txt:
* wtf/DumbPtrTraits.h:
* wtf/DumbValueTraits.h:
* wtf/MathExtras.h:
(WTF::clzConstexpr):
(WTF::clz):
(WTF::ctzConstexpr):
(WTF::ctz):
(WTF::getLSBSetConstexpr):
(WTF::getMSBSetConstexpr):
* wtf/Packed.h: Added.
(WTF::Packed::Packed):
(WTF::Packed::get const):
(WTF::Packed::set):
(WTF::Packed::operator=):
(WTF::Packed::exchange):
(WTF::Packed::swap):
(WTF::alignof):
(WTF::PackedPtrTraits::exchange):
(WTF::PackedPtrTraits::swap):
(WTF::PackedPtrTraits::unwrap):
* wtf/Platform.h:
* wtf/SentinelLinkedList.h:
(WTF::BasicRawSentinelNode::BasicRawSentinelNode):
(WTF::BasicRawSentinelNode::prev):
(WTF::BasicRawSentinelNode::next):
(WTF::PtrTraits>::remove):
(WTF::PtrTraits>::prepend):
(WTF::PtrTraits>::append):
(WTF::RawNode>::SentinelLinkedList):
(WTF::RawNode>::remove):
(WTF::BasicRawSentinelNode<T>::remove): Deleted.
(WTF::BasicRawSentinelNode<T>::prepend): Deleted.
(WTF::BasicRawSentinelNode<T>::append): Deleted.
* wtf/StdLibExtras.h:
(WTF::roundUpToMultipleOfImpl):
(WTF::roundUpToMultipleOfImpl0): Deleted.
* wtf/UnalignedAccess.h:
(WTF::unalignedLoad):
(WTF::unalignedStore):
Tools:
* TestWebKitAPI/CMakeLists.txt:
* TestWebKitAPI/TestWebKitAPI.xcodeproj/project.pbxproj:
* TestWebKitAPI/Tests/WTF/MathExtras.cpp:
(TestWebKitAPI::TEST):
* TestWebKitAPI/Tests/WTF/Packed.cpp: Added.
(TestWebKitAPI::TEST):
Canonical link: https://commits.webkit.org/211952@main
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@245214 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2019-05-12 22:50:21 +00:00
|
|
|
} // namespace WTF
|
|
|
|
|
|
|
|
using WTF::Packed;
|
|
|
|
using WTF::PackedAlignedPtr;
|
|
|
|
using WTF::PackedPtr;
|