16976 lines
707 KiB
Plaintext
16976 lines
707 KiB
Plaintext
2021-08-04 Filip Pizlo <fpizlo@apple.com>
|
||
|
||
[libpas] medium size class lookup needs to correctly fence the counting lock read path
|
||
https://bugs.webkit.org/show_bug.cgi?id=228799
|
||
|
||
Reviewed by Tadeu Zagallo.
|
||
|
||
The medium size class lookup does a binary search on a data structure that may mutate; we
|
||
catch that using a counting lock. But the algorithm wasn't fencing the tail end; it's supposed
|
||
to reread the count at the end but that read was not fenced.
|
||
|
||
This adds the fencing using pas_depend. I confirmed that the disassembly does the right thing.
|
||
It adds very little code.
|
||
|
||
Also rebased a test. Libpas tests are very specific about memory usage in some cases, and so
|
||
sometimes you will encounter a test run that requires limits to be adjusted. This happens
|
||
because some tests can sometimes create very complex heap layouts that really do use more
|
||
memory than we asserted, but the assertion had always worked because the test never ran with
|
||
the "wrong" kind of layout. This fixes a one-off test failure I saw when debugging this fix.
|
||
|
||
* libpas/src/libpas/pas_mutation_count.h:
|
||
(pas_mutation_count_matches_with_dependency):
|
||
(pas_mutation_count_matches): Deleted.
|
||
* libpas/src/libpas/pas_segregated_heap.c:
|
||
(medium_directory_tuple_for_index_impl):
|
||
(medium_directory_tuple_for_index_with_lock):
|
||
(pas_segregated_heap_medium_directory_tuple_for_index):
|
||
* libpas/src/test/ThingyAndUtilityHeapAllocationTests.cpp:
|
||
(std::addLargeHeapTests):
|
||
|
||
2021-08-03 Filip Pizlo <fpizlo@apple.com>
|
||
|
||
pas_segmented_vector's iterate functions should handle memory ordering correctly
|
||
https://bugs.webkit.org/show_bug.cgi?id=228746
|
||
|
||
Reviewed by Mark Lam.
|
||
|
||
Also fixed a missing store-store fence in bitfit_directory's use of a segmented vector.
|
||
|
||
* libpas/src/libpas/pas_bitfit_directory.c:
|
||
(pas_bitfit_directory_get_first_free_view):
|
||
* libpas/src/libpas/pas_segmented_vector.h:
|
||
|
||
2021-07-14 Michael Saboff <msaboff@apple.com>
|
||
|
||
[BMalloc] Lazily allocate physical pages
|
||
https://bugs.webkit.org/show_bug.cgi?id=227957
|
||
|
||
Reviewed by Mark Lam.
|
||
|
||
For the Darwin platform we don't need to call madvise(..., MADV_FREE_REUSE) to commit physical
|
||
memory to back a range of allocated virtual memory. Instead the kernel will commit pages
|
||
as they are touched. This can reduce the footprint of a process when there is sparse access
|
||
of allocated memory.
|
||
|
||
Also fixed a minor bug where we allocate memory, but don't update the amount of physical memory
|
||
associated with it.
|
||
|
||
* bmalloc/Heap.cpp:
|
||
(bmalloc::Heap::tryAllocateLargeChunk):
|
||
* bmalloc/VMAllocate.h:
|
||
(bmalloc::vmAllocatePhysicalPages):
|
||
|
||
2021-07-12 Filip Pizlo <fpizlo@apple.com> and Yusuke Suzuki <ysuzuki@apple.com>
|
||
|
||
New malloc algorithm
|
||
https://bugs.webkit.org/show_bug.cgi?id=226404
|
||
|
||
Reviewed by Yusuke Suzuki.
|
||
|
||
This change replaces bmalloc with libpas, by importing libpas and making it expose API that
|
||
looks like the current bmalloc API. Libpas replaces everything bmalloc gave us: the main
|
||
bmalloc API, the Gigacage API, and the IsoHeap API. Libpas also replaces the JSC
|
||
ExecutableAllocator (more on that in the JSC ChangeLog). I've been working on libpas for about
|
||
three years, and you'll be able to see the age of some of it in the copyrights.
|
||
|
||
Libpas is a totally new malloc implementation that focuses on allowing for the creation of lots
|
||
of isolated heaps, and making it possible for those heaps to provide certain type guarantees.
|
||
I'm not going to summarize everything that it does in this changelog; I'll mostly focus on
|
||
things that are interesting to its use in WebKit.
|
||
|
||
Libpas tries hard to make having lots of heaps cheap both by having a good isolated heap
|
||
implementation and by offsetting the cost by having a bunch of space saving improvements for all
|
||
kinds of heaps. Libpas heaps can be configured to be either very compact or very fast. In
|
||
WebKit, we configure libpas to be as fast as possible, except for the jit_heap, where we
|
||
configure it to be as compact as possible. The fast configuration relies on the segregated heap
|
||
for most allocations, while the compact configuration relies on bitfit for most configurations.
|
||
It's possible to use both in combination with any size cut-off before you switch to bitfit.
|
||
|
||
Libpas's segregated heap is competitive to bmalloc in speed. This patch is a speed-up on AS, and
|
||
it's only enabled on AS for now. Libpas is still a regression relative to bmalloc on some
|
||
devices that I have measured, so it should probably stay enabled on AS only for now. This patch
|
||
ought to be a 1% speed-up on Speedometer and MotionMark and be neutral elsewhere.
|
||
|
||
When it comes to memory usage, libpas's segregated heap is a 19% improvement on membuster and a
|
||
11% improvement on RAMification on AS, with most other devices exhibiting similar wins if you
|
||
enable libpas on them.
|
||
|
||
Here's the basic design:
|
||
|
||
- Libpas makes extreme use of generic programming in C, sometimes using macros, but mostly using
|
||
a technique where ALWAYS_INLINE functions are passed structs with function pointers to other
|
||
ALWAYS_INLINE functions, sometimes many layers deep. I've perfected this so that I can cause
|
||
outlining to happen wherever I want without losing the specialization, and I can cause
|
||
anything I want to become generic (i.e. it gets a pointer to that struct with function
|
||
pointers and it really has to call them).
|
||
|
||
This allows libpas to have many copies of the same algorithm, but configured differently.
|
||
That's why, for example, I was able to make libpas's bitfit obey ExecutableAllocator rules
|
||
like that you cannot put any metadata inside the memory you're managing. That's also how
|
||
libpas "scales" it's algorithm: there are the small configurations, and then with a different
|
||
struct, we get the medium configurations (larger overall page size, ability to decommit each
|
||
physical page within the logical page), and the marge configurations (optimized for even
|
||
bigger object sizes).
|
||
|
||
- Allocation uses thread-local caches that hold on to up to a page of memory. This is similar
|
||
overall to bmalloc, but rather different in the details. When allocating from that memory (see
|
||
bmalloc_heap_inlines.h, then pas_try_allocate_intrinsic_primitive.h, then
|
||
pas_try_allocate_common.h, then pas_local_allocator_inlines.h), libpas uses either bump
|
||
pointer or bit search. The bit search is optimized to death. Libpas does lots of clever things
|
||
with bits: for example, each bit represents just a "minalign" worth of memory (so like 16
|
||
bytes), and bits are set at the beginnings of objects. Lots of interesting bit manipulation
|
||
happens on the slow path to quickly turn this into a bitvector where the bits are set at
|
||
free object beginnings, which is what the allocator consumes (see for example
|
||
pas_local_allocator_scan_bits_to_set_up_free_bits() in pas_local_allocator_inlines.h). This
|
||
saves libpas from having to use multiplications on the allocation and deallocation paths.
|
||
|
||
Allocation fast paths are super fast and require no locks. Slow paths use fine-grained
|
||
locking so the likelihood of contention is very low. Each page has a lock, and they do crazy
|
||
things with their locks: pages will all share the same lock so long as that's efficient, but
|
||
will pick up different locks if that causes contention, and then they will share the same
|
||
lock again if contention goes away.
|
||
|
||
Allocation caches can be reclaimed by the scavenger. This is done using an
|
||
ask-nicely-then-more-firmly kind of mechanism: first the scavenger will just ask allocators
|
||
that haven't been used in a while to return their memory, and the thread will service this
|
||
request on the next allocation slow path for any thread-local allocator. But if the thread
|
||
doesn't do this, then the scavenger will eventually suspend the thread and take its caches.
|
||
|
||
- Deallocation uses a log. The log flush amortizes lock acquisition and memory access to libpas
|
||
metadata. This is quite a bit like bmalloc. The differences are: libpas may acquire multiple
|
||
locks during deallocation log flush, but because pages try to share locks, that usually
|
||
doesn't happen. Usually, all pages that were most recently used by some CPU will share the
|
||
same lock. The other difference is that the libpas scavenger can flush deallocation logs, and
|
||
does so on every tick.
|
||
|
||
The design described so far is for the libpas segregated heap, which is the most
|
||
performance-optimized heap, as well as the heap most suited for isoheaps. Segregated heap has
|
||
many isoheap optimizations, like allowing multiple heaps to share different slabs of the same
|
||
page. But libpas also has two other heap kinds:
|
||
|
||
- Bitfit. The bitfit heap uses a bit-per-minalign but uses it to implement first-fit. Allocation
|
||
searches for a span of set "free" bits and then clears them. Deallocation uses a second
|
||
bitvector, the "object ends" bits, to find the end of the object and then sets the whole range
|
||
of free bits. This allocator uses per-page locking and a clever short-circuiting algorithm
|
||
to find the first page that will have a contiguous slab of free memory big enough for the
|
||
size you're trying to allocate. This is much slower than the segregated heap, but also much
|
||
faster than the other option, which is the libpas large allocator. Bitfit is the most
|
||
space-efficient heap in libpas, except for super large objects. The space efficiency comes
|
||
from the fact that it's first-fit. It should be noted that segregated is also first-fit, but
|
||
within size classes, so segregated gets more external fragmentation than bitfit.
|
||
|
||
This patch causes us to use bitfit for the mini mode. We also use bitfit for marge allocations
|
||
(so bigger than segregated medium but too small for large) and for jit_heap.
|
||
|
||
I think the most interesting file for bitfit is pas_bitfit_page_inlines.h.
|
||
|
||
- Large. This is a first-fit allocator implemented using cartesian trees. Something special
|
||
about it is that it can do type-safe array allocations with complex combinations of memalign.
|
||
For example, you can tell it to create a heap that allocates arrays of a type whose size is
|
||
42, and then it will guarantee you that no matter how big or how small of an array you
|
||
allocate, no element of that array will ever be an "offset overlap" with a previously freed
|
||
element. So, it won't create any type confusions. It will guarantee this even if you did
|
||
memalign with any alignment (I needed to use extended GCD for this -- see pas_extended_gcd.c
|
||
and pas_coalign.c). The large algorithm is implemented generically
|
||
(pas_generic_large_free_heap.h), so it can use either cartesian trees
|
||
(pas_fast_large_free_heap.c) or just an array (pas_simple_large_free_heap.c). The large heap
|
||
is used internally in libpas a lot, for example for the bootstrap heap, which is how all of
|
||
libpas gets its memory. One particularly neat thing about this is that the bootstrap heap
|
||
allocates a freelist array inside of the memory it is managing using a kind of wild
|
||
metacircular design.
|
||
|
||
One of the main space saving features is that libpas has a fine-grained-locked LRU decommit
|
||
policy using a 10Hz scavenger thread. This scavenger incurs almost no perf cost but is able
|
||
to return a ton of memory. At any tick, it will decommit any totally free pages that haven't
|
||
been used in the last 300ms. This includes pages used for internal libpas metadata, like the
|
||
thread-local caches. Decommitting pages holds locks that other threads are unlikely to want to
|
||
grab (because of policies that actively avoid it), so decommit happens with minimal interference
|
||
to the running program.
|
||
|
||
Libpas is written in C and strongly assumes that the C compiler is modern enough to fixpoint
|
||
inlining of always-inline functions with the right other optimizations. Llvm does this, which
|
||
enables a wonderful style of generic programming. The fact that the same code can be used
|
||
for generic, specialized-and-inline, and specialized-but-out-of-line functions is great. I use
|
||
WebKit C++ style (so foo* p rather than foo *p) but with lower_case_identifiers for anything
|
||
that isn't a macro. Anything that isn't static is prefixed pas_, unless it's some specific
|
||
heap configuration, like the bmalloc one, which prefixes every non-static with bmalloc_.
|
||
|
||
Libpas includes a giant test suite. Lots of the heap configurations (iso_test, thingy,
|
||
minalign32, pagesize64k) are just for testing. The test suite has both unit tests and chaos
|
||
tests as well as everything in between. The test suite is written in C++.
|
||
|
||
This glues libpas into the DebugHeap mechanism in bmalloc, so Malloc=1 will give you system
|
||
malloc. Libpas supports libmalloc enumeration, and this patch exposes it (though you need a
|
||
shared cache rebuild to enjoy the benefits).
|
||
|
||
All of bmalloc's external-facing API are replaced with libpas when the BUSE(LIBPAS) flag is
|
||
set (so the bmalloc::XYZ functions, Gigacage::XYZ, and IsoHeap<> and friends). This also
|
||
exposes the jit_heap API for the ExecutableAllocator to use.
|
||
|
||
Notes for porting: this uses lots of Darwin APIs, like the APIs for getting an approximate time
|
||
very quickly, APIs for fast TLS, thread suspension, and probably other stuff. It'll be hard,
|
||
but possible, to port to Linux. It may never perform as well on Linux as it does on Darwin.
|
||
It's also worth noting that libpas strongly assumes 64-bit, that the CPU can do 128-bit
|
||
compare-and-swap, that there is no big downside to misaligned loads and stores, and that the
|
||
memory model is either the one in ARM64 or the one on x86_64 (or something not weaker than
|
||
either of them). Trying to make libpas work on 32-bit CPUs will quickly run into cases where
|
||
some pointer math doesn't work right because it was written strongly assuming properties
|
||
unique to 64-bit CPUs (like that some number of top bits are not used for the address), as well
|
||
as cases where lock-free algorithms are using uintptr_t for a versioning scheme (which may not
|
||
be as reliable as you want on 32-bit).
|
||
|
||
* CMakeLists.txt:
|
||
* Configurations/Base.xcconfig:
|
||
* bmalloc.xcodeproj/project.pbxproj:
|
||
* bmalloc/BPlatform.h:
|
||
* bmalloc/DebugHeap.cpp:
|
||
(bmalloc::DebugHeap::tryGetSlow):
|
||
(pas_debug_heap_is_enabled):
|
||
(pas_debug_heap_malloc):
|
||
(pas_debug_heap_memalign):
|
||
(pas_debug_heap_realloc):
|
||
(pas_debug_heap_free):
|
||
* bmalloc/DebugHeap.h:
|
||
(bmalloc::debugHeapDisabled):
|
||
(bmalloc::DebugHeap::tryGet):
|
||
(bmalloc::DebugHeap::getExisting):
|
||
* bmalloc/Gigacage.cpp:
|
||
(Gigacage::ensureGigacage):
|
||
(Gigacage::allocBase):
|
||
(Gigacage::size):
|
||
* bmalloc/Gigacage.h:
|
||
* bmalloc/GigacageConfig.h:
|
||
(Gigacage::Config::allocBasePtr const):
|
||
(Gigacage::Config::setAllocBasePtr):
|
||
(Gigacage::Config::allocSize const):
|
||
(Gigacage::Config::setAllocSize):
|
||
* bmalloc/Heap.cpp:
|
||
(bmalloc::Heap::gigacageSize):
|
||
* bmalloc/Heap.h:
|
||
* bmalloc/IsoHeap.cpp: Added.
|
||
(bmalloc::api::isoAllocate):
|
||
(bmalloc::api::isoTryAllocate):
|
||
(bmalloc::api::isoDeallocate):
|
||
* bmalloc/IsoHeap.h:
|
||
(bmalloc::api::IsoHeap::IsoHeap):
|
||
(bmalloc::api::IsoHeap::allocate):
|
||
(bmalloc::api::IsoHeap::tryAllocate):
|
||
(bmalloc::api::IsoHeap::deallocate):
|
||
(bmalloc::api::IsoHeap::scavenge):
|
||
(bmalloc::api::IsoHeap::initialize):
|
||
(bmalloc::api::IsoHeap::isInitialized):
|
||
* bmalloc/IsoHeapImplInlines.h:
|
||
(bmalloc::IsoHeapImpl<Config>::IsoHeapImpl):
|
||
* bmalloc/IsoHeapInlines.h:
|
||
* bmalloc/IsoMallocFallback.cpp: Added.
|
||
(bmalloc::IsoMallocFallback::tryMalloc):
|
||
(bmalloc::IsoMallocFallback::tryFree):
|
||
* bmalloc/IsoMallocFallback.h: Copied from Source/bmalloc/bmalloc/DebugHeap.h.
|
||
(bmalloc::IsoMallocFallback::shouldTryToFallBack):
|
||
(bmalloc::IsoMallocFallback::MallocResult::MallocResult):
|
||
* bmalloc/IsoTLS.cpp:
|
||
(bmalloc::IsoTLS::determineMallocFallbackState): Deleted.
|
||
* bmalloc/IsoTLS.h:
|
||
* bmalloc/IsoTLSInlines.h:
|
||
(bmalloc::IsoTLS::allocateSlow):
|
||
(bmalloc::IsoTLS::deallocateSlow):
|
||
* bmalloc/PerThread.h:
|
||
(bmalloc::PerThreadStorage<PerHeapKind<Cache>>::init):
|
||
* bmalloc/bmalloc.cpp:
|
||
(bmalloc::api::tryLargeZeroedMemalignVirtual):
|
||
(bmalloc::api::freeLargeVirtual):
|
||
(bmalloc::api::scavengeThisThread):
|
||
(bmalloc::api::scavenge):
|
||
(bmalloc::api::setScavengerThreadQOSClass):
|
||
(bmalloc::api::commitAlignedPhysical):
|
||
(bmalloc::api::decommitAlignedPhysical):
|
||
(bmalloc::api::enableMiniMode):
|
||
(bmalloc::api::disableScavenger):
|
||
* bmalloc/bmalloc.h:
|
||
(bmalloc::api::heapForKind):
|
||
(bmalloc::api::tryMalloc):
|
||
(bmalloc::api::malloc):
|
||
(bmalloc::api::tryMemalign):
|
||
(bmalloc::api::memalign):
|
||
(bmalloc::api::tryRealloc):
|
||
(bmalloc::api::realloc):
|
||
(bmalloc::api::free):
|
||
(bmalloc::api::scavengeThisThread): Deleted.
|
||
* libpas/.gitignore: Added.
|
||
* libpas/build.sh: Added.
|
||
* libpas/build_and_test.sh: Added.
|
||
* libpas/common.sh: Added.
|
||
* libpas/libpas.xcodeproj/project.pbxproj: Added.
|
||
* libpas/scripts/tally_verifier_output: Added.
|
||
* libpas/src/chaos/Chaos.cpp: Added.
|
||
(std::Object::Object):
|
||
(std::Packet::~Packet):
|
||
(std::Locker::Locker):
|
||
(std::Locker::~Locker):
|
||
(std::threadMain):
|
||
(main):
|
||
* libpas/src/libpas/bmalloc_heap.c: Added.
|
||
(bmalloc_try_allocate):
|
||
(bmalloc_try_allocate_with_alignment):
|
||
(bmalloc_try_allocate_zeroed):
|
||
(bmalloc_allocate):
|
||
(bmalloc_allocate_with_alignment):
|
||
(bmalloc_allocate_zeroed):
|
||
(bmalloc_try_reallocate):
|
||
(bmalloc_reallocate):
|
||
(bmalloc_try_iso_allocate):
|
||
(bmalloc_iso_allocate):
|
||
(bmalloc_heap_ref_get_heap):
|
||
(bmalloc_try_allocate_auxiliary):
|
||
(bmalloc_allocate_auxiliary):
|
||
(bmalloc_try_allocate_auxiliary_zeroed):
|
||
(bmalloc_allocate_auxiliary_zeroed):
|
||
(bmalloc_try_allocate_auxiliary_with_alignment):
|
||
(bmalloc_allocate_auxiliary_with_alignment):
|
||
(bmalloc_try_reallocate_auxiliary):
|
||
(bmalloc_reallocate_auxiliary):
|
||
(bmalloc_deallocate):
|
||
(bmalloc_force_auxiliary_heap_into_reserved_memory):
|
||
* libpas/src/libpas/bmalloc_heap.h: Added.
|
||
* libpas/src/libpas/bmalloc_heap_config.c: Copied from Source/WTF/wtf/FastTLS.h.
|
||
(bmalloc_heap_config_activate):
|
||
* libpas/src/libpas/bmalloc_heap_config.h: Added.
|
||
* libpas/src/libpas/bmalloc_heap_inlines.h: Added.
|
||
(bmalloc_try_allocate_inline):
|
||
(bmalloc_try_allocate_with_alignment_inline):
|
||
(bmalloc_try_allocate_zeroed_inline):
|
||
(bmalloc_allocate_inline):
|
||
(bmalloc_allocate_with_alignment_inline):
|
||
(bmalloc_allocate_zeroed_inline):
|
||
(bmalloc_try_reallocate_inline):
|
||
(bmalloc_reallocate_inline):
|
||
(bmalloc_try_iso_allocate_inline):
|
||
(bmalloc_iso_allocate_inline):
|
||
(bmalloc_try_allocate_auxiliary_inline):
|
||
(bmalloc_allocate_auxiliary_inline):
|
||
(bmalloc_try_allocate_auxiliary_zeroed_inline):
|
||
(bmalloc_allocate_auxiliary_zeroed_inline):
|
||
(bmalloc_try_allocate_auxiliary_with_alignment_inline):
|
||
(bmalloc_allocate_auxiliary_with_alignment_inline):
|
||
(bmalloc_try_reallocate_auxiliary_inline):
|
||
(bmalloc_reallocate_auxiliary_inline):
|
||
(bmalloc_deallocate_inline):
|
||
* libpas/src/libpas/bmalloc_heap_innards.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
* libpas/src/libpas/bmalloc_heap_ref.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
* libpas/src/libpas/hotbit_heap.c: Copied from Source/WTF/wtf/FastTLS.h.
|
||
(hotbit_try_allocate):
|
||
(hotbit_try_allocate_with_alignment):
|
||
(hotbit_try_reallocate):
|
||
(hotbit_deallocate):
|
||
* libpas/src/libpas/hotbit_heap.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
* libpas/src/libpas/hotbit_heap_config.c: Copied from Source/WTF/wtf/FastTLS.h.
|
||
(hotbit_heap_config_activate):
|
||
* libpas/src/libpas/hotbit_heap_config.h: Added.
|
||
* libpas/src/libpas/hotbit_heap_inlines.h: Added.
|
||
(hotbit_try_allocate_inline):
|
||
(hotbit_try_allocate_with_alignment_inline):
|
||
(hotbit_try_reallocate_inline):
|
||
(hotbit_deallocate_inline):
|
||
* libpas/src/libpas/hotbit_heap_innards.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
* libpas/src/libpas/iso_heap.c: Added.
|
||
(iso_try_allocate_common_primitive):
|
||
(iso_try_allocate_common_primitive_with_alignment):
|
||
(iso_try_allocate_common_primitive_zeroed):
|
||
(iso_allocate_common_primitive):
|
||
(iso_allocate_common_primitive_with_alignment):
|
||
(iso_allocate_common_primitive_zeroed):
|
||
(iso_try_reallocate_common_primitive):
|
||
(iso_reallocate_common_primitive):
|
||
(iso_try_allocate_dynamic_primitive):
|
||
(iso_try_allocate_dynamic_primitive_with_alignment):
|
||
(iso_try_allocate_dynamic_primitive_zeroed):
|
||
(iso_try_reallocate_dynamic_primitive):
|
||
(iso_heap_ref_construct):
|
||
(iso_try_allocate):
|
||
(iso_allocate):
|
||
(iso_try_allocate_array):
|
||
(iso_allocate_array):
|
||
(iso_try_allocate_array_zeroed):
|
||
(iso_allocate_array_zeroed):
|
||
(iso_try_reallocate_array):
|
||
(iso_reallocate_array):
|
||
(iso_heap_ref_get_heap):
|
||
(iso_primitive_heap_ref_construct):
|
||
(iso_try_allocate_primitive):
|
||
(iso_allocate_primitive):
|
||
(iso_try_allocate_primitive_zeroed):
|
||
(iso_allocate_primitive_zeroed):
|
||
(iso_try_allocate_primitive_with_alignment):
|
||
(iso_allocate_primitive_with_alignment):
|
||
(iso_try_reallocate_primitive):
|
||
(iso_reallocate_primitive):
|
||
(iso_try_allocate_for_objc):
|
||
(iso_has_object):
|
||
(iso_get_allocation_size):
|
||
(iso_get_heap):
|
||
(iso_deallocate):
|
||
(iso_force_primitive_heap_into_reserved_memory):
|
||
* libpas/src/libpas/iso_heap.h: Added.
|
||
* libpas/src/libpas/iso_heap_config.c: Copied from Source/WTF/wtf/FastTLS.h.
|
||
* libpas/src/libpas/iso_heap_config.h: Added.
|
||
* libpas/src/libpas/iso_heap_inlines.h: Added.
|
||
(iso_try_allocate_common_primitive_inline):
|
||
(iso_try_allocate_common_primitive_with_alignment_inline):
|
||
(iso_try_allocate_common_primitive_zeroed_inline):
|
||
(iso_allocate_common_primitive_inline):
|
||
(iso_allocate_common_primitive_with_alignment_inline):
|
||
(iso_allocate_common_primitive_zeroed_inline):
|
||
(iso_try_reallocate_common_primitive_inline):
|
||
(iso_reallocate_common_primitive_inline):
|
||
(iso_try_allocate_inline):
|
||
(iso_allocate_inline):
|
||
(iso_try_allocate_array_inline):
|
||
(iso_allocate_array_inline):
|
||
(iso_try_allocate_array_zeroed_inline):
|
||
(iso_allocate_array_zeroed_inline):
|
||
(iso_try_reallocate_array_inline):
|
||
(iso_reallocate_array_inline):
|
||
(iso_try_allocate_primitive_inline):
|
||
(iso_allocate_primitive_inline):
|
||
(iso_try_allocate_primitive_zeroed_inline):
|
||
(iso_allocate_primitive_zeroed_inline):
|
||
(iso_try_allocate_primitive_with_alignment_inline):
|
||
(iso_allocate_primitive_with_alignment_inline):
|
||
(iso_try_reallocate_primitive_inline):
|
||
(iso_reallocate_primitive_inline):
|
||
(iso_try_allocate_for_objc_inline):
|
||
(iso_has_object_inline):
|
||
(iso_get_allocation_size_inline):
|
||
(iso_get_heap_inline):
|
||
(iso_deallocate_inline):
|
||
* libpas/src/libpas/iso_heap_innards.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
* libpas/src/libpas/iso_heap_ref.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
* libpas/src/libpas/iso_test_heap.c: Added.
|
||
(iso_test_allocate_common_primitive):
|
||
(iso_test_allocate):
|
||
(iso_test_allocate_array):
|
||
(iso_test_deallocate):
|
||
(iso_test_heap_ref_get_heap):
|
||
* libpas/src/libpas/iso_test_heap.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
* libpas/src/libpas/iso_test_heap_config.c: Copied from Source/WTF/wtf/FastTLS.h.
|
||
* libpas/src/libpas/iso_test_heap_config.h: Added.
|
||
* libpas/src/libpas/jit_heap.c: Added.
|
||
(jit_heap_add_fresh_memory):
|
||
(jit_heap_try_allocate):
|
||
(jit_heap_shrink):
|
||
(jit_heap_get_size):
|
||
(jit_heap_deallocate):
|
||
* libpas/src/libpas/jit_heap.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
* libpas/src/libpas/jit_heap_config.c: Added.
|
||
(fresh_memory_aligned_allocator):
|
||
(initialize_fresh_memory_config):
|
||
(allocate_from_fresh):
|
||
(page_provider):
|
||
(jit_page_header_for_boundary_remote):
|
||
(jit_small_bitfit_allocate_page):
|
||
(jit_small_bitfit_create_page_header):
|
||
(jit_small_bitfit_destroy_page_header):
|
||
(jit_medium_bitfit_allocate_page):
|
||
(jit_medium_bitfit_create_page_header):
|
||
(jit_medium_bitfit_destroy_page_header):
|
||
(jit_aligned_allocator):
|
||
(jit_prepare_to_enumerate):
|
||
(jit_heap_config_for_each_shared_page_directory):
|
||
(jit_heap_config_for_each_shared_page_directory_remote):
|
||
(jit_heap_config_add_fresh_memory):
|
||
* libpas/src/libpas/jit_heap_config.h: Added.
|
||
(jit_type_size):
|
||
(jit_type_alignment):
|
||
(jit_heap_config_fast_megapage_kind):
|
||
(jit_small_bitfit_page_header_for_boundary):
|
||
(jit_small_bitfit_boundary_for_page_header):
|
||
(jit_medium_bitfit_page_header_for_boundary):
|
||
(jit_medium_bitfit_boundary_for_page_header):
|
||
(jit_heap_config_page_header):
|
||
* libpas/src/libpas/jit_heap_config_root_data.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
* libpas/src/libpas/minalign32_heap.c: Added.
|
||
(minalign32_allocate_common_primitive):
|
||
(minalign32_allocate):
|
||
(minalign32_allocate_array):
|
||
(minalign32_deallocate):
|
||
(minalign32_heap_ref_get_heap):
|
||
* libpas/src/libpas/minalign32_heap.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
* libpas/src/libpas/minalign32_heap_config.c: Copied from Source/WTF/wtf/FastTLS.h.
|
||
(minalign32_heap_config_activate):
|
||
* libpas/src/libpas/minalign32_heap_config.h: Added.
|
||
* libpas/src/libpas/pagesize64k_heap.c: Added.
|
||
(pagesize64k_allocate_common_primitive):
|
||
(pagesize64k_allocate):
|
||
(pagesize64k_allocate_array):
|
||
(pagesize64k_deallocate):
|
||
(pagesize64k_heap_ref_get_heap):
|
||
* libpas/src/libpas/pagesize64k_heap.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
* libpas/src/libpas/pagesize64k_heap_config.c: Copied from Source/WTF/wtf/FastTLS.h.
|
||
(pagesize64k_heap_config_activate):
|
||
* libpas/src/libpas/pagesize64k_heap_config.h: Added.
|
||
* libpas/src/libpas/pas_aligned_allocation_result.h: Added.
|
||
(pas_aligned_allocation_result_create_empty):
|
||
(pas_aligned_allocation_result_as_allocation_result):
|
||
* libpas/src/libpas/pas_aligned_allocator.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
* libpas/src/libpas/pas_alignment.c: Copied from Source/WTF/wtf/FastTLS.h.
|
||
(pas_alignment_dump):
|
||
* libpas/src/libpas/pas_alignment.h: Added.
|
||
(pas_alignment_create):
|
||
(pas_alignment_create_traditional):
|
||
(pas_alignment_create_trivial):
|
||
(pas_alignment_validate):
|
||
(pas_alignment_is_ptr_aligned):
|
||
(pas_alignment_round_up):
|
||
(pas_alignment_is_equal):
|
||
* libpas/src/libpas/pas_all_biasing_directories.c: Added.
|
||
(pas_all_biasing_directories_append):
|
||
(pas_all_biasing_directories_activate):
|
||
(scavenge_for_each_set_bit_bits_source):
|
||
(scavenge_for_each_set_bit_callback):
|
||
(scavenge_bitvector_word_callback):
|
||
(pas_all_biasing_directories_scavenge):
|
||
* libpas/src/libpas/pas_all_biasing_directories.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
* libpas/src/libpas/pas_all_heap_configs.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
* libpas/src/libpas/pas_all_heaps.c: Added.
|
||
(pas_all_heaps_add_heap):
|
||
(pas_all_heaps_for_each_static_heap):
|
||
(pas_all_heaps_for_each_dynamic_heap):
|
||
(pas_all_heaps_for_each_heap):
|
||
(pas_all_heaps_for_each_static_segregated_heap_not_part_of_a_heap):
|
||
(for_each_segregated_heap_callback):
|
||
(pas_all_heaps_for_each_static_segregated_heap):
|
||
(pas_all_heaps_for_each_segregated_heap):
|
||
(get_num_free_bytes_for_each_heap_callback):
|
||
(pas_all_heaps_get_num_free_bytes):
|
||
(reset_heap_ref_for_each_heap_callback):
|
||
(pas_all_heaps_reset_heap_ref):
|
||
(for_each_segregated_directory_global_size_directory_callback):
|
||
(for_each_segregated_directory_shared_page_directory_callback):
|
||
(for_each_segregated_directory_segregated_heap_callback):
|
||
(pas_all_heaps_for_each_segregated_directory):
|
||
(dump_directory_nicely):
|
||
(dump_view_nicely):
|
||
(verify_in_steady_state_segregated_directory_callback):
|
||
(pas_all_heaps_verify_in_steady_state):
|
||
(compute_total_non_utility_segregated_summary_directory_callback):
|
||
(pas_all_heaps_compute_total_non_utility_segregated_summary):
|
||
(compute_total_non_utility_bitfit_summary_heap_callback):
|
||
(pas_all_heaps_compute_total_non_utility_bitfit_summary):
|
||
(compute_total_non_utility_large_summary_heap_callback):
|
||
(pas_all_heaps_compute_total_non_utility_large_summary):
|
||
(pas_all_heaps_compute_total_non_utility_summary):
|
||
* libpas/src/libpas/pas_all_heaps.h: Added.
|
||
* libpas/src/libpas/pas_all_magazines.c: Added.
|
||
(pas_all_magazines_get_current):
|
||
* libpas/src/libpas/pas_all_magazines.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
* libpas/src/libpas/pas_all_shared_page_directories.c: Copied from Source/WTF/wtf/FastTLS.h.
|
||
(pas_all_shared_page_directories_add):
|
||
(pas_all_shared_page_directories_for_each):
|
||
* libpas/src/libpas/pas_all_shared_page_directories.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
* libpas/src/libpas/pas_allocation_callbacks.c: Copied from Source/WTF/wtf/FastTLS.h.
|
||
* libpas/src/libpas/pas_allocation_callbacks.h: Added.
|
||
(pas_did_allocate):
|
||
(pas_will_deallocate):
|
||
* libpas/src/libpas/pas_allocation_config.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
* libpas/src/libpas/pas_allocation_kind.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
(pas_allocation_kind_get_string):
|
||
* libpas/src/libpas/pas_allocation_result.c: Copied from Source/WTF/wtf/FastTLS.h.
|
||
(pas_allocation_result_zero):
|
||
* libpas/src/libpas/pas_allocation_result.h: Added.
|
||
(pas_allocation_result_create_failure):
|
||
(pas_allocation_result_create_success_with_zero_mode):
|
||
(pas_allocation_result_create_success):
|
||
* libpas/src/libpas/pas_allocator_counts.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
* libpas/src/libpas/pas_allocator_index.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
* libpas/src/libpas/pas_allocator_scavenge_action.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
(pas_allocator_scavenge_action_get_string):
|
||
* libpas/src/libpas/pas_baseline_allocator.c: Copied from Source/WTF/wtf/FastTLS.h.
|
||
(pas_baseline_allocator_attach_directory):
|
||
(pas_baseline_allocator_detach_directory):
|
||
* libpas/src/libpas/pas_baseline_allocator.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
* libpas/src/libpas/pas_baseline_allocator_result.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
(pas_baseline_allocator_result_create_failure):
|
||
(pas_baseline_allocator_result_create_success):
|
||
* libpas/src/libpas/pas_baseline_allocator_table.c: Added.
|
||
(initialize):
|
||
(pas_baseline_allocator_table_initialize_if_necessary):
|
||
(pas_baseline_allocator_table_get_random_index):
|
||
(pas_baseline_allocator_table_for_all):
|
||
* libpas/src/libpas/pas_baseline_allocator_table.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
* libpas/src/libpas/pas_basic_heap_config_enumerator_data.c: Added.
|
||
(pas_basic_heap_config_enumerator_data_add_page_header_table):
|
||
* libpas/src/libpas/pas_basic_heap_config_enumerator_data.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
* libpas/src/libpas/pas_basic_heap_config_root_data.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
* libpas/src/libpas/pas_basic_heap_page_caches.h: Added.
|
||
(pas_basic_heap_page_caches_get_shared_page_directories):
|
||
* libpas/src/libpas/pas_basic_heap_runtime_config.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
* libpas/src/libpas/pas_basic_segregated_page_caches.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
* libpas/src/libpas/pas_biasing_directory.c: Added.
|
||
(pas_biasing_directory_construct):
|
||
(pas_biasing_directory_get_sharing_pool):
|
||
(pas_biasing_directory_did_create_delta):
|
||
(pas_biasing_directory_take_last_unused):
|
||
(pas_biasing_directory_did_use_index_slow):
|
||
(pas_biasing_directory_index_did_become_eligible):
|
||
* libpas/src/libpas/pas_biasing_directory.h: Added.
|
||
(pas_is_segregated_biasing_directory):
|
||
(pas_is_bitfit_biasing_directory):
|
||
(pas_biasing_directory_magazine_index):
|
||
(pas_biasing_directory_ownership_threshold):
|
||
(pas_biasing_directory_unused_span_size):
|
||
(pas_biasing_directory_did_use_index):
|
||
* libpas/src/libpas/pas_biasing_directory_inlines.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
(pas_unwrap_segregated_biasing_directory):
|
||
(pas_unwrap_bitfit_biasing_directory):
|
||
* libpas/src/libpas/pas_biasing_directory_kind.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
(pas_biasing_directory_kind_get_string):
|
||
* libpas/src/libpas/pas_bitfield_vector.h: Added.
|
||
(pas_bitfield_vector_get):
|
||
(pas_bitfield_vector_set):
|
||
* libpas/src/libpas/pas_bitfit_allocation_result.h: Added.
|
||
(pas_bitfit_allocation_result_create_success):
|
||
(pas_bitfit_allocation_result_create_failure):
|
||
(pas_bitfit_allocation_result_create_empty):
|
||
(pas_bitfit_allocation_result_create_need_to_lock_commit_lock):
|
||
* libpas/src/libpas/pas_bitfit_allocator.c: Added.
|
||
(pas_bitfit_allocator_commit_view):
|
||
(pas_bitfit_allocator_finish_failing):
|
||
* libpas/src/libpas/pas_bitfit_allocator.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
* libpas/src/libpas/pas_bitfit_allocator_inlines.h: Added.
|
||
(pas_bitfit_allocator_reset):
|
||
(pas_bitfit_allocator_assert_reset):
|
||
(pas_bitfit_allocator_try_allocate):
|
||
* libpas/src/libpas/pas_bitfit_biasing_directory.c: Added.
|
||
(pas_bitfit_biasing_directory_create):
|
||
(pas_bitfit_biasing_directory_take_last_unused):
|
||
* libpas/src/libpas/pas_bitfit_biasing_directory.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
* libpas/src/libpas/pas_bitfit_directory.c: Added.
|
||
(pas_bitfit_directory_construct):
|
||
(pas_bitfit_directory_update_biasing_eligibility):
|
||
(pas_bitfit_directory_max_free_did_become_unprocessed):
|
||
(pas_bitfit_directory_max_free_did_become_unprocessed_unchecked):
|
||
(pas_bitfit_directory_max_free_did_become_empty_without_biasing_update):
|
||
(pas_bitfit_directory_max_free_did_become_empty):
|
||
(pas_bitfit_directory_get_first_free_view):
|
||
(pas_bitfit_directory_compute_summary):
|
||
(for_each_live_object_callback):
|
||
(pas_bitfit_directory_for_each_live_object):
|
||
* libpas/src/libpas/pas_bitfit_directory.h: Added.
|
||
(pas_bitfit_directory_size):
|
||
(pas_bitfit_directory_get_max_free_ptr):
|
||
(pas_bitfit_directory_get_max_free):
|
||
(pas_bitfit_directory_set_max_free_unchecked):
|
||
(pas_bitfit_directory_set_max_free_not_empty_impl):
|
||
(pas_bitfit_directory_set_processed_max_free):
|
||
(pas_bitfit_directory_set_unprocessed_max_free_unchecked):
|
||
(pas_bitfit_directory_set_empty_max_free):
|
||
(pas_bitfit_directory_set_unprocessed_max_free):
|
||
(pas_bitfit_directory_get_view_ptr):
|
||
(pas_bitfit_directory_get_view):
|
||
* libpas/src/libpas/pas_bitfit_directory_and_index.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
(pas_bitfit_directory_and_index_create):
|
||
* libpas/src/libpas/pas_bitfit_directory_inlines.h: Added.
|
||
(pas_bitfit_directory_get_global):
|
||
(pas_bitfit_directory_find_first_free_for_num_bits_iterate_callback):
|
||
(pas_bitfit_directory_find_first_free_for_num_bits):
|
||
(pas_bitfit_directory_find_first_free):
|
||
(pas_bitfit_directory_find_first_empty_iterate_callback):
|
||
(pas_bitfit_directory_find_first_empty):
|
||
* libpas/src/libpas/pas_bitfit_directory_kind.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
(pas_bitfit_directory_kind_get_string):
|
||
* libpas/src/libpas/pas_bitfit_global_directory.c: Added.
|
||
(pas_bitfit_global_directory_construct):
|
||
(pas_bitfit_global_directory_does_sharing):
|
||
(pas_bitfit_global_directory_get_use_epoch):
|
||
(pas_bitfit_global_directory_get_empty_bit_at_index):
|
||
(pas_bitfit_global_directory_set_empty_bit_at_index):
|
||
(pas_bitfit_global_directory_view_did_become_empty_at_index):
|
||
(pas_bitfit_global_directory_view_did_become_empty):
|
||
(pas_bitfit_global_directory_take_last_empty):
|
||
(pas_bitfit_global_directory_dump_reference):
|
||
(pas_bitfit_global_directory_dump_for_spectrum):
|
||
* libpas/src/libpas/pas_bitfit_global_directory.h: Added.
|
||
* libpas/src/libpas/pas_bitfit_global_size_class.c: Added.
|
||
(pas_bitfit_global_size_class_create):
|
||
(pas_bitfit_global_size_class_select_for_magazine):
|
||
* libpas/src/libpas/pas_bitfit_global_size_class.h: Added.
|
||
* libpas/src/libpas/pas_bitfit_heap.c: Added.
|
||
(pas_bitfit_heap_create):
|
||
(pas_bitfit_heap_select_variant):
|
||
(pas_bitfit_heap_ensure_global_size_class):
|
||
(pas_bitfit_heap_compute_summary):
|
||
(for_each_live_object_callback):
|
||
(pas_bitfit_heap_for_each_live_object):
|
||
* libpas/src/libpas/pas_bitfit_heap.h: Added.
|
||
(pas_bitfit_heap_get_directory):
|
||
* libpas/src/libpas/pas_bitfit_max_free.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
* libpas/src/libpas/pas_bitfit_page.c: Added.
|
||
(pas_bitfit_page_construct):
|
||
(pas_bitfit_page_get_config):
|
||
(pas_bitfit_page_log_bits):
|
||
(pas_bitfit_page_deallocation_did_fail):
|
||
(pas_bitfit_page_for_each_live_object):
|
||
(verify_for_each_object_callback):
|
||
(pas_bitfit_page_verify):
|
||
* libpas/src/libpas/pas_bitfit_page.h: Added.
|
||
(pas_bitfit_page_header_size):
|
||
(pas_bitfit_page_free_bits):
|
||
(pas_bitfit_page_object_end_bits):
|
||
(pas_bitfit_page_get_granule_use_counts):
|
||
(pas_bitfit_page_offset_to_first_object):
|
||
(pas_bitfit_page_offset_to_end_of_last_object):
|
||
(pas_bitfit_page_payload_size):
|
||
(pas_bitfit_page_for_boundary):
|
||
(pas_bitfit_page_for_boundary_or_null):
|
||
(pas_bitfit_page_for_boundary_unchecked):
|
||
(pas_bitfit_page_boundary):
|
||
(pas_bitfit_page_boundary_or_null):
|
||
(pas_bitfit_page_for_address_and_page_config):
|
||
(pas_bitfit_page_testing_verify):
|
||
* libpas/src/libpas/pas_bitfit_page_config.h: Added.
|
||
(pas_bitfit_page_config_is_enabled):
|
||
(pas_bitfit_page_config_num_alloc_bits):
|
||
(pas_bitfit_page_config_num_alloc_words):
|
||
(pas_bitfit_page_config_num_alloc_words64):
|
||
(pas_bitfit_page_config_num_alloc_bit_bytes):
|
||
(pas_bitfit_page_config_byte_offset_for_object_bits):
|
||
(pas_bitfit_page_config_uses_subpages):
|
||
* libpas/src/libpas/pas_bitfit_page_config_inlines.h: Added.
|
||
* libpas/src/libpas/pas_bitfit_page_config_kind.c: Added.
|
||
(pas_bitfit_page_config_kind_for_each):
|
||
* libpas/src/libpas/pas_bitfit_page_config_kind.def: Added.
|
||
* libpas/src/libpas/pas_bitfit_page_config_kind.h: Added.
|
||
(pas_bitfit_page_config_kind_get_string):
|
||
(pas_bitfit_page_config_kind_get_config):
|
||
* libpas/src/libpas/pas_bitfit_page_config_utils.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
* libpas/src/libpas/pas_bitfit_page_config_utils_inlines.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
* libpas/src/libpas/pas_bitfit_page_config_variant.h: Added.
|
||
(pas_bitfit_page_config_variant_get_string):
|
||
(pas_bitfit_page_config_variant_get_capitalized_string):
|
||
* libpas/src/libpas/pas_bitfit_page_inlines.h: Added.
|
||
(pas_bitfit_page_compute_offset):
|
||
(pas_bitfit_page_allocation_satisfies_alignment):
|
||
(pas_bitfit_page_allocation_commit_granules_or_reloop):
|
||
(pas_bitfit_page_finish_allocation):
|
||
(pas_bitfit_page_allocate):
|
||
(pas_bitfit_page_deallocate_with_page_impl_mode_get_string):
|
||
(pas_bitfit_page_deallocate_with_page_impl):
|
||
(pas_bitfit_page_deallocate_with_page):
|
||
(pas_bitfit_page_deallocate):
|
||
(pas_bitfit_page_get_allocation_size_with_page):
|
||
(pas_bitfit_page_get_allocation_size):
|
||
(pas_bitfit_page_shrink_with_page):
|
||
(pas_bitfit_page_shrink):
|
||
* libpas/src/libpas/pas_bitfit_size_class.c: Added.
|
||
(pas_bitfit_size_class_find_insertion_point):
|
||
(pas_bitfit_size_class_construct):
|
||
(pas_bitfit_size_class_create):
|
||
(pas_bitfit_size_class_get_first_free_view):
|
||
* libpas/src/libpas/pas_bitfit_size_class.h: Added.
|
||
* libpas/src/libpas/pas_bitfit_view.c: Added.
|
||
(pas_bitfit_view_create):
|
||
(pas_bitfit_view_lock_ownership_lock_slow):
|
||
(pas_bitfit_view_note_nonemptiness):
|
||
(did_become_empty_for_bits):
|
||
(pas_bitfit_view_note_full_emptiness):
|
||
(pas_bitfit_view_note_partial_emptiness):
|
||
(pas_bitfit_view_note_max_free):
|
||
(compute_summary):
|
||
(pas_bitfit_view_compute_summary):
|
||
(for_each_live_object_callback):
|
||
(for_each_live_object):
|
||
(pas_bitfit_view_for_each_live_object):
|
||
* libpas/src/libpas/pas_bitfit_view.h: Added.
|
||
(pas_bitfit_view_lock_ownership_lock):
|
||
* libpas/src/libpas/pas_bitfit_view_and_index.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
(pas_bitfit_view_and_index_create):
|
||
(pas_bitfit_view_and_index_create_empty):
|
||
* libpas/src/libpas/pas_bitfit_view_inlines.h: Added.
|
||
(pas_bitfit_view_current_directory_and_index):
|
||
(pas_bitfit_view_current_directory):
|
||
(pas_bitfit_view_index_in_current):
|
||
(pas_bitfit_view_is_empty):
|
||
* libpas/src/libpas/pas_bitvector.h: Added.
|
||
(pas_bitvector_get_from_word):
|
||
(pas_bitvector_get):
|
||
(pas_bitvector_get_from_one_word):
|
||
(pas_backward_bitvector_get):
|
||
(pas_bitvector_set_in_word):
|
||
(pas_bitvector_set):
|
||
(pas_bitvector_set_in_one_word):
|
||
(pas_bitvector_set_atomic_in_word):
|
||
(pas_bitvector_set_atomic):
|
||
(pas_backward_bitvector_set):
|
||
(pas_bitvector_find_first_set):
|
||
(pas_bitvector_for_each_set_bit):
|
||
(pas_bitvector64_set_range):
|
||
* libpas/src/libpas/pas_bootstrap_free_heap.c: Copied from Source/WTF/wtf/FastTLS.h.
|
||
(bootstrap_source_allocate_aligned):
|
||
(initialize_config):
|
||
* libpas/src/libpas/pas_bootstrap_free_heap.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
* libpas/src/libpas/pas_bootstrap_heap_page_provider.c: Copied from Source/WTF/wtf/FastTLS.h.
|
||
(pas_bootstrap_heap_page_provider):
|
||
* libpas/src/libpas/pas_bootstrap_heap_page_provider.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
* libpas/src/libpas/pas_cares_about_size_mode.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
* libpas/src/libpas/pas_cartesian_tree.h: Added.
|
||
(pas_cartesian_tree_node_child_ptr):
|
||
(pas_cartesian_tree_node_is_null_constrained):
|
||
(pas_cartesian_tree_node_minimum):
|
||
(pas_cartesian_tree_node_minimum_constrained):
|
||
(pas_cartesian_tree_node_maximum):
|
||
(pas_cartesian_tree_node_maximum_constrained):
|
||
(pas_cartesian_tree_node_successor):
|
||
(pas_cartesian_tree_node_successor_constrained):
|
||
(pas_cartesian_tree_node_predecessor):
|
||
(pas_cartesian_tree_node_predecessor_constrained):
|
||
(pas_cartesian_tree_node_reset):
|
||
(pas_cartesian_tree_construct):
|
||
(pas_cartesian_tree_node_find_exact):
|
||
(pas_cartesian_tree_find_exact):
|
||
(pas_cartesian_tree_node_find_least_greater_than_or_equal):
|
||
(pas_cartesian_tree_find_least_greater_than_or_equal):
|
||
(pas_cartesian_tree_node_find_least_greater_than):
|
||
(pas_cartesian_tree_find_least_greater_than):
|
||
(pas_cartesian_tree_node_find_greatest_less_than_or_equal):
|
||
(pas_cartesian_tree_find_greatest_less_than_or_equal):
|
||
(pas_cartesian_tree_node_find_greatest_less_than):
|
||
(pas_cartesian_tree_find_greatest_less_than):
|
||
(pas_cartesian_tree_minimum):
|
||
(pas_cartesian_tree_minimum_constrained):
|
||
(pas_cartesian_tree_maximum):
|
||
(pas_cartesian_tree_maximum_constrained):
|
||
(pas_cartesian_tree_is_empty):
|
||
(pas_cartesian_tree_insert):
|
||
(pas_cartesian_tree_remove):
|
||
(pas_cartesian_tree_validate_recurse):
|
||
(pas_cartesian_tree_validate):
|
||
(pas_cartesian_tree_size):
|
||
* libpas/src/libpas/pas_coalign.c: Added.
|
||
(formal_mod):
|
||
(ceiling_div):
|
||
(pas_coalign_one_sided):
|
||
(pas_coalign):
|
||
* libpas/src/libpas/pas_coalign.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
(pas_coalign_empty_result):
|
||
(pas_coalign_result_create):
|
||
* libpas/src/libpas/pas_commit_mode.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
(pas_commit_mode_get_string):
|
||
* libpas/src/libpas/pas_commit_span.c: Added.
|
||
(pas_commit_span_construct):
|
||
(pas_commit_span_add_to_change):
|
||
(pas_commit_span_add_unchanged):
|
||
(commit):
|
||
(pas_commit_span_add_unchanged_and_commit):
|
||
(decommit):
|
||
(pas_commit_span_add_unchanged_and_decommit):
|
||
* libpas/src/libpas/pas_commit_span.h: Added.
|
||
* libpas/src/libpas/pas_compact_atomic_allocator_index_ptr.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
* libpas/src/libpas/pas_compact_atomic_biasing_directory_ptr.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
* libpas/src/libpas/pas_compact_atomic_bitfit_biasing_directory_ptr.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
* libpas/src/libpas/pas_compact_atomic_bitfit_global_size_class_ptr.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
* libpas/src/libpas/pas_compact_atomic_bitfit_heap_ptr.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
* libpas/src/libpas/pas_compact_atomic_bitfit_size_class_ptr.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
* libpas/src/libpas/pas_compact_atomic_bitfit_view_ptr.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
* libpas/src/libpas/pas_compact_atomic_enumerable_range_list_chunk_ptr.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
* libpas/src/libpas/pas_compact_atomic_page_sharing_pool_ptr.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
* libpas/src/libpas/pas_compact_atomic_ptr.h: Added.
|
||
* libpas/src/libpas/pas_compact_atomic_segregated_exclusive_view_ptr.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
* libpas/src/libpas/pas_compact_atomic_segregated_global_size_directory_ptr.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
* libpas/src/libpas/pas_compact_atomic_segregated_heap_page_sharing_pools_ptr.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
* libpas/src/libpas/pas_compact_atomic_segregated_partial_view_ptr.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
* libpas/src/libpas/pas_compact_atomic_segregated_view.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
* libpas/src/libpas/pas_compact_atomic_thread_local_cache_layout_node.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
* libpas/src/libpas/pas_compact_atomic_unsigned_ptr.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
* libpas/src/libpas/pas_compact_biasing_directory_ptr.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
* libpas/src/libpas/pas_compact_bitfit_directory_ptr.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
* libpas/src/libpas/pas_compact_bitfit_global_directory_ptr.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
* libpas/src/libpas/pas_compact_bitfit_view_ptr.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
* libpas/src/libpas/pas_compact_bootstrap_free_heap.c: Copied from Source/WTF/wtf/FastTLS.h.
|
||
(compact_bootstrap_source_allocate_aligned):
|
||
(initialize_config):
|
||
* libpas/src/libpas/pas_compact_bootstrap_free_heap.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
* libpas/src/libpas/pas_compact_cartesian_tree_node_ptr.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
* libpas/src/libpas/pas_compact_heap_ptr.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
* libpas/src/libpas/pas_compact_heap_reservation.c: Added.
|
||
(pas_compact_heap_reservation_try_allocate):
|
||
* libpas/src/libpas/pas_compact_heap_reservation.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
* libpas/src/libpas/pas_compact_large_utility_free_heap.c: Copied from Source/WTF/wtf/FastTLS.h.
|
||
* libpas/src/libpas/pas_compact_large_utility_free_heap.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
* libpas/src/libpas/pas_compact_page_granule_use_count_ptr.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
* libpas/src/libpas/pas_compact_ptr.h: Added.
|
||
* libpas/src/libpas/pas_compact_segregated_biasing_directory_ptr.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
* libpas/src/libpas/pas_compact_segregated_global_size_directory_ptr.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
* libpas/src/libpas/pas_compact_segregated_heap_ptr.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
* libpas/src/libpas/pas_compact_segregated_shared_page_directory_ptr.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
* libpas/src/libpas/pas_compact_segregated_shared_view_ptr.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
* libpas/src/libpas/pas_compact_segregated_view.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
* libpas/src/libpas/pas_compact_skip_list_node_ptr.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
* libpas/src/libpas/pas_compact_skip_list_node_ptr_ptr.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
* libpas/src/libpas/pas_compact_subpage_map_entry_ptr.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
* libpas/src/libpas/pas_compact_tagged_atomic_ptr.h: Added.
|
||
* libpas/src/libpas/pas_compact_tagged_page_granule_use_count_ptr.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
* libpas/src/libpas/pas_compact_tagged_ptr.h: Added.
|
||
* libpas/src/libpas/pas_compact_tagged_unsigned_ptr.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
* libpas/src/libpas/pas_compact_unsigned_ptr.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
* libpas/src/libpas/pas_compute_summary_object_callbacks.c: Added.
|
||
(pas_compute_summary_live_object_callback):
|
||
(pas_compute_summary_live_object_callback_without_physical_sharing):
|
||
(config):
|
||
(pas_compute_summary_dead_object_callback):
|
||
(pas_compute_summary_dead_object_callback_without_physical_sharing):
|
||
* libpas/src/libpas/pas_compute_summary_object_callbacks.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
* libpas/src/libpas/pas_config.h: Added.
|
||
* libpas/src/libpas/pas_config_prefix.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
* libpas/src/libpas/pas_count_lookup_mode.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
(pas_count_lookup_mode_get_string):
|
||
* libpas/src/libpas/pas_create_basic_heap_page_caches_with_reserved_memory.c: Added.
|
||
(pas_create_basic_heap_page_caches_with_reserved_memory):
|
||
* libpas/src/libpas/pas_create_basic_heap_page_caches_with_reserved_memory.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
* libpas/src/libpas/pas_deallocate.c: Added.
|
||
(pas_try_deallocate_known_large):
|
||
(pas_deallocate_known_large):
|
||
(pas_try_deallocate_slow):
|
||
(deallocate_segregated):
|
||
(pas_try_deallocate_slow_no_cache):
|
||
* libpas/src/libpas/pas_deallocate.h: Added.
|
||
(pas_deallocate_known_segregated_impl):
|
||
(pas_deallocate_known_segregated):
|
||
(pas_try_deallocate_not_small):
|
||
(pas_try_deallocate_impl):
|
||
(pas_try_deallocate):
|
||
(pas_deallocate):
|
||
* libpas/src/libpas/pas_deallocation_mode.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
(pas_deallocation_mode_get_string):
|
||
* libpas/src/libpas/pas_deallocator.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
* libpas/src/libpas/pas_deallocator_scavenge_action.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
(pas_deallocator_scavenge_action_get_string):
|
||
* libpas/src/libpas/pas_debug_heap.h: Added.
|
||
(pas_debug_heap_is_enabled):
|
||
(pas_debug_heap_malloc):
|
||
(pas_debug_heap_memalign):
|
||
(pas_debug_heap_realloc):
|
||
(pas_debug_heap_free):
|
||
* libpas/src/libpas/pas_debug_spectrum.c: Added.
|
||
(pas_debug_spectrum_add):
|
||
(pas_debug_spectrum_dump):
|
||
(pas_debug_spectrum_reset):
|
||
* libpas/src/libpas/pas_debug_spectrum.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
* libpas/src/libpas/pas_deferred_decommit_log.c: Added.
|
||
(pas_deferred_decommit_log_construct):
|
||
(pas_deferred_decommit_log_destruct):
|
||
(already_holds_lock):
|
||
(pas_deferred_decommit_log_lock_for_adding):
|
||
(pas_deferred_decommit_log_add):
|
||
(pas_deferred_decommit_log_add_already_locked):
|
||
(pas_deferred_decommit_log_add_maybe_locked):
|
||
(pas_deferred_decommit_log_unlock_after_aborted_add):
|
||
(decommit_all):
|
||
(pas_deferred_decommit_log_decommit_all):
|
||
(pas_deferred_decommit_log_pretend_to_decommit_all):
|
||
* libpas/src/libpas/pas_deferred_decommit_log.h: Added.
|
||
* libpas/src/libpas/pas_designated_intrinsic_heap.c: Added.
|
||
(set_up_range):
|
||
(pas_designated_intrinsic_heap_initialize):
|
||
* libpas/src/libpas/pas_designated_intrinsic_heap.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
* libpas/src/libpas/pas_designated_intrinsic_heap_inlines.h: Added.
|
||
(pas_designated_intrinsic_heap_num_allocator_indices):
|
||
(pas_designated_index_result_create_failure):
|
||
(pas_designated_index_result_create_success):
|
||
(pas_designated_intrinsic_heap_num_designated_indices):
|
||
(pas_designated_intrinsic_heap_designated_index):
|
||
* libpas/src/libpas/pas_dyld_state.c: Added.
|
||
(pas_dyld_is_libsystem_initialized):
|
||
* libpas/src/libpas/pas_dyld_state.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
* libpas/src/libpas/pas_dynamic_primitive_heap_map.c: Added.
|
||
(pas_dynamic_primitive_heap_map_find_slow):
|
||
* libpas/src/libpas/pas_dynamic_primitive_heap_map.h: Added.
|
||
(pas_dynamic_primitive_heap_map_heaps_for_size_table_entry_create_empty):
|
||
(pas_dynamic_primitive_heap_map_heaps_for_size_table_entry_create_deleted):
|
||
(pas_dynamic_primitive_heap_map_heaps_for_size_table_entry_is_empty_or_deleted):
|
||
(pas_dynamic_primitive_heap_map_heaps_for_size_table_entry_is_empty):
|
||
(pas_dynamic_primitive_heap_map_heaps_for_size_table_entry_is_deleted):
|
||
(pas_dynamic_primitive_heap_map_heaps_for_size_table_entry_get_key):
|
||
(pas_dynamic_primitive_heap_map_heaps_for_size_table_key_get_hash):
|
||
(pas_dynamic_primitive_heap_map_heaps_for_size_table_key_is_equal):
|
||
(pas_dynamic_primitive_heap_map_hash):
|
||
(pas_dynamic_primitive_heap_map_find):
|
||
* libpas/src/libpas/pas_ensure_heap_forced_into_reserved_memory.c: Copied from Source/WTF/wtf/FastTLS.h.
|
||
(pas_ensure_heap_forced_into_reserved_memory):
|
||
* libpas/src/libpas/pas_ensure_heap_forced_into_reserved_memory.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
* libpas/src/libpas/pas_ensure_heap_with_page_caches.c: Copied from Source/WTF/wtf/FastTLS.h.
|
||
(pas_ensure_heap_with_page_caches):
|
||
* libpas/src/libpas/pas_ensure_heap_with_page_caches.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
* libpas/src/libpas/pas_enumerable_page_malloc.c: Copied from Source/WTF/wtf/FastTLS.h.
|
||
(pas_enumerable_page_malloc_try_allocate_without_deallocating_padding):
|
||
* libpas/src/libpas/pas_enumerable_page_malloc.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
* libpas/src/libpas/pas_enumerable_range_list.c: Added.
|
||
(pas_enumerable_range_list_append):
|
||
(pas_enumerable_range_list_iterate):
|
||
(pas_enumerable_range_list_iterate_remote):
|
||
* libpas/src/libpas/pas_enumerable_range_list.h: Added.
|
||
* libpas/src/libpas/pas_enumerate_bitfit_heaps.c: Added.
|
||
(view_callback):
|
||
(enumerate_bitfit_directory):
|
||
(enumerate_bitfit_heap_callback):
|
||
(pas_enumerate_bitfit_heaps):
|
||
* libpas/src/libpas/pas_enumerate_bitfit_heaps.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
* libpas/src/libpas/pas_enumerate_initially_unaccounted_pages.c: Added.
|
||
(range_list_iterate_add_unaccounted_callback):
|
||
(range_list_iterate_exclude_accounted_callback):
|
||
(pas_enumerate_initially_unaccounted_pages):
|
||
* libpas/src/libpas/pas_enumerate_initially_unaccounted_pages.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
* libpas/src/libpas/pas_enumerate_large_heaps.c: Added.
|
||
(range_list_iterate_add_large_payload_callback):
|
||
(record_span):
|
||
(large_map_hashtable_entry_callback):
|
||
(small_large_map_hashtable_entry_callback):
|
||
(tiny_large_map_second_level_hashtable_entry_callback):
|
||
(tiny_large_map_hashtable_entry_callback):
|
||
(pas_enumerate_large_heaps):
|
||
* libpas/src/libpas/pas_enumerate_large_heaps.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
* libpas/src/libpas/pas_enumerate_segregated_heaps.c: Added.
|
||
(local_allocator_map_entry_create_empty):
|
||
(local_allocator_map_entry_create_deleted):
|
||
(local_allocator_map_entry_is_empty_or_deleted):
|
||
(local_allocator_map_entry_is_empty):
|
||
(local_allocator_map_entry_is_deleted):
|
||
(local_allocator_map_entry_get_key):
|
||
(local_allocator_map_key_get_hash):
|
||
(local_allocator_map_key_is_equal):
|
||
(for_each_view):
|
||
(collect_shared_page_directories_shared_page_directory_callback):
|
||
(collect_shared_page_directories_heap_callback):
|
||
(record_page_payload_and_meta):
|
||
(record_page_objects):
|
||
(enumerate_exclusive_view):
|
||
(enumerate_shared_view):
|
||
(enumerate_partial_view):
|
||
(shared_page_directory_view_callback):
|
||
(size_directory_view_callback):
|
||
(enumerate_segregated_heap_callback):
|
||
(consider_allocator):
|
||
(pas_enumerate_segregated_heaps):
|
||
* libpas/src/libpas/pas_enumerate_segregated_heaps.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
* libpas/src/libpas/pas_enumerate_unaccounted_pages_as_meta.c: Added.
|
||
(pas_enumerate_unaccounted_pages_as_meta):
|
||
* libpas/src/libpas/pas_enumerate_unaccounted_pages_as_meta.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
* libpas/src/libpas/pas_enumerator.c: Added.
|
||
(allocate):
|
||
(deallocate):
|
||
(pas_enumerator_create):
|
||
(pas_enumerator_destroy):
|
||
(pas_enumerator_allocate):
|
||
(pas_enumerator_read_compact):
|
||
(pas_enumerator_read):
|
||
(pas_enumerator_add_unaccounted_pages):
|
||
(pas_enumerator_exclude_accounted_page):
|
||
(pas_enumerator_exclude_accounted_pages):
|
||
(pas_enumerator_record):
|
||
(record_payload_span):
|
||
(pas_enumerator_record_page_payload_and_meta):
|
||
(pas_enumerator_for_each_heap):
|
||
(pas_enumerator_enumerate_all):
|
||
* libpas/src/libpas/pas_enumerator.h: Added.
|
||
(pas_enumerator_record_kind_get_string):
|
||
* libpas/src/libpas/pas_enumerator_internal.h: Added.
|
||
* libpas/src/libpas/pas_enumerator_region.c: Added.
|
||
(pas_enumerator_region_allocate):
|
||
(pas_enumerator_region_destroy):
|
||
* libpas/src/libpas/pas_enumerator_region.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
* libpas/src/libpas/pas_epoch.c: Copied from Source/WTF/wtf/FastTLS.h.
|
||
(pas_get_epoch):
|
||
* libpas/src/libpas/pas_epoch.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
* libpas/src/libpas/pas_exclusive_view_template_memo_table.c: Copied from Source/WTF/wtf/FastTLS.h.
|
||
* libpas/src/libpas/pas_exclusive_view_template_memo_table.h: Added.
|
||
(pas_exclusive_view_template_memo_key_create):
|
||
(pas_exclusive_view_template_memo_entry_create_empty):
|
||
(pas_exclusive_view_template_memo_entry_create_deleted):
|
||
(pas_exclusive_view_template_memo_entry_is_empty_or_deleted):
|
||
(pas_exclusive_view_template_memo_entry_is_empty):
|
||
(pas_exclusive_view_template_memo_entry_is_deleted):
|
||
(pas_exclusive_view_template_memo_entry_get_key):
|
||
(pas_exclusive_view_template_memo_key_get_hash):
|
||
(pas_exclusive_view_template_memo_key_is_equal):
|
||
* libpas/src/libpas/pas_extended_gcd.c: Added.
|
||
(pas_extended_gcd):
|
||
* libpas/src/libpas/pas_extended_gcd.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
* libpas/src/libpas/pas_fast_large_free_heap.c: Added.
|
||
(key_compare_callback):
|
||
(get_x_key_callback):
|
||
(get_y_key_callback):
|
||
(initialize_cartesian_config):
|
||
(pas_fast_large_free_heap_construct):
|
||
(insert_node):
|
||
(remove_node):
|
||
(dump_heap):
|
||
(fast_find_first):
|
||
(fast_find_by_end):
|
||
(fast_read_cursor):
|
||
(fast_write_cursor):
|
||
(fast_merge):
|
||
(fast_remove):
|
||
(fast_append):
|
||
(fast_commit):
|
||
(fast_dump):
|
||
(fast_add_mapped_bytes):
|
||
(initialize_generic_heap_config):
|
||
(pas_fast_large_free_heap_try_allocate):
|
||
(pas_fast_large_free_heap_deallocate):
|
||
(pas_fast_large_free_heap_for_each_free):
|
||
(pas_fast_large_free_heap_get_num_free_bytes):
|
||
(pas_fast_large_free_heap_validate):
|
||
(pas_fast_large_free_heap_dump_to_printf):
|
||
* libpas/src/libpas/pas_fast_large_free_heap.h: Added.
|
||
(pas_fast_large_free_heap_get_num_mapped_bytes):
|
||
* libpas/src/libpas/pas_fast_megapage_cache.c: Added.
|
||
(table_set_by_index):
|
||
(pas_fast_megapage_cache_try_allocate):
|
||
* libpas/src/libpas/pas_fast_megapage_cache.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
* libpas/src/libpas/pas_fast_megapage_kind.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
(pas_fast_megapage_kind_get_string):
|
||
* libpas/src/libpas/pas_fast_megapage_table.c: Added.
|
||
(pas_fast_megapage_table_initialize_static):
|
||
(pas_fast_megapage_table_set_by_index):
|
||
* libpas/src/libpas/pas_fast_megapage_table.h: Added.
|
||
(pas_fast_megapage_table_get_by_index):
|
||
(pas_fast_megapage_table_get):
|
||
* libpas/src/libpas/pas_fast_path_allocation_result.h: Added.
|
||
(pas_fast_path_allocation_result_create):
|
||
(pas_fast_path_allocation_result_create_need_slow):
|
||
(pas_fast_path_allocation_result_create_out_of_memory):
|
||
(pas_fast_path_allocation_result_create_success):
|
||
(pas_fast_path_allocation_result_from_allocation_result):
|
||
(pas_fast_path_allocation_result_to_allocation_result):
|
||
* libpas/src/libpas/pas_fast_path_allocation_result_kind.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
(pas_fast_path_allocation_result_kind_get_string):
|
||
* libpas/src/libpas/pas_fast_tls.h: Added.
|
||
* libpas/src/libpas/pas_fd_stream.c: Copied from Source/WTF/wtf/FastTLS.h.
|
||
(fd_stream_vprintf):
|
||
(pas_fd_stream_vprintf):
|
||
* libpas/src/libpas/pas_fd_stream.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
* libpas/src/libpas/pas_first_level_tiny_large_map_entry.h: Added.
|
||
(pas_first_level_tiny_large_map_entry_create_empty):
|
||
(pas_first_level_tiny_large_map_entry_create_deleted):
|
||
(pas_first_level_tiny_large_map_entry_is_empty_or_deleted):
|
||
(pas_first_level_tiny_large_map_entry_is_empty):
|
||
(pas_first_level_tiny_large_map_entry_is_deleted):
|
||
(pas_first_level_tiny_large_map_entry_get_key):
|
||
(pas_first_level_tiny_large_map_key_get_hash):
|
||
(pas_first_level_tiny_large_map_key_is_equal):
|
||
* libpas/src/libpas/pas_found_bit_index.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
(pas_found_bit_index_create_empty):
|
||
(pas_found_bit_index_create):
|
||
* libpas/src/libpas/pas_found_index.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
(pas_found_index_create_empty):
|
||
(pas_found_index_create):
|
||
(pas_found_index_create_unsuccessful):
|
||
* libpas/src/libpas/pas_free_granules.c: Added.
|
||
(pas_free_granules_compute_and_mark_decommitted):
|
||
(pas_free_granules_unmark_decommitted):
|
||
(pas_free_granules_decommit_after_locking_range):
|
||
* libpas/src/libpas/pas_free_granules.h: Added.
|
||
(pas_free_granules_is_free):
|
||
* libpas/src/libpas/pas_free_mode.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
* libpas/src/libpas/pas_free_range_kind.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
(pas_free_range_kind_get_string):
|
||
* libpas/src/libpas/pas_full_alloc_bits.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
(pas_full_alloc_bits_create_empty):
|
||
(pas_full_alloc_bits_create):
|
||
* libpas/src/libpas/pas_full_alloc_bits_inlines.h: Added.
|
||
(pas_full_alloc_bits_create_for_exclusive):
|
||
(pas_full_alloc_bits_create_for_partial):
|
||
(pas_full_alloc_bits_create_for_view_and_directory):
|
||
(pas_full_alloc_bits_create_for_view):
|
||
* libpas/src/libpas/pas_generic_large_free_heap.h: Added.
|
||
(pas_generic_large_free_heap_merge_physical):
|
||
(pas_generic_large_free_heap_try_allocate_test_allocation_candidate):
|
||
(pas_generic_large_free_heap_try_allocate):
|
||
* libpas/src/libpas/pas_get_allocation_size.h: Added.
|
||
(pas_get_allocation_size):
|
||
* libpas/src/libpas/pas_get_heap.h: Added.
|
||
(pas_get_heap_known_segregated):
|
||
(pas_get_heap):
|
||
* libpas/src/libpas/pas_get_object_kind.h: Added.
|
||
(pas_get_object_kind):
|
||
* libpas/src/libpas/pas_get_page_base.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
(pas_get_page_base):
|
||
* libpas/src/libpas/pas_has_object.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
(pas_has_object):
|
||
* libpas/src/libpas/pas_hashtable.h: Added.
|
||
* libpas/src/libpas/pas_heap.c: Added.
|
||
(pas_heap_create):
|
||
(pas_heap_get_type_size):
|
||
(pas_heap_get_num_free_bytes):
|
||
(for_each_live_object_small_object_callback):
|
||
(for_each_live_object_large_object_callback):
|
||
(pas_heap_for_each_live_object):
|
||
(pas_heap_compute_summary):
|
||
(pas_heap_reset_heap_ref):
|
||
(pas_heap_ensure_size_directory_for_count_slow):
|
||
* libpas/src/libpas/pas_heap.h: Added.
|
||
(pas_heap_for_large_heap):
|
||
(pas_heap_for_segregated_heap):
|
||
* libpas/src/libpas/pas_heap_config.c: Copied from Source/WTF/wtf/FastTLS.h.
|
||
(pas_heap_config_activate):
|
||
* libpas/src/libpas/pas_heap_config.h: Added.
|
||
(pas_heap_config_segregated_page_config_ptr_for_variant):
|
||
(pas_heap_config_bitfit_page_config_ptr_for_variant):
|
||
(pas_heap_config_segregated_page_config_for_variant):
|
||
(pas_heap_config_bitfit_page_config_for_variant):
|
||
* libpas/src/libpas/pas_heap_config_inlines.h: Added.
|
||
* libpas/src/libpas/pas_heap_config_kind.c: Added.
|
||
(pas_heap_config_kind_for_each):
|
||
* libpas/src/libpas/pas_heap_config_kind.def: Added.
|
||
* libpas/src/libpas/pas_heap_config_kind.h: Added.
|
||
(pas_heap_config_kind_get_string):
|
||
(pas_heap_config_kind_get_config):
|
||
(pas_heap_config_kind_is_active):
|
||
* libpas/src/libpas/pas_heap_config_utils.c: Added.
|
||
(pas_heap_config_utils_null_activate):
|
||
(pas_heap_config_utils_for_each_shared_page_directory):
|
||
(pas_heap_config_utils_for_each_shared_page_directory_remote):
|
||
(pas_heap_config_utils_allocate_aligned):
|
||
(pas_heap_config_utils_prepare_to_enumerate):
|
||
* libpas/src/libpas/pas_heap_config_utils.h: Added.
|
||
* libpas/src/libpas/pas_heap_config_utils_inlines.h: Added.
|
||
* libpas/src/libpas/pas_heap_for_config.c: Added.
|
||
(pas_heap_for_config_allocate):
|
||
(pas_heap_for_page_config_kind_allocate):
|
||
(pas_heap_for_page_config_allocate):
|
||
(pas_heap_for_config_allocate_with_alignment):
|
||
(pas_heap_for_page_config_allocate_with_alignment):
|
||
(pas_heap_for_config_allocate_with_manual_alignment):
|
||
(pas_heap_for_page_config_kind_allocate_with_manual_alignment):
|
||
(pas_heap_for_page_config_allocate_with_manual_alignment):
|
||
(pas_heap_for_config_deallocate):
|
||
(pas_heap_for_page_config_kind_deallocate):
|
||
(pas_heap_for_page_config_deallocate):
|
||
* libpas/src/libpas/pas_heap_for_config.h: Added.
|
||
* libpas/src/libpas/pas_heap_inlines.h: Added.
|
||
(pas_heap_ensure_size_directory_for_count):
|
||
* libpas/src/libpas/pas_heap_kind.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
(pas_heap_kind_get_string):
|
||
* libpas/src/libpas/pas_heap_lock.c: Copied from Source/WTF/wtf/FastTLS.h.
|
||
* libpas/src/libpas/pas_heap_lock.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
* libpas/src/libpas/pas_heap_page_provider.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
* libpas/src/libpas/pas_heap_ref.c: Copied from Source/WTF/wtf/FastTLS.h.
|
||
(pas_ensure_heap_slow):
|
||
* libpas/src/libpas/pas_heap_ref.h: Added.
|
||
(pas_ensure_heap):
|
||
* libpas/src/libpas/pas_heap_ref_kind.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
(pas_heap_ref_kind_get_string):
|
||
* libpas/src/libpas/pas_heap_ref_prefix.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
* libpas/src/libpas/pas_heap_runtime_config.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
* libpas/src/libpas/pas_heap_summary.c: Added.
|
||
(pas_heap_summary_validate):
|
||
(pas_heap_summary_dump):
|
||
* libpas/src/libpas/pas_heap_summary.h: Added.
|
||
(pas_heap_summary_create_empty):
|
||
(pas_heap_summary_is_empty):
|
||
(pas_heap_summary_add):
|
||
(pas_heap_summary_committed_objects):
|
||
(pas_heap_summary_total):
|
||
(pas_heap_summary_fragmentation):
|
||
* libpas/src/libpas/pas_heap_table.c: Copied from Source/WTF/wtf/FastTLS.h.
|
||
(pas_heap_table_try_allocate_index):
|
||
* libpas/src/libpas/pas_heap_table.h: Added.
|
||
(pas_heap_table_has_index):
|
||
(pas_heap_table_get_index):
|
||
* libpas/src/libpas/pas_heap_table_state.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
(pas_heap_table_state_get_string):
|
||
* libpas/src/libpas/pas_immortal_heap.c: Added.
|
||
(bump_is_ok):
|
||
(pas_immortal_heap_allocate_with_manual_alignment):
|
||
(pas_immortal_heap_allocate_with_alignment):
|
||
(pas_immortal_heap_allocate):
|
||
(pas_immortal_heap_hold_lock_and_allocate):
|
||
(pas_immortal_heap_allocate_with_heap_lock_hold_mode):
|
||
(pas_immortal_heap_allocate_with_alignment_and_heap_lock_hold_mode):
|
||
* libpas/src/libpas/pas_immortal_heap.h: Added.
|
||
* libpas/src/libpas/pas_immutable_vector.h: Added.
|
||
* libpas/src/libpas/pas_internal_config.h: Added.
|
||
* libpas/src/libpas/pas_intrinsic_allocation_result.h: Added.
|
||
(pas_intrinsic_allocation_result_create_empty):
|
||
(pas_intrinsic_allocation_result_create):
|
||
(pas_intrinsic_allocation_result_identity):
|
||
(pas_intrinsic_allocation_result_zero):
|
||
(pas_intrinsic_allocation_result_set_errno):
|
||
(pas_intrinsic_allocation_result_crash_on_error):
|
||
* libpas/src/libpas/pas_intrinsic_heap_support.h: Added.
|
||
* libpas/src/libpas/pas_large_free.h: Added.
|
||
(pas_large_free_create_empty):
|
||
(pas_large_free_is_empty):
|
||
(pas_large_free_size):
|
||
(pas_large_free_offset_in_type_at_end):
|
||
* libpas/src/libpas/pas_large_free_heap_config.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
* libpas/src/libpas/pas_large_free_heap_declarations.def: Copied from Source/WTF/wtf/FastTLS.h.
|
||
* libpas/src/libpas/pas_large_free_heap_deferred_commit_log.c: Added.
|
||
(pas_large_free_heap_deferred_commit_log_construct):
|
||
(pas_large_free_heap_deferred_commit_log_destruct):
|
||
(pas_large_free_heap_deferred_commit_log_add):
|
||
(dump_large_commit):
|
||
(commit):
|
||
(commit_all):
|
||
(pas_large_free_heap_deferred_commit_log_commit_all):
|
||
(pas_large_free_heap_deferred_commit_log_pretend_to_commit_all):
|
||
* libpas/src/libpas/pas_large_free_heap_deferred_commit_log.h: Added.
|
||
* libpas/src/libpas/pas_large_free_heap_definitions.def: Added.
|
||
* libpas/src/libpas/pas_large_free_heap_helpers.c: Added.
|
||
(large_utility_aligned_allocator):
|
||
(initialize_config):
|
||
(pas_large_free_heap_helpers_try_allocate_with_alignment):
|
||
(pas_large_free_heap_helpers_deallocate):
|
||
(pas_large_free_heap_helpers_compute_summary):
|
||
* libpas/src/libpas/pas_large_free_heap_helpers.h: Added.
|
||
* libpas/src/libpas/pas_large_free_inlines.h: Added.
|
||
(pas_large_free_create_merged):
|
||
(pas_large_free_split):
|
||
(pas_large_free_allocation_candidate):
|
||
(pas_large_free_usable_space):
|
||
(pas_large_allocation_result_create_empty):
|
||
(pas_large_free_allocate):
|
||
(pas_large_free_create_merged_for_sure):
|
||
(pas_large_free_can_merge):
|
||
(pas_large_allocation_result_as_allocation_result):
|
||
* libpas/src/libpas/pas_large_free_visitor.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
* libpas/src/libpas/pas_large_heap.c: Added.
|
||
(pas_large_heap_construct):
|
||
(aligned_allocator):
|
||
(initialize_config):
|
||
(pas_large_heap_try_allocate):
|
||
(pas_large_heap_try_deallocate):
|
||
(pas_large_heap_try_shrink):
|
||
(pas_large_heap_shove_into_free):
|
||
(for_each_live_object_entry_callback):
|
||
(pas_large_heap_for_each_live_object):
|
||
(pas_large_heap_for_object):
|
||
(pas_large_heap_get_num_free_bytes):
|
||
(compute_summary_live_object_callback):
|
||
(pas_large_heap_compute_summary):
|
||
* libpas/src/libpas/pas_large_heap.h: Added.
|
||
* libpas/src/libpas/pas_large_heap_physical_page_sharing_cache.c: Added.
|
||
(large_aligned_allocator):
|
||
(pas_large_heap_physical_page_sharing_cache_construct):
|
||
(pas_large_heap_physical_page_sharing_cache_try_allocate_with_alignment):
|
||
* libpas/src/libpas/pas_large_heap_physical_page_sharing_cache.h: Added.
|
||
* libpas/src/libpas/pas_large_map.c: Added.
|
||
(pas_large_map_find):
|
||
(pas_large_map_add):
|
||
(pas_large_map_take):
|
||
(pas_large_map_for_each_entry):
|
||
* libpas/src/libpas/pas_large_map.h: Added.
|
||
* libpas/src/libpas/pas_large_map_entry.h: Added.
|
||
(pas_large_map_entry_create_empty):
|
||
(pas_large_map_entry_create_deleted):
|
||
(pas_large_map_entry_is_empty_or_deleted):
|
||
(pas_large_map_entry_is_empty):
|
||
(pas_large_map_entry_is_deleted):
|
||
(pas_large_map_entry_get_key):
|
||
(pas_large_map_key_get_hash):
|
||
(pas_large_map_key_is_equal):
|
||
* libpas/src/libpas/pas_large_sharing_pool.c: Added.
|
||
(node_compare_callback):
|
||
(inner_key_compare_callback):
|
||
(update_min_epoch):
|
||
(validate_min_heap):
|
||
(validate_node):
|
||
(validate_node_if_asserting_aggressively):
|
||
(create_node):
|
||
(create_and_insert):
|
||
(boot_tree):
|
||
(destroy_node):
|
||
(remove_from_min_heap):
|
||
(remove_and_destroy):
|
||
(predecessor):
|
||
(successor):
|
||
(states_match):
|
||
(is_eligible):
|
||
(belongs_in_min_heap):
|
||
(update_min_heap):
|
||
(split_node_and_get_right_impl):
|
||
(split_node_and_get_right):
|
||
(split_node_and_get_left):
|
||
(merge_if_possible):
|
||
(node_containing):
|
||
(min_node_for_range):
|
||
(max_node_for_range):
|
||
(splat_live_bytes):
|
||
(should_do_commit_stuff_to):
|
||
(splat_command_get_string):
|
||
(splat_command_get_free_mode):
|
||
(dump_large_commit):
|
||
(try_splat_impl):
|
||
(try_splat):
|
||
(splat):
|
||
(pas_large_sharing_pool_boot_free):
|
||
(pas_large_sharing_pool_free):
|
||
(pas_large_sharing_pool_allocate_and_commit):
|
||
(pas_large_sharing_pool_decommit_least_recently_used):
|
||
(pas_large_sharing_pool_validate):
|
||
(pas_large_sharing_pool_compute_summary):
|
||
(pas_large_sharing_pool_for_each):
|
||
* libpas/src/libpas/pas_large_sharing_pool.h: Added.
|
||
(pas_large_sharing_node_heap_compare):
|
||
(pas_large_sharing_node_heap_get_index):
|
||
(pas_large_sharing_node_heap_set_index):
|
||
* libpas/src/libpas/pas_large_sharing_pool_epoch_update_mode.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
(pas_large_sharing_pool_epoch_update_mode_get_string):
|
||
* libpas/src/libpas/pas_large_utility_free_heap.c: Copied from Source/WTF/wtf/FastTLS.h.
|
||
* libpas/src/libpas/pas_large_utility_free_heap.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
* libpas/src/libpas/pas_line_word_config.h: Added.
|
||
(pas_line_word_config_count_low_zeroes_8_bit):
|
||
(pas_line_word_config_count_high_zeroes_8_bit):
|
||
(pas_line_word_config_count_low_zeroes_16_bit):
|
||
(pas_line_word_config_count_high_zeroes_16_bit):
|
||
(pas_line_word_config_count_low_zeroes_32_bit):
|
||
(pas_line_word_config_count_high_zeroes_32_bit):
|
||
(pas_line_word_config_count_low_zeroes_64_bit):
|
||
(pas_line_word_config_count_high_zeroes_64_bit):
|
||
(pas_line_word_config_construct):
|
||
* libpas/src/libpas/pas_list_direction.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
(pas_list_direction_get_string):
|
||
(pas_list_direction_invert):
|
||
* libpas/src/libpas/pas_local_allocator.c: Added.
|
||
(pas_local_allocator_construct):
|
||
(pas_local_allocator_destruct):
|
||
(pas_local_allocator_reset):
|
||
(pas_local_allocator_move):
|
||
(pas_local_allocator_refill_with_bitfit):
|
||
(pas_local_allocator_finish_refill_with_bitfit):
|
||
(stop_impl):
|
||
(pas_local_allocator_stop):
|
||
(pas_local_allocator_scavenge):
|
||
* libpas/src/libpas/pas_local_allocator.h: Added.
|
||
(pas_local_allocator_is_null):
|
||
(pas_local_allocator_is_active):
|
||
(pas_local_allocator_has_bitfit):
|
||
(pas_local_allocator_get_bitfit):
|
||
(pas_local_allocator_page_boundary):
|
||
(pas_local_allocator_alignment):
|
||
* libpas/src/libpas/pas_local_allocator_config_kind.h: Added.
|
||
(pas_local_allocator_config_kind_is_primordial_partial):
|
||
(pas_local_allocator_config_kind_is_bitfit):
|
||
(pas_local_allocator_config_kind_create_normal):
|
||
(pas_local_allocator_config_kind_create_primordial_partial):
|
||
(pas_local_allocator_config_kind_create_bitfit):
|
||
(pas_local_allocator_config_kind_get_segregated_page_config_kind):
|
||
(pas_local_allocator_config_kind_get_bitfit_page_config_kind):
|
||
(pas_local_allocator_config_kind_get_string):
|
||
* libpas/src/libpas/pas_local_allocator_inlines.h: Added.
|
||
(pas_local_allocator_reset_impl):
|
||
(pas_local_allocator_set_up_bump):
|
||
(pas_local_allocator_scan_bits_to_set_up_use_counts_bits_source):
|
||
(pas_local_allocator_scan_bits_to_set_up_use_counts_bit_callback):
|
||
(pas_local_allocator_scan_bits_to_set_up_free_bits):
|
||
(pas_local_allocator_set_up_free_bits):
|
||
(pas_local_allocator_make_bump):
|
||
(pas_local_allocator_prepare_to_allocate):
|
||
(pas_local_allocator_set_up_primordial_bump):
|
||
(pas_local_allocator_start_allocating_in_primordial_partial_view):
|
||
(pas_local_allocator_bless_primordial_partial_view_before_stopping):
|
||
(pas_local_allocator_try_allocate_in_primordial_partial_view):
|
||
(pas_local_allocator_refill_with_known_config):
|
||
(pas_local_allocator_return_memory_to_page_bits_source):
|
||
(pas_local_allocator_return_memory_to_page_set_bit_callback):
|
||
(pas_local_allocator_return_memory_to_page):
|
||
(pas_local_allocator_try_allocate_with_free_bits):
|
||
(pas_local_allocator_try_allocate_inline_cases):
|
||
(pas_local_allocator_try_allocate_small_segregated_slow_impl):
|
||
(pas_local_allocator_try_allocate_small_segregated_slow):
|
||
(pas_local_allocator_try_allocate_out_of_line_cases):
|
||
(pas_local_allocator_try_allocate_slow_impl):
|
||
(pas_local_allocator_try_allocate_slow):
|
||
(pas_local_allocator_try_allocate):
|
||
* libpas/src/libpas/pas_local_allocator_line.h: Added.
|
||
(pas_local_allocator_line_fits_without_shift):
|
||
(pas_local_allocator_line_decode):
|
||
(pas_local_allocator_line_encode):
|
||
(pas_local_allocator_line_remaining):
|
||
(pas_local_allocator_line_set_remaining):
|
||
(pas_local_allocator_line_payload_end_delta):
|
||
(pas_local_allocator_line_set_payload_end_delta):
|
||
* libpas/src/libpas/pas_local_allocator_refill_mode.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
* libpas/src/libpas/pas_local_allocator_result.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
(pas_local_allocator_result_create_failure):
|
||
(pas_local_allocator_result_create_success):
|
||
* libpas/src/libpas/pas_lock.c: Copied from Source/WTF/wtf/FastTLS.h.
|
||
(pas_lock_lock_slow):
|
||
* libpas/src/libpas/pas_lock.h: Added.
|
||
(pas_lock_construct):
|
||
(pas_lock_construct_disabled):
|
||
(pas_lock_lock):
|
||
(pas_lock_try_lock):
|
||
(pas_lock_unlock):
|
||
(pas_lock_assert_held):
|
||
(pas_lock_testing_assert_held):
|
||
(pas_lock_lock_with_mode):
|
||
(pas_lock_switch_with_mode):
|
||
(pas_lock_switch):
|
||
(pas_lock_lock_conditionally):
|
||
(pas_lock_unlock_conditionally):
|
||
(pas_lock_for_switch_conditionally):
|
||
(pas_lock_switch_conditionally):
|
||
* libpas/src/libpas/pas_lock_free_read_hashtable.h: Added.
|
||
(if):
|
||
* libpas/src/libpas/pas_lock_free_read_ptr_ptr_hashtable.c: Added.
|
||
(pas_lock_free_read_ptr_ptr_hashtable_set):
|
||
* libpas/src/libpas/pas_lock_free_read_ptr_ptr_hashtable.h: Added.
|
||
(pas_lock_free_read_ptr_ptr_hashtable_find):
|
||
(pas_lock_free_read_ptr_ptr_hashtable_size):
|
||
* libpas/src/libpas/pas_log.c: Added.
|
||
(pas_vlog_fd):
|
||
(pas_log_fd):
|
||
(pas_vlog):
|
||
(pas_log):
|
||
(pas_start_crash_logging):
|
||
* libpas/src/libpas/pas_log.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
* libpas/src/libpas/pas_magazine.c: Copied from Source/WTF/wtf/FastTLS.h.
|
||
(pas_magazine_create):
|
||
* libpas/src/libpas/pas_magazine.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
* libpas/src/libpas/pas_medium_megapage_cache.c: Copied from Source/WTF/wtf/FastTLS.h.
|
||
(pas_medium_megapage_cache_try_allocate):
|
||
* libpas/src/libpas/pas_medium_megapage_cache.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
* libpas/src/libpas/pas_megapage_cache.c: Added.
|
||
(megapage_cache_allocate_aligned):
|
||
(pas_megapage_cache_construct):
|
||
(pas_megapage_cache_try_allocate):
|
||
* libpas/src/libpas/pas_megapage_cache.h: Added.
|
||
* libpas/src/libpas/pas_min_heap.h: Added.
|
||
* libpas/src/libpas/pas_monotonic_time.c: Copied from Source/WTF/wtf/FastTLS.h.
|
||
(get_timebase_info):
|
||
(pas_get_current_monotonic_time_nanoseconds):
|
||
* libpas/src/libpas/pas_monotonic_time.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
* libpas/src/libpas/pas_mutation_count.h: Added.
|
||
(pas_mutation_count_start_mutating):
|
||
(pas_mutation_count_stop_mutating):
|
||
(pas_mutation_count_is_mutating):
|
||
(pas_mutation_count_matches):
|
||
(pas_mutation_count_depend):
|
||
* libpas/src/libpas/pas_object_kind.h: Added.
|
||
(pas_object_kind_get_string):
|
||
(pas_object_kind_for_segregated_variant):
|
||
(pas_object_kind_for_bitfit_variant):
|
||
* libpas/src/libpas/pas_object_size_mode.h: Added.
|
||
(pas_object_size_mode_get_string):
|
||
* libpas/src/libpas/pas_page_base.c: Added.
|
||
(pas_page_base_get_config):
|
||
(pas_page_base_get_granule_use_counts):
|
||
(pas_page_base_compute_committed_when_owned):
|
||
(pas_page_base_is_empty):
|
||
(pas_page_base_add_free_range):
|
||
* libpas/src/libpas/pas_page_base.h: Added.
|
||
(pas_page_base_construct):
|
||
(pas_page_base_get_kind):
|
||
(pas_page_base_get_config_kind):
|
||
(pas_page_base_is_segregated):
|
||
(pas_page_base_get_segregated):
|
||
(pas_page_base_is_bitfit):
|
||
(pas_page_base_get_bitfit):
|
||
(pas_page_base_index_of_object_at_offset_from_page_boundary):
|
||
(pas_page_base_object_offset_from_page_boundary_at_index):
|
||
(pas_page_base_boundary):
|
||
(pas_page_base_boundary_or_null):
|
||
(pas_page_base_for_boundary):
|
||
(pas_page_base_for_boundary_or_null):
|
||
(pas_page_base_boundary_for_address_and_page_config):
|
||
(pas_page_base_for_address_and_page_config):
|
||
* libpas/src/libpas/pas_page_base_config.h: Added.
|
||
(pas_page_base_config_min_align):
|
||
(pas_page_base_config_object_payload_end_offset_from_boundary):
|
||
(pas_page_base_config_num_granule_bytes):
|
||
(pas_page_base_config_get_config_kind):
|
||
(pas_page_base_config_is_segregated):
|
||
(pas_page_base_config_is_bitfit):
|
||
(pas_page_base_config_get_segregated):
|
||
(pas_page_base_config_get_bitfit):
|
||
* libpas/src/libpas/pas_page_base_config_inlines.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
(pas_page_base_config_is_utility):
|
||
* libpas/src/libpas/pas_page_base_config_utils.h: Added.
|
||
* libpas/src/libpas/pas_page_base_config_utils_inlines.h: Added.
|
||
* libpas/src/libpas/pas_page_base_inlines.h: Added.
|
||
(pas_page_base_free_granule_uses_in_range_action):
|
||
(pas_page_base_free_granule_uses_in_range):
|
||
* libpas/src/libpas/pas_page_config_kind.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
(pas_page_config_kind_get_string):
|
||
* libpas/src/libpas/pas_page_granule_use_count.h: Added.
|
||
(pas_page_granule_get_indices):
|
||
(pas_page_granule_for_each_use_in_range):
|
||
(pas_page_granule_use_count_increment):
|
||
(pas_page_granule_increment_uses_for_range):
|
||
* libpas/src/libpas/pas_page_header_placement_mode.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
(pas_page_header_placement_mode_get_string):
|
||
* libpas/src/libpas/pas_page_header_table.c: Added.
|
||
(pas_page_header_table_add):
|
||
(pas_page_header_table_remove):
|
||
* libpas/src/libpas/pas_page_header_table.h: Added.
|
||
(pas_page_header_table_hash):
|
||
(pas_page_header_table_get_boundary_ptr):
|
||
(pas_page_header_table_get_boundary):
|
||
(pas_page_header_table_get_for_boundary):
|
||
(pas_page_header_table_get_for_address):
|
||
* libpas/src/libpas/pas_page_kind.h: Added.
|
||
(pas_page_kind_get_string):
|
||
(pas_page_kind_get_config_kind):
|
||
(pas_page_kind_get_segregated_variant):
|
||
(pas_page_kind_get_bitfit_variant):
|
||
* libpas/src/libpas/pas_page_malloc.c: Added.
|
||
(pas_page_malloc_alignment_slow):
|
||
(pas_page_malloc_alignment_shift_slow):
|
||
(pas_page_malloc_try_allocate_without_deallocating_padding):
|
||
(pas_page_malloc_commit):
|
||
(pas_page_malloc_decommit):
|
||
(pas_page_malloc_deallocate):
|
||
* libpas/src/libpas/pas_page_malloc.h: Added.
|
||
(pas_page_malloc_alignment):
|
||
(pas_page_malloc_alignment_shift):
|
||
* libpas/src/libpas/pas_page_sharing_mode.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
(pas_page_sharing_mode_does_sharing):
|
||
(pas_page_sharing_mode_get_string):
|
||
* libpas/src/libpas/pas_page_sharing_participant.c: Added.
|
||
(pas_page_sharing_participant_create):
|
||
(pas_page_sharing_participant_get_payload):
|
||
(pas_page_sharing_participant_payload_construct):
|
||
(pas_page_sharing_participant_payload_with_use_epoch_construct):
|
||
(pas_page_sharing_participant_get_use_epoch):
|
||
(pas_page_sharing_participant_set_parent_pool):
|
||
(pas_page_sharing_participant_get_parent_pool):
|
||
(pas_page_sharing_participant_is_eligible):
|
||
(pas_page_sharing_participant_take_least_recently_used):
|
||
* libpas/src/libpas/pas_page_sharing_participant.h: Added.
|
||
(pas_page_sharing_participant_get_ptr):
|
||
(pas_page_sharing_participant_get_kind):
|
||
* libpas/src/libpas/pas_page_sharing_participant_kind.h: Added.
|
||
(pas_page_sharing_participant_kind_select_for_segregated_directory):
|
||
(pas_page_sharing_participant_kind_get_string):
|
||
* libpas/src/libpas/pas_page_sharing_pool.c: Added.
|
||
(verify_participants):
|
||
(verify_min_heap):
|
||
(dump_min_heap):
|
||
(pas_page_sharing_pool_construct):
|
||
(pas_page_sharing_pool_add_at_index):
|
||
(pas_page_sharing_pool_add):
|
||
(get_current_participant):
|
||
(take_from):
|
||
(pas_page_sharing_pool_take_least_recently_used):
|
||
(atomic_add_balance):
|
||
(pas_physical_page_sharing_pool_take):
|
||
(pas_physical_page_sharing_pool_scavenge):
|
||
(pas_physical_page_sharing_pool_take_later):
|
||
(pas_physical_page_sharing_pool_give_back):
|
||
(pas_physical_page_sharing_pool_take_for_page_config):
|
||
(pas_bias_page_sharing_pool_take):
|
||
(pas_page_sharing_pool_did_create_delta):
|
||
(pas_page_sharing_pool_verify):
|
||
(pas_page_sharing_pool_has_delta):
|
||
(pas_page_sharing_pool_has_current_participant):
|
||
* libpas/src/libpas/pas_page_sharing_pool.h: Added.
|
||
(pas_page_sharing_participant_compare):
|
||
(pas_page_sharing_participant_get_index):
|
||
(pas_page_sharing_participant_set_index):
|
||
(pas_page_sharing_pool_num_participants):
|
||
(pas_page_sharing_pool_get_participant):
|
||
* libpas/src/libpas/pas_page_sharing_pool_take_result.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
(pas_page_sharing_pool_take_result_get_string):
|
||
* libpas/src/libpas/pas_payload_reservation_page_list.c: Copied from Source/WTF/wtf/FastTLS.h.
|
||
(pas_payload_reservation_page_list_append):
|
||
* libpas/src/libpas/pas_payload_reservation_page_list.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
* libpas/src/libpas/pas_physical_memory_synchronization_style.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
(pas_physical_memory_synchronization_style_get_string):
|
||
* libpas/src/libpas/pas_physical_memory_transaction.c: Added.
|
||
(pas_physical_memory_transaction_construct):
|
||
(pas_physical_memory_transaction_begin):
|
||
(pas_physical_memory_transaction_end):
|
||
(pas_physical_memory_transaction_did_fail_to_acquire_lock):
|
||
* libpas/src/libpas/pas_physical_memory_transaction.h: Added.
|
||
* libpas/src/libpas/pas_primitive_heap_ref.c: Copied from Source/WTF/wtf/FastTLS.h.
|
||
* libpas/src/libpas/pas_primitive_heap_ref.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
* libpas/src/libpas/pas_promote_intrinsic_heap.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
* libpas/src/libpas/pas_ptr_hash_map.h: Added.
|
||
(pas_ptr_hash_map_entry_create_empty):
|
||
(pas_ptr_hash_map_entry_create_deleted):
|
||
(pas_ptr_hash_map_entry_is_empty_or_deleted):
|
||
(pas_ptr_hash_map_entry_is_empty):
|
||
(pas_ptr_hash_map_entry_is_deleted):
|
||
(pas_ptr_hash_map_entry_get_key):
|
||
(pas_ptr_hash_map_key_get_hash):
|
||
(pas_ptr_hash_map_key_is_equal):
|
||
* libpas/src/libpas/pas_ptr_hash_set.h: Added.
|
||
(pas_ptr_hash_set_entry_create_empty):
|
||
(pas_ptr_hash_set_entry_create_deleted):
|
||
(pas_ptr_hash_set_entry_is_empty_or_deleted):
|
||
(pas_ptr_hash_set_entry_is_empty):
|
||
(pas_ptr_hash_set_entry_is_deleted):
|
||
(pas_ptr_hash_set_entry_get_key):
|
||
(pas_ptr_hash_set_key_get_hash):
|
||
(pas_ptr_hash_set_key_is_equal):
|
||
* libpas/src/libpas/pas_ptr_min_heap.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
(pas_ptr_min_heap_compare):
|
||
(pas_ptr_min_heap_get_index):
|
||
(pas_ptr_min_heap_set_index):
|
||
* libpas/src/libpas/pas_ptr_worklist.c: Added.
|
||
(pas_ptr_worklist_construct):
|
||
(pas_ptr_worklist_destruct):
|
||
(pas_ptr_worklist_push):
|
||
(pas_ptr_worklist_pop):
|
||
* libpas/src/libpas/pas_ptr_worklist.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
* libpas/src/libpas/pas_race_test_hooks.c: Copied from Source/WTF/wtf/FastTLS.h.
|
||
* libpas/src/libpas/pas_race_test_hooks.h: Added.
|
||
(pas_race_test_hook_kind_get_string):
|
||
(pas_race_test_hook):
|
||
(pas_race_test_will_lock):
|
||
(pas_race_test_did_lock):
|
||
(pas_race_test_did_try_lock):
|
||
(pas_race_test_will_unlock):
|
||
* libpas/src/libpas/pas_random.c: Copied from Source/WTF/wtf/FastTLS.h.
|
||
* libpas/src/libpas/pas_random.h: Added.
|
||
(pas_get_random):
|
||
* libpas/src/libpas/pas_range.h: Added.
|
||
(pas_range_create):
|
||
(pas_range_create_empty):
|
||
(pas_range_create_forgiving):
|
||
(pas_range_is_empty):
|
||
(pas_range_size):
|
||
(pas_range_contains):
|
||
(pas_range_subsumes):
|
||
(pas_range_overlaps):
|
||
(pas_range_create_intersection):
|
||
(pas_range_compare):
|
||
* libpas/src/libpas/pas_range16.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
(pas_range16_create):
|
||
* libpas/src/libpas/pas_range_begin_min_heap.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
(pas_range_begin_min_heap_compare):
|
||
(pas_range_begin_min_heap_get_index):
|
||
(pas_range_begin_min_heap_set_index):
|
||
* libpas/src/libpas/pas_range_locked_mode.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
(pas_range_locked_mode_get_string):
|
||
* libpas/src/libpas/pas_range_min_heap.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
(pas_range_compare_begin):
|
||
(pas_range_get_index):
|
||
(pas_range_set_index):
|
||
* libpas/src/libpas/pas_reallocate_free_mode.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
* libpas/src/libpas/pas_reallocate_heap_teleport_rule.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
* libpas/src/libpas/pas_red_black_tree.c: Added.
|
||
(tree_insert):
|
||
(left_rotate):
|
||
(right_rotate):
|
||
(remove_fixup):
|
||
(pas_red_black_tree_insert):
|
||
(pas_red_black_tree_remove):
|
||
(pas_red_black_tree_size):
|
||
* libpas/src/libpas/pas_red_black_tree.h: Added.
|
||
(pas_red_black_tree_validate_enumerable):
|
||
(pas_red_black_tree_get_root):
|
||
(pas_red_black_tree_node_get_left):
|
||
(pas_red_black_tree_node_get_right):
|
||
(pas_red_black_tree_node_get_parent):
|
||
(pas_red_black_tree_node_get_color):
|
||
(pas_red_black_tree_set_root):
|
||
(pas_red_black_tree_node_set_left):
|
||
(pas_red_black_tree_node_set_right):
|
||
(pas_red_black_tree_node_set_parent):
|
||
(pas_red_black_tree_node_set_color):
|
||
(pas_red_black_tree_node_minimum):
|
||
(pas_red_black_tree_node_maximum):
|
||
(pas_red_black_tree_node_successor):
|
||
(pas_red_black_tree_node_predecessor):
|
||
(pas_red_black_tree_node_reset):
|
||
(pas_red_black_tree_construct):
|
||
(pas_red_black_tree_node_find_exact):
|
||
(pas_red_black_tree_find_exact):
|
||
(pas_red_black_tree_node_find_least_greater_than_or_equal):
|
||
(pas_red_black_tree_find_least_greater_than_or_equal):
|
||
(pas_red_black_tree_node_find_least_greater_than):
|
||
(pas_red_black_tree_find_least_greater_than):
|
||
(pas_red_black_tree_node_find_greatest_less_than_or_equal):
|
||
(pas_red_black_tree_find_greatest_less_than_or_equal):
|
||
(pas_red_black_tree_node_find_greatest_less_than):
|
||
(pas_red_black_tree_find_greatest_less_than):
|
||
(pas_red_black_tree_minimum):
|
||
(pas_red_black_tree_maximum):
|
||
(pas_red_black_tree_is_empty):
|
||
* libpas/src/libpas/pas_redundant_local_allocator_node.c: Copied from Source/WTF/wtf/FastTLS.h.
|
||
(pas_redundant_local_allocator_node_create):
|
||
* libpas/src/libpas/pas_redundant_local_allocator_node.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
* libpas/src/libpas/pas_reserved_memory_provider.c: Added.
|
||
(null_aligned_allocator):
|
||
(initialize_config):
|
||
(pas_reserved_memory_provider_construct):
|
||
(pas_reserved_memory_provider_try_allocate):
|
||
* libpas/src/libpas/pas_reserved_memory_provider.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
* libpas/src/libpas/pas_root.c: Added.
|
||
(count_static_heaps_callback):
|
||
(collect_static_heaps_callback):
|
||
(pas_root_construct):
|
||
(pas_root_create):
|
||
(default_reader):
|
||
(enumerator_reader):
|
||
(enumerator_recorder):
|
||
(pas_root_enumerate_for_libmalloc):
|
||
(pas_root_enumerate_for_libmalloc_with_root_after_zone):
|
||
(malloc_introspect_good_size):
|
||
(malloc_introspect_check):
|
||
(malloc_introspect_print):
|
||
(malloc_introspect_log):
|
||
(malloc_introspect_force_lock):
|
||
(malloc_introspect_force_unlock):
|
||
(malloc_introspect_statistics):
|
||
(pas_root_ensure_for_libmalloc_enumeration):
|
||
* libpas/src/libpas/pas_root.h: Added.
|
||
* libpas/src/libpas/pas_scavenger.c: Added.
|
||
(ensure_data_instance):
|
||
(get_time_in_milliseconds):
|
||
(timed_wait):
|
||
(scavenger_thread_main):
|
||
(pas_scavenger_did_create_eligible):
|
||
(pas_scavenger_notify_eligibility_if_needed):
|
||
(pas_scavenger_suspend):
|
||
(pas_scavenger_resume):
|
||
(pas_scavenger_clear_all_non_tlc_caches):
|
||
(pas_scavenger_clear_all_caches_except_remote_tlcs):
|
||
(pas_scavenger_clear_all_caches):
|
||
(pas_scavenger_decommit_free_memory):
|
||
(pas_scavenger_run_synchronously_now):
|
||
(pas_scavenger_perform_synchronous_operation):
|
||
* libpas/src/libpas/pas_scavenger.h: Added.
|
||
* libpas/src/libpas/pas_segmented_vector.h: Added.
|
||
* libpas/src/libpas/pas_segregated_biasing_directory.c: Added.
|
||
(pas_segregated_biasing_directory_create):
|
||
(take_first_eligible_loop_head_callback):
|
||
(take_first_eligible_create_new_view_callback):
|
||
(pas_segregated_biasing_directory_take_first_eligible):
|
||
(pas_segregated_biasing_directory_take_last_unused):
|
||
* libpas/src/libpas/pas_segregated_biasing_directory.h: Added.
|
||
(pas_segregated_biasing_directory_magazine_index):
|
||
* libpas/src/libpas/pas_segregated_biasing_view.c: Added.
|
||
(pas_segregated_biasing_view_create):
|
||
(pas_segregated_biasing_view_is_owned):
|
||
(pas_segregated_biasing_view_lock_ownership_lock):
|
||
(pas_segregated_biasing_view_should_table):
|
||
(pas_segregated_biasing_view_note_eligibility):
|
||
(pas_segregated_biasing_view_note_emptiness):
|
||
(pas_segregated_biasing_view_compute_summary):
|
||
(pas_segregated_biasing_view_is_eligible):
|
||
(pas_segregated_biasing_view_is_empty):
|
||
* libpas/src/libpas/pas_segregated_biasing_view.h: Added.
|
||
(pas_segregated_biasing_view_as_view):
|
||
(pas_segregated_biasing_view_as_ineligible_view):
|
||
(pas_segregated_biasing_view_as_view_non_null):
|
||
(pas_segregated_biasing_view_as_ineligible_view_non_null):
|
||
* libpas/src/libpas/pas_segregated_directory.c: Added.
|
||
(pas_segregated_directory_construct):
|
||
(pas_segregated_directory_get_data_slow):
|
||
(pas_segregated_directory_get_use_epoch):
|
||
(pas_segregated_directory_get_sharing_payload):
|
||
(pas_segregated_directory_minimize_first_eligible):
|
||
(pas_segregated_directory_update_first_eligible_after_search):
|
||
(pas_segregated_directory_view_did_become_eligible_at_index_without_biasing_update):
|
||
(pas_segregated_directory_view_did_become_eligible_at_index):
|
||
(pas_segregated_directory_view_did_become_eligible_without_biasing_update):
|
||
(pas_segregated_directory_view_did_become_eligible):
|
||
(maximize_last_empty):
|
||
(pas_segregated_directory_view_did_become_empty_at_index):
|
||
(pas_segregated_directory_view_did_become_empty):
|
||
(pas_segregated_directory_is_committed):
|
||
(pas_segregated_directory_num_committed_views):
|
||
(num_empty_views_should_consider_view_parallel):
|
||
(pas_segregated_directory_num_empty_views):
|
||
(pas_segregated_directory_update_last_empty_plus_one_after_search):
|
||
(pas_segregated_directory_append):
|
||
(pas_segregated_directory_compute_summary):
|
||
(pas_segregated_directory_num_empty_granules):
|
||
* libpas/src/libpas/pas_segregated_directory.h: Added.
|
||
(pas_segregated_directory_size):
|
||
(pas_segregated_directory_bits_get_by_mask):
|
||
(pas_segregated_directory_bits_set_by_mask):
|
||
(pas_segregated_directory_get_misc_bit):
|
||
(pas_segregated_directory_set_misc_bit):
|
||
(pas_segregated_directory_get_other_misc_bit):
|
||
(pas_segregated_directory_set_other_misc_bit):
|
||
(pas_segregated_directory_spoof_inline_segment):
|
||
(pas_segregated_directory_get_empty_bit):
|
||
(pas_segregated_directory_set_empty_bit):
|
||
(pas_segregated_directory_get_data):
|
||
(pas_segregated_directory_can_do_sharing):
|
||
(pas_segregated_directory_is_doing_sharing):
|
||
(pas_segregated_directory_data_try_get_sharing_payload):
|
||
(pas_segregated_directory_start_sharing_if_necessary):
|
||
(pas_segregated_directory_data_get_first_eligible_ptr):
|
||
(pas_segregated_directory_is_eligible):
|
||
(pas_segregated_directory_is_empty):
|
||
(pas_segregated_directory_is_tabled):
|
||
(pas_segregated_directory_get_first_eligible_impl):
|
||
(pas_segregated_directory_get_first_eligible):
|
||
(pas_segregated_directory_get_first_eligible_torn):
|
||
(pas_segregated_directory_watch_first_eligible):
|
||
(pas_segregated_directory_get_last_empty_plus_one_impl):
|
||
(pas_segregated_directory_get_last_empty_plus_one):
|
||
(pas_segregated_directory_get_last_empty_plus_one_value):
|
||
(pas_segregated_directory_watch_last_empty_plus_one):
|
||
(pas_segregated_directory_last_empty_plus_one):
|
||
(pas_segregated_directory_get):
|
||
* libpas/src/libpas/pas_segregated_directory_bit_reference.h: Added.
|
||
(pas_segregated_directory_bit_reference_create_null):
|
||
(pas_segregated_directory_bit_reference_create_inline):
|
||
(pas_segregated_directory_bit_reference_create_out_of_line):
|
||
(pas_segregated_directory_bit_reference_is_null):
|
||
(pas_segregated_directory_bit_reference_is_inline):
|
||
(pas_segregated_directory_bit_reference_is_out_of_line):
|
||
* libpas/src/libpas/pas_segregated_directory_first_eligible_kind.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
(pas_segregated_directory_find_eligible_kind_get_string):
|
||
* libpas/src/libpas/pas_segregated_directory_inlines.h: Added.
|
||
(pas_segregated_directory_iterate_iterate_callback):
|
||
(pas_segregated_directory_iterate_forward_iterate_callback):
|
||
(pas_segregated_directory_iterate_forward):
|
||
(pas_segregated_directory_iterate_backward_iterate_callback):
|
||
(pas_segregated_directory_iterate_backward):
|
||
(pas_segregated_directory_iterate_forward_to_take_first_eligible):
|
||
(pas_segregated_directory_iterate_backward_to_take_last_empty):
|
||
* libpas/src/libpas/pas_segregated_directory_kind.h: Added.
|
||
(pas_segregated_directory_kind_get_string):
|
||
* libpas/src/libpas/pas_segregated_exclusive_view.c: Added.
|
||
(pas_segregated_exclusive_view_create):
|
||
(pas_segregated_exclusive_view_should_table):
|
||
(pas_segregated_exclusive_ish_view_note_eligibility_impl):
|
||
(pas_segregated_exclusive_view_note_eligibility):
|
||
(pas_segregated_exclusive_view_note_emptiness):
|
||
(pas_segregated_exclusive_ish_view_compute_summary_impl):
|
||
(pas_segregated_exclusive_view_compute_summary):
|
||
(pas_segregated_exclusive_view_install_full_use_counts):
|
||
(pas_segregated_exclusive_view_is_eligible):
|
||
(pas_segregated_exclusive_view_is_empty):
|
||
* libpas/src/libpas/pas_segregated_exclusive_view.h: Added.
|
||
(pas_segregated_exclusive_view_as_view):
|
||
(pas_segregated_exclusive_view_as_ineligible_view):
|
||
(pas_segregated_exclusive_view_as_view_non_null):
|
||
(pas_segregated_exclusive_view_as_ineligible_view_non_null):
|
||
* libpas/src/libpas/pas_segregated_exclusive_view_inlines.h: Added.
|
||
(pas_segregated_exclusive_ish_view_did_start_allocating):
|
||
* libpas/src/libpas/pas_segregated_exclusive_view_ownership_kind.h: Added.
|
||
(pas_segregated_exclusive_view_ownership_kind_is_owned):
|
||
(pas_segregated_exclusive_view_ownership_kind_is_biased):
|
||
(pas_segregated_exclusive_view_ownership_kind_with_owned):
|
||
(pas_segregated_exclusive_view_ownership_kind_with_biased):
|
||
(pas_segregated_exclusive_view_ownership_kind_get_string):
|
||
* libpas/src/libpas/pas_segregated_global_size_directory.c: Added.
|
||
(pas_segregated_global_size_directory_create):
|
||
(pas_segregated_global_size_directory_ensure_data):
|
||
(pas_segregated_global_size_directory_get_extended_data):
|
||
(pas_segregated_global_size_directory_create_tlc_allocator):
|
||
(pas_segregated_global_size_directory_enable_exclusive_views):
|
||
(pas_segregated_global_size_directory_explode):
|
||
(pas_segregated_global_size_directory_select_allocator_slow):
|
||
(take_first_eligible_direct_loop_head_callback):
|
||
(take_first_eligible_direct_create_new_view_callback):
|
||
(pas_segregated_global_size_directory_take_first_eligible_direct):
|
||
(pas_segregated_global_size_directory_take_first_eligible):
|
||
(take_last_empty_should_consider_view_parallel):
|
||
(take_last_empty_consider_view):
|
||
(pas_segregated_global_size_directory_take_last_empty):
|
||
(pas_segregated_global_size_directory_for_object):
|
||
(pas_segregated_global_size_directory_get_allocator_from_tlc):
|
||
(pas_segregated_global_size_directory_compute_summary_for_unowned_exclusive):
|
||
(for_each_live_object_callback):
|
||
(pas_segregated_global_size_directory_for_each_live_object):
|
||
(pas_segregated_global_size_directory_local_allocator_size):
|
||
(pas_segregated_global_size_directory_num_allocator_indices):
|
||
(pas_segregated_global_size_directory_dump_reference):
|
||
(pas_segregated_global_size_directory_dump_for_spectrum):
|
||
* libpas/src/libpas/pas_segregated_global_size_directory.h: Added.
|
||
(pas_segregated_global_size_directory_as_view):
|
||
(pas_segregated_global_size_directory_alignment):
|
||
(pas_segregated_global_size_directory_contention_did_trigger_explosion):
|
||
(pas_segregated_global_size_directory_set_contention_did_trigger_explosion):
|
||
(pas_segregated_global_size_directory_has_tlc_allocator):
|
||
(pas_segregated_global_size_directory_are_exclusive_views_enabled):
|
||
(pas_segregated_global_size_directory_bias_sharing_pool):
|
||
* libpas/src/libpas/pas_segregated_global_size_directory_inlines.h: Added.
|
||
(pas_segregated_global_size_directory_select_allocator):
|
||
(pas_segregated_global_size_directory_local_allocator_size_for_null_config):
|
||
(pas_segregated_global_size_directory_local_allocator_size_for_config):
|
||
(pas_segregated_global_size_directory_num_allocator_indices_for_allocator_size):
|
||
(pas_segregated_global_size_directory_num_allocator_indices_for_config):
|
||
* libpas/src/libpas/pas_segregated_heap.c: Added.
|
||
(min_object_size_for_heap_config):
|
||
(max_count_for_page_config):
|
||
(max_object_size_for_page_config):
|
||
(max_small_count_for_heap_config):
|
||
(max_segregated_count_for_heap_config):
|
||
(max_bitfit_count_for_heap_config):
|
||
(max_count_for_heap_config):
|
||
(max_bitfit_object_size_for_heap_config):
|
||
(max_object_size_for_heap_config):
|
||
(pas_segregated_heap_construct):
|
||
(pas_segregated_heap_get_bitfit):
|
||
(medium_directory_tuple_for_index_impl):
|
||
(medium_directory_tuple_for_index_with_lock):
|
||
(pas_segregated_heap_medium_directory_tuple_for_index):
|
||
(pas_segregated_heap_medium_allocator_index_for_index):
|
||
(pas_segregated_heap_medium_size_directory_for_index):
|
||
(compute_small_index_upper_bound):
|
||
(ensure_count_lookup):
|
||
(pas_segregated_heap_ensure_allocator_index):
|
||
(compute_ideal_object_size):
|
||
(check_medium_directories):
|
||
(check_part_of_all_heaps_callback):
|
||
(check_part_of_all_segregated_heaps_callback):
|
||
(ensure_count_lookup_if_necessary):
|
||
(pas_segregated_heap_ensure_size_directory_for_count):
|
||
(pas_segregated_heap_get_num_free_bytes):
|
||
(pas_segregated_heap_for_each_global_size_directory):
|
||
(for_each_committed_size_directory_callback):
|
||
(pas_segregated_heap_for_each_committed_view):
|
||
(for_each_view_index_directory_callback):
|
||
(pas_segregated_heap_for_each_view_index):
|
||
(for_each_live_object_object_callback):
|
||
(for_each_live_object_directory_callback):
|
||
(for_each_live_object_bitfit_callback):
|
||
(pas_segregated_heap_for_each_live_object):
|
||
(num_committed_views_directory_callback):
|
||
(pas_segregated_heap_num_committed_views):
|
||
(num_empty_views_directory_callback):
|
||
(pas_segregated_heap_num_empty_views):
|
||
(num_empty_granules_directory_callback):
|
||
(pas_segregated_heap_num_empty_granules):
|
||
(num_views_directory_callback):
|
||
(pas_segregated_heap_num_views):
|
||
(compute_summary_directory_callback):
|
||
(pas_segregated_heap_compute_summary):
|
||
* libpas/src/libpas/pas_segregated_heap.h: Added.
|
||
(pas_segregated_heap_index_for_primitive_count):
|
||
(pas_segregated_heap_primitive_count_for_index):
|
||
(pas_segregated_heap_index_for_count):
|
||
(pas_segregated_heap_count_for_index):
|
||
(pas_segregated_heap_allocator_index_for_index):
|
||
(pas_segregated_heap_allocator_index_for_count_not_primitive):
|
||
* libpas/src/libpas/pas_segregated_heap_inlines.h: Added.
|
||
(pas_segregated_heap_size_directory_for_index):
|
||
(pas_segregated_heap_size_directory_for_count):
|
||
* libpas/src/libpas/pas_segregated_heap_lookup_kind.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
* libpas/src/libpas/pas_segregated_heap_page_sharing_mode.h: Added.
|
||
(pas_segregated_heap_page_sharing_mode_does_any_sharing):
|
||
(pas_segregated_heap_page_sharing_mode_does_virtual_sharing):
|
||
(pas_segregated_heap_page_sharing_mode_does_physical_sharing):
|
||
(pas_segregated_heap_page_sharing_mode_get_string):
|
||
* libpas/src/libpas/pas_segregated_page.c: Added.
|
||
(pas_segregated_page_switch_lock_slow):
|
||
(pas_segregated_page_switch_lock_and_rebias_to_magazine_while_ineligible_impl):
|
||
(pas_segregated_page_construct):
|
||
(pas_segregated_page_note_emptiness):
|
||
(commit_lock_for):
|
||
(pas_segregated_page_take_empty_granules):
|
||
(pas_segregated_page_take_physically):
|
||
(pas_segregated_page_commit_fully):
|
||
(verify_granules_live_object_callback):
|
||
(pas_segregated_page_verify_granules):
|
||
(pas_segregated_page_deallocation_did_fail):
|
||
(pas_segregated_page_get_num_empty_granules):
|
||
(pas_segregated_page_get_num_committed_granules):
|
||
(pas_segregated_page_get_config):
|
||
(pas_segregated_page_add_commit_range):
|
||
(pas_segregated_page_and_config_for_address_and_heap_config):
|
||
(pas_segregated_page_verify_num_non_empty_words):
|
||
* libpas/src/libpas/pas_segregated_page.h: Added.
|
||
(pas_segregated_page_header_size):
|
||
(pas_segregated_page_offset_from_page_boundary_to_first_object_for_hugging_mode):
|
||
(pas_segregated_page_offset_from_page_boundary_to_end_of_last_object_for_hugging_mode):
|
||
(pas_segregated_page_useful_object_payload_size_for_hugging_mode):
|
||
(pas_segregated_page_best_hugging_mode):
|
||
(pas_segregated_page_offset_from_page_boundary_to_first_object_exclusive):
|
||
(pas_segregated_page_offset_from_page_boundary_to_end_of_last_object_exclusive):
|
||
(pas_segregated_page_useful_object_payload_size):
|
||
(pas_segregated_page_number_of_objects):
|
||
(pas_segregated_page_bytes_dirtied_per_object):
|
||
(pas_segregated_page_get_granule_use_counts):
|
||
(pas_segregated_page_for_boundary):
|
||
(pas_segregated_page_for_boundary_or_null):
|
||
(pas_segregated_page_for_boundary_unchecked):
|
||
(pas_segregated_page_boundary):
|
||
(pas_segregated_page_boundary_or_null):
|
||
(pas_segregated_page_for_address_and_page_config):
|
||
(pas_segregated_page_is_allocated_with_page):
|
||
(pas_segregated_page_is_allocated):
|
||
(pas_segregated_page_for_address_and_heap_config):
|
||
* libpas/src/libpas/pas_segregated_page_and_config.h: Added.
|
||
(pas_segregated_page_and_config_create):
|
||
(pas_segregated_page_and_config_create_empty):
|
||
(pas_segregated_page_and_config_is_empty):
|
||
* libpas/src/libpas/pas_segregated_page_config.c: Added.
|
||
(pas_segregated_page_config_validate):
|
||
* libpas/src/libpas/pas_segregated_page_config.h: Added.
|
||
(pas_segregated_page_config_is_enabled):
|
||
(pas_segregated_page_config_min_align):
|
||
(pas_segregated_page_config_object_payload_end_offset_from_boundary):
|
||
(pas_segregated_page_config_num_alloc_words):
|
||
(pas_segregated_page_config_num_alloc_bytes):
|
||
(pas_segregated_page_config_get_kind):
|
||
(pas_segregated_page_config_kind_is_utility):
|
||
(pas_segregated_page_config_is_utility):
|
||
(pas_segregated_page_config_kind_heap_lock_hold_mode):
|
||
(pas_segregated_page_config_heap_lock_hold_mode):
|
||
* libpas/src/libpas/pas_segregated_page_config_inlines.h: Added.
|
||
* libpas/src/libpas/pas_segregated_page_config_kind.c: Copied from Source/WTF/wtf/FastTLS.h.
|
||
(pas_segregated_page_config_kind_for_each):
|
||
* libpas/src/libpas/pas_segregated_page_config_kind.def: Added.
|
||
* libpas/src/libpas/pas_segregated_page_config_kind.h: Added.
|
||
(pas_segregated_page_config_kind_get_string):
|
||
(pas_segregated_page_config_kind_get_config):
|
||
* libpas/src/libpas/pas_segregated_page_config_utils.h: Added.
|
||
(pas_segregated_page_config_verify_dealloc):
|
||
* libpas/src/libpas/pas_segregated_page_config_utils_inlines.h: Added.
|
||
* libpas/src/libpas/pas_segregated_page_config_variant.h: Added.
|
||
(pas_segregated_page_config_variant_get_string):
|
||
* libpas/src/libpas/pas_segregated_page_emptiness_kind.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
(pas_page_emptiness_kind_get_inverted):
|
||
(pas_page_emptiness_kind_get_string):
|
||
* libpas/src/libpas/pas_segregated_page_granule_use_count.h: Added.
|
||
(pas_segregated_page_granule_use_count_increment):
|
||
(pas_segregated_page_granule_for_each_use_in_range):
|
||
(pas_segregated_page_granule_increment_uses_for_range):
|
||
* libpas/src/libpas/pas_segregated_page_header_placement_mode.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
(pas_segregated_page_header_placement_mode_get_string):
|
||
* libpas/src/libpas/pas_segregated_page_inlines.h: Added.
|
||
(pas_segregated_page_offset_from_page_boundary_to_first_object):
|
||
(pas_segregated_page_initialize_full_use_counts):
|
||
(pas_segregated_page_lock_with_unbias_not_utility):
|
||
(pas_segregated_page_lock_with_unbias):
|
||
(pas_segregated_page_lock):
|
||
(pas_segregated_page_unlock):
|
||
(pas_segregated_page_switch_lock_with_mode):
|
||
(pas_segregated_page_switch_lock):
|
||
(pas_segregated_page_switch_lock_and_rebias_to_magazine_while_ineligible):
|
||
(pas_segregated_page_deallocate_with_page):
|
||
(pas_segregated_page_deallocate):
|
||
(pas_segregated_page_qualifies_for_decommit):
|
||
(pas_segregated_page_get_directory_for_address_in_page):
|
||
(pas_segregated_page_get_directory_for_address_and_page_config):
|
||
(pas_segregated_page_get_object_size_for_address_in_page):
|
||
(pas_segregated_page_get_object_size_for_address_and_page_config):
|
||
* libpas/src/libpas/pas_segregated_partial_view.c: Added.
|
||
(pas_segregated_partial_view_create):
|
||
(pas_segregated_partial_view_note_eligibility):
|
||
(pas_segregated_partial_view_set_is_in_use_for_allocation):
|
||
(pas_segregated_partial_view_should_table):
|
||
(compute_summary):
|
||
(pas_segregated_partial_view_compute_summary):
|
||
(pas_segregated_partial_view_is_eligible):
|
||
* libpas/src/libpas/pas_segregated_partial_view.h: Added.
|
||
(pas_segregated_partial_view_as_view):
|
||
(pas_segregated_partial_view_as_view_non_null):
|
||
* libpas/src/libpas/pas_segregated_partial_view_inlines.h: Added.
|
||
(pas_segregated_partial_view_tell_shared_handle_for_word_general_case_source):
|
||
(pas_segregated_partial_view_tell_shared_handle_for_word_general_case_callback):
|
||
(pas_segregated_partial_view_tell_shared_handle_for_word_general_case):
|
||
(pas_segregated_partial_view_did_start_allocating):
|
||
* libpas/src/libpas/pas_segregated_shared_handle.c: Added.
|
||
(pas_segregated_shared_handle_create):
|
||
(pas_segregated_shared_handle_destroy):
|
||
(pas_segregated_shared_handle_note_emptiness):
|
||
* libpas/src/libpas/pas_segregated_shared_handle.h: Added.
|
||
(pas_segregated_shared_handle_num_views):
|
||
(pas_segregated_shared_handle_size):
|
||
(pas_segregated_shared_handle_as_view):
|
||
(pas_segregated_shared_handle_as_view_non_null):
|
||
* libpas/src/libpas/pas_segregated_shared_handle_inlines.h: Added.
|
||
(pas_segregated_shared_handle_partial_view_ptr_for_index):
|
||
(pas_segregated_shared_handle_partial_view_for_index):
|
||
(pas_segregated_shared_handle_partial_view_for_object):
|
||
* libpas/src/libpas/pas_segregated_shared_page_directory.c: Added.
|
||
(find_first_eligible_should_consider_view_parallel):
|
||
(find_first_eligible_consider_view):
|
||
(pas_segregated_shared_page_directory_find_first_eligible):
|
||
(take_last_empty_should_consider_view_parallel):
|
||
(switch_to_ownership):
|
||
(take_last_empty_consider_view):
|
||
(pas_segregated_shared_page_directory_take_last_empty):
|
||
(pas_segregated_shared_page_directory_dump_reference):
|
||
(pas_segregated_shared_page_directory_dump_for_spectrum):
|
||
* libpas/src/libpas/pas_segregated_shared_page_directory.h: Added.
|
||
* libpas/src/libpas/pas_segregated_shared_view.c: Added.
|
||
(pas_segregated_shared_view_create):
|
||
(pas_segregated_shared_view_commit_page):
|
||
(compute_summary_for_each_live_object_callback):
|
||
(compute_summary):
|
||
(pas_segregated_shared_view_compute_summary):
|
||
(pas_segregated_shared_view_is_empty):
|
||
* libpas/src/libpas/pas_segregated_shared_view.h: Added.
|
||
(pas_segregated_shared_view_as_view):
|
||
(pas_segregated_shared_view_as_view_non_null):
|
||
(pas_shared_view_computed_bump_result_create_empty):
|
||
(pas_segregated_shared_view_compute_initial_new_bump):
|
||
(pas_segregated_shared_view_can_bump):
|
||
(pas_segregated_shared_view_compute_new_bump):
|
||
(pas_segregated_shared_view_bump):
|
||
* libpas/src/libpas/pas_segregated_shared_view_inlines.h: Added.
|
||
(pas_segregated_shared_view_commit_page_if_necessary):
|
||
* libpas/src/libpas/pas_segregated_size_directory.c: Added.
|
||
(pas_segregated_size_directory_construct):
|
||
(for_each_live_object_object_callback):
|
||
(pas_segregated_size_directory_for_each_live_object):
|
||
(pas_segregated_size_directory_for_object):
|
||
* libpas/src/libpas/pas_segregated_size_directory.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
* libpas/src/libpas/pas_segregated_size_directory_inlines.h: Added.
|
||
(pas_segregated_size_directory_take_first_eligible_impl_should_consider_view_not_tabled_parallel):
|
||
(pas_segregated_size_directory_take_first_eligible_impl_should_consider_view_tabled_parallel):
|
||
(pas_segregated_size_directory_take_first_eligible_impl):
|
||
(pas_segregated_size_directory_get_global):
|
||
* libpas/src/libpas/pas_segregated_view.c: Added.
|
||
(pas_segregated_view_get_size_directory_slow):
|
||
(pas_segregated_view_get_global_size_directory_slow):
|
||
(pas_segregated_view_get_page_config_kind):
|
||
(pas_segregated_view_get_page_config):
|
||
(pas_segregated_view_get_index):
|
||
(pas_segregated_view_get_page_boundary):
|
||
(pas_segregated_view_get_page):
|
||
(pas_segregated_view_should_restart):
|
||
(pas_segregated_view_could_bump):
|
||
(pas_segregated_view_get_commit_lock):
|
||
(pas_segregated_view_get_ownership_lock):
|
||
(pas_segregated_view_is_owned):
|
||
(pas_segregated_view_should_table):
|
||
(pas_segregated_view_is_biased_exclusive):
|
||
(pas_segregated_view_lock_ownership_lock):
|
||
(pas_segregated_view_lock_ownership_lock_conditionally):
|
||
(pas_segregated_view_lock_ownership_lock_if_owned):
|
||
(pas_segregated_view_lock_ownership_lock_if_owned_conditionally):
|
||
(pas_segregated_view_unlock_ownership_lock):
|
||
(pas_segregated_view_unlock_ownership_lock_conditionally):
|
||
(pas_segregated_view_is_primordial_partial):
|
||
(pas_segregated_view_note_eligibility):
|
||
(pas_segregated_view_note_emptiness):
|
||
(for_each_live_object):
|
||
(pas_segregated_view_for_each_live_object):
|
||
(should_be_eligible):
|
||
(pas_segregated_view_should_be_eligible):
|
||
(pas_segregated_view_for_object):
|
||
(pas_segregated_view_compute_summary):
|
||
(pas_segregated_view_is_eligible):
|
||
(pas_segregated_view_is_eligible_or_biased):
|
||
(is_payload_empty_callback):
|
||
(pas_segregated_view_is_payload_empty):
|
||
(pas_segregated_view_is_empty):
|
||
(pas_segregated_view_is_empty_or_biased):
|
||
* libpas/src/libpas/pas_segregated_view.h: Added.
|
||
(pas_segregated_view_create):
|
||
(pas_segregated_view_create_non_null):
|
||
(pas_segregated_view_get_ptr):
|
||
(pas_segregated_view_get_kind):
|
||
(pas_segregated_view_is_null):
|
||
(pas_segregated_view_is_exclusive):
|
||
(pas_segregated_view_is_ineligible_exclusive):
|
||
(pas_segregated_view_is_some_exclusive):
|
||
(pas_segregated_view_is_exclusive_ish):
|
||
(pas_segregated_view_is_biasing):
|
||
(pas_segregated_view_is_ineligible_biasing):
|
||
(pas_segregated_view_is_some_biasing):
|
||
(pas_segregated_view_get_biasing):
|
||
(pas_segregated_view_is_eligible_kind):
|
||
(pas_segregated_view_is_ineligible_kind):
|
||
(pas_segregated_view_as_ineligible):
|
||
(pas_segregated_view_as_eligible):
|
||
(pas_segregated_view_is_shared):
|
||
(pas_segregated_view_get_shared):
|
||
(pas_segregated_view_is_shared_handle):
|
||
(pas_segregated_view_get_shared_handle):
|
||
(pas_segregated_view_is_partial):
|
||
(pas_segregated_view_get_partial):
|
||
(pas_segregated_view_is_global_size_directory):
|
||
(pas_segregated_view_get_size_directory):
|
||
(pas_segregated_view_get_global_size_directory):
|
||
* libpas/src/libpas/pas_segregated_view_allocator_inlines.h: Added.
|
||
(pas_segregated_view_will_start_allocating):
|
||
(pas_segregated_view_did_stop_allocating):
|
||
* libpas/src/libpas/pas_segregated_view_inlines.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
(pas_segregated_view_get_exclusive):
|
||
* libpas/src/libpas/pas_segregated_view_kind.h: Added.
|
||
(pas_segregated_view_kind_get_character_code):
|
||
(pas_segregated_view_kind_get_string):
|
||
(pas_segregated_view_kind_is_eligible):
|
||
(pas_segregated_view_kind_is_exclusive_ish):
|
||
(pas_segregated_view_kind_is_ineligible):
|
||
(pas_segregated_view_kind_can_become_empty):
|
||
* libpas/src/libpas/pas_shared_handle_or_page.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
(pas_wrap_page):
|
||
(pas_is_wrapped_shared_handle):
|
||
(pas_is_wrapped_page):
|
||
(pas_unwrap_page):
|
||
* libpas/src/libpas/pas_shared_handle_or_page_boundary.h: Added.
|
||
(pas_wrap_page_boundary):
|
||
(pas_is_wrapped_shared_handle):
|
||
(pas_is_wrapped_page_boundary):
|
||
(pas_unwrap_page_boundary):
|
||
* libpas/src/libpas/pas_shared_handle_or_page_boundary_inlines.h: Added.
|
||
(pas_wrap_shared_handle):
|
||
(pas_unwrap_shared_handle_no_liveness_checks):
|
||
(pas_unwrap_shared_handle):
|
||
(pas_shared_handle_or_page_boundary_get_page_boundary_no_liveness_checks):
|
||
(pas_shared_handle_or_page_boundary_get_page_boundary):
|
||
* libpas/src/libpas/pas_shared_handle_or_page_inlines.h: Added.
|
||
(pas_wrap_shared_handle):
|
||
(pas_unwrap_shared_handle_no_liveness_checks):
|
||
(pas_unwrap_shared_handle):
|
||
(pas_shared_handle_or_page_get_page_no_liveness_checks):
|
||
(pas_shared_handle_or_page_get_page):
|
||
* libpas/src/libpas/pas_shared_page_directory_by_size.c: Added.
|
||
(pas_shared_page_directory_by_size_get):
|
||
(pas_shared_page_directory_by_size_for_each):
|
||
(pas_shared_page_directory_by_size_for_each_remote):
|
||
* libpas/src/libpas/pas_shared_page_directory_by_size.h: Added.
|
||
* libpas/src/libpas/pas_simple_free_heap_declarations.def: Added.
|
||
* libpas/src/libpas/pas_simple_free_heap_definitions.def: Added.
|
||
* libpas/src/libpas/pas_simple_free_heap_helpers.c: Added.
|
||
(pas_simple_free_heap_helpers_try_allocate_with_manual_alignment):
|
||
(pas_simple_free_heap_helpers_deallocate):
|
||
* libpas/src/libpas/pas_simple_free_heap_helpers.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
* libpas/src/libpas/pas_simple_large_free_heap.c: Added.
|
||
(free_list_entry):
|
||
(pas_simple_large_free_heap_construct):
|
||
(dump_free_list):
|
||
(consider_expanding):
|
||
(append):
|
||
(merge):
|
||
(remove_entry):
|
||
(index_to_cursor):
|
||
(cursor_to_index):
|
||
(simple_find_first):
|
||
(simple_find_by_end):
|
||
(simple_read_cursor):
|
||
(simple_write_cursor):
|
||
(simple_merge):
|
||
(simple_remove):
|
||
(simple_append):
|
||
(simple_commit):
|
||
(simple_dump):
|
||
(simple_add_mapped_bytes):
|
||
(initialize_generic_heap_config):
|
||
(merge_physical):
|
||
(try_allocate_without_fixing):
|
||
(fix_free_list_if_necessary):
|
||
(pas_simple_large_free_heap_try_allocate):
|
||
(pas_simple_large_free_heap_deallocate):
|
||
(pas_simple_large_free_heap_for_each_free):
|
||
(pas_simple_large_free_heap_get_num_free_bytes):
|
||
(pas_simple_large_free_heap_dump_to_printf):
|
||
* libpas/src/libpas/pas_simple_large_free_heap.h: Added.
|
||
* libpas/src/libpas/pas_simple_type.h: Added.
|
||
(pas_simple_type_has_key):
|
||
(pas_simple_type_get_key_data):
|
||
(pas_simple_type_unwrap):
|
||
(pas_simple_type_size):
|
||
(pas_simple_type_alignment):
|
||
(pas_simple_type_key):
|
||
(pas_simple_type_create):
|
||
(pas_simple_type_create_with_key_data):
|
||
(pas_simple_type_as_heap_type_get_type_size):
|
||
(pas_simple_type_as_heap_type_get_type_alignment):
|
||
* libpas/src/libpas/pas_size_thunk.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
(pas_trivial_size_thunk):
|
||
* libpas/src/libpas/pas_skip_list.c: Added.
|
||
(pas_skip_list_construct):
|
||
(pas_skip_list_node_allocate_with_height):
|
||
(pas_skip_list_node_allocate):
|
||
(pas_skip_list_node_deallocate):
|
||
(pas_skip_list_remove):
|
||
(pas_skip_list_size):
|
||
(validate_other_node):
|
||
(pas_skip_list_validate):
|
||
* libpas/src/libpas/pas_skip_list.h: Added.
|
||
(pas_skip_list_node_prev):
|
||
(pas_skip_list_node_next):
|
||
(pas_skip_list_head):
|
||
(pas_skip_list_is_empty):
|
||
* libpas/src/libpas/pas_skip_list_inlines.h: Added.
|
||
(pas_skip_list_find_result_create_exact):
|
||
(pas_skip_list_find_result_create_inexact):
|
||
(pas_skip_list_level_get_direction):
|
||
(pas_skip_list_find_impl):
|
||
(pas_skip_list_find_ignore_head_attachment):
|
||
(pas_skip_list_find_ignore_pole_attachment):
|
||
(pas_skip_list_find):
|
||
(pas_skip_list_insert_after_note_head_attachment):
|
||
(pas_skip_list_insert_after_note_pole_attachment):
|
||
(pas_skip_list_insert):
|
||
* libpas/src/libpas/pas_slow_path_mode.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
* libpas/src/libpas/pas_slow_path_mode_prefix.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
* libpas/src/libpas/pas_small_large_map_entry.h: Added.
|
||
(pas_small_large_map_entry_create_empty):
|
||
(pas_small_large_map_entry_create_deleted):
|
||
(pas_small_large_map_entry_is_empty_or_deleted):
|
||
(pas_small_large_map_entry_is_empty):
|
||
(pas_small_large_map_entry_is_deleted):
|
||
(pas_small_large_map_entry_create):
|
||
(pas_small_large_map_entry_begin):
|
||
(pas_small_large_map_entry_get_key):
|
||
(pas_small_large_map_entry_end):
|
||
(pas_small_large_map_entry_heap):
|
||
(pas_small_large_map_entry_get_entry):
|
||
(pas_small_large_map_entry_can_create):
|
||
* libpas/src/libpas/pas_snprintf.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
* libpas/src/libpas/pas_status_reporter.c: Added.
|
||
(dump_ratio_initial):
|
||
(dump_occupancy_initial):
|
||
(dump_arrow):
|
||
(report_bitfit_directory_contents):
|
||
(pas_status_reporter_dump_bitfit_biasing_directory):
|
||
(pas_status_reporter_dump_bitfit_global_directory):
|
||
(report_segregated_directory_contents):
|
||
(pas_status_reporter_dump_segregated_biasing_directory):
|
||
(pas_status_reporter_dump_segregated_global_size_directory):
|
||
(pas_status_reporter_dump_segregated_shared_page_directory):
|
||
(pas_status_reporter_dump_large_heap):
|
||
(pas_status_reporter_dump_large_map):
|
||
(pas_status_reporter_dump_heap_table):
|
||
(pas_status_reporter_dump_immortal_heap):
|
||
(pas_status_reporter_dump_compact_large_utility_free_heap):
|
||
(pas_status_reporter_dump_large_utility_free_heap):
|
||
(pas_status_reporter_dump_compact_bootstrap_free_heap):
|
||
(pas_status_reporter_dump_bootstrap_free_heap):
|
||
(dump_segregated_heap_directory_callback):
|
||
(pas_status_reporter_dump_bitfit_heap):
|
||
(pas_status_reporter_dump_segregated_heap):
|
||
(pas_status_reporter_dump_heap):
|
||
(dump_all_heaps_heap_callback):
|
||
(pas_status_reporter_dump_all_heaps):
|
||
(dump_all_shared_page_directories_directory_callback):
|
||
(pas_status_reporter_dump_all_shared_page_directories):
|
||
(pas_status_reporter_dump_all_heaps_non_utility_summaries):
|
||
(pas_status_reporter_dump_utility_heap):
|
||
(add_to_size_histogram):
|
||
(dump_histogram):
|
||
(total_fragmentation_global_size_directory_callback):
|
||
(total_fragmentation_heap_callback):
|
||
(total_fragmentation_shared_page_directory_callback):
|
||
(pas_status_reporter_dump_total_fragmentation):
|
||
(tier_up_rate_global_size_directory_callback):
|
||
(tier_up_rate_heap_callback):
|
||
(dump_directory_tier_up_rate):
|
||
(pas_status_reporter_dump_tier_up_rates):
|
||
(allocator_state):
|
||
(pas_status_reporter_dump_baseline_allocators):
|
||
(pas_status_reporter_dump_thread_local_caches):
|
||
(pas_status_reporter_dump_configuration):
|
||
(pas_status_reporter_dump_everything):
|
||
(status_reporter_thread_main):
|
||
(start_reporter):
|
||
(pas_status_reporter_start_if_necessary):
|
||
* libpas/src/libpas/pas_status_reporter.h: Added.
|
||
* libpas/src/libpas/pas_stream.c: Copied from Source/WTF/wtf/FastTLS.h.
|
||
(pas_stream_vprintf):
|
||
(pas_stream_printf):
|
||
* libpas/src/libpas/pas_stream.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
(pas_stream_print_comma):
|
||
* libpas/src/libpas/pas_string_stream.c: Added.
|
||
(string_stream_vprintf):
|
||
(pas_string_stream_destruct):
|
||
(pas_string_stream_reset):
|
||
(pas_string_stream_vprintf):
|
||
* libpas/src/libpas/pas_string_stream.h: Added.
|
||
(pas_string_stream_get_string):
|
||
(pas_string_stream_get_string_length):
|
||
* libpas/src/libpas/pas_subpage_map.c: Added.
|
||
(pas_subpage_map_add):
|
||
(pas_subpage_map_get):
|
||
(pas_subpage_map_get_full_base):
|
||
* libpas/src/libpas/pas_subpage_map.h: Added.
|
||
(pas_subpage_map_hashtable_entry_create_empty):
|
||
(pas_subpage_map_hashtable_entry_create_deleted):
|
||
(pas_subpage_map_hashtable_entry_is_empty_or_deleted):
|
||
(pas_subpage_map_hashtable_entry_is_empty):
|
||
(pas_subpage_map_hashtable_entry_is_deleted):
|
||
(pas_subpage_map_hashtable_entry_get_key):
|
||
(pas_subpage_map_hashtable_key_get_hash):
|
||
(pas_subpage_map_hashtable_key_is_equal):
|
||
* libpas/src/libpas/pas_subpage_map_entry.c: Added.
|
||
(set_packed_value):
|
||
(pas_subpage_map_entry_create):
|
||
(prepare_indices):
|
||
(pas_subpage_map_assert_commit_state):
|
||
(set_bits):
|
||
(pas_subpage_map_entry_commit):
|
||
(pas_subpage_map_entry_decommit):
|
||
* libpas/src/libpas/pas_subpage_map_entry.h: Added.
|
||
(pas_subpage_map_entry_full_base):
|
||
(pas_subpage_map_entry_bits):
|
||
* libpas/src/libpas/pas_thread_local_cache.c: Added.
|
||
(pas_thread_local_cache_size_for_allocator_index_capacity):
|
||
(deallocate):
|
||
(destroy):
|
||
(destructor):
|
||
(allocate_cache):
|
||
(dump_thread_diagnostics):
|
||
(pas_thread_local_cache_create):
|
||
(pas_thread_local_cache_destroy):
|
||
(pas_thread_local_cache_get_slow):
|
||
(pas_thread_local_cache_get_local_allocator_slow):
|
||
(pas_thread_local_cache_get_local_allocator_if_can_set_cache_slow):
|
||
(pas_thread_local_cache_stop_local_allocators):
|
||
(stop_local_allocators_if_necessary_set_bit_source):
|
||
(stop_local_allocators_if_necessary_set_bit_callback):
|
||
(pas_thread_local_cache_stop_local_allocators_if_necessary):
|
||
(process_deallocation_log_with_config):
|
||
(flush_deallocation_log):
|
||
(pas_thread_local_cache_flush_deallocation_log):
|
||
(suspend):
|
||
(resume):
|
||
(pas_thread_local_cache_for_all):
|
||
(pas_thread_local_cache_append_deallocation_slow):
|
||
(pas_thread_local_cache_shrink):
|
||
* libpas/src/libpas/pas_thread_local_cache.h: Added.
|
||
(pas_thread_local_cache_try_get):
|
||
(pas_thread_local_cache_is_guaranteed_to_destruct):
|
||
(pas_thread_local_cache_can_set):
|
||
(pas_thread_local_cache_set_impl):
|
||
(pas_thread_local_cache_get_already_initialized):
|
||
(pas_thread_local_cache_get_with_heap_lock_hold_mode):
|
||
(pas_thread_local_cache_get):
|
||
(pas_thread_local_cache_get_holding_heap_lock):
|
||
(pas_thread_local_cache_get_local_allocator_impl):
|
||
(pas_thread_local_cache_get_local_allocator):
|
||
(pas_thread_local_cache_try_get_local_allocator):
|
||
(pas_thread_local_cache_get_local_allocator_if_can_set_cache):
|
||
(pas_thread_local_cache_encode_object):
|
||
(pas_thread_local_cache_append_deallocation):
|
||
(pas_thread_local_cache_append_deallocation_with_size):
|
||
* libpas/src/libpas/pas_thread_local_cache_layout.c: Added.
|
||
(pas_thread_local_cache_layout_add_node):
|
||
(pas_thread_local_cache_layout_add):
|
||
(pas_thread_local_cache_layout_duplicate):
|
||
* libpas/src/libpas/pas_thread_local_cache_layout.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
* libpas/src/libpas/pas_thread_local_cache_layout_node.c: Added.
|
||
(pas_thread_local_cache_layout_node_get_directory):
|
||
(allocator_index_ptr):
|
||
(pas_thread_local_cache_layout_node_get_allocator_index):
|
||
(pas_thread_local_cache_layout_node_set_allocator_index):
|
||
(next_ptr):
|
||
(pas_thread_local_cache_layout_node_get_next):
|
||
(pas_thread_local_cache_layout_node_set_next):
|
||
* libpas/src/libpas/pas_thread_local_cache_layout_node.h: Added.
|
||
(pas_wrap_segregated_global_size_directory):
|
||
(pas_wrap_redundant_local_allocator_node):
|
||
(pas_is_wrapped_segregated_global_size_directory):
|
||
(pas_is_wrapped_redundant_local_allocator_node):
|
||
(pas_unwrap_segregated_global_size_directory):
|
||
(pas_unwrap_redundant_local_allocator_node):
|
||
* libpas/src/libpas/pas_thread_local_cache_node.c: Added.
|
||
(pas_thread_local_cache_node_allocate):
|
||
(pas_thread_local_cache_node_deallocate):
|
||
* libpas/src/libpas/pas_thread_local_cache_node.h: Added.
|
||
* libpas/src/libpas/pas_tiny_large_map_entry.h: Added.
|
||
(pas_tiny_large_map_entry_base):
|
||
(pas_tiny_large_map_entry_create):
|
||
(pas_tiny_large_map_entry_begin):
|
||
(pas_tiny_large_map_entry_end):
|
||
(pas_tiny_large_map_entry_heap):
|
||
(pas_tiny_large_map_entry_get_entry):
|
||
(pas_tiny_large_map_entry_can_create):
|
||
(pas_tiny_large_map_entry_create_empty):
|
||
(pas_tiny_large_map_entry_create_deleted):
|
||
(pas_tiny_large_map_entry_is_empty_or_deleted):
|
||
(pas_tiny_large_map_entry_is_empty):
|
||
(pas_tiny_large_map_entry_is_deleted):
|
||
(pas_tiny_large_map_entry_get_key):
|
||
(pas_tiny_large_map_key_get_hash):
|
||
(pas_tiny_large_map_key_is_equal):
|
||
* libpas/src/libpas/pas_tree_direction.h: Added.
|
||
(pas_tree_direction_get_string):
|
||
(pas_tree_direction_invert):
|
||
(pas_tree_direction_invert_comparison_result_if_right):
|
||
* libpas/src/libpas/pas_tri_state.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
(pas_tri_state_get_string):
|
||
(pas_tri_state_equals_boolean):
|
||
* libpas/src/libpas/pas_try_allocate.h: Added.
|
||
(pas_try_allocate_impl_size_thunk):
|
||
(pas_try_allocate_impl):
|
||
* libpas/src/libpas/pas_try_allocate_array.h: Added.
|
||
(pas_try_allocate_array_impl):
|
||
* libpas/src/libpas/pas_try_allocate_common.h: Added.
|
||
(pas_try_allocate_common_can_go_fast):
|
||
(pas_try_allocate_common_impl_fast):
|
||
(pas_try_allocate_common_impl_slow):
|
||
(pas_try_allocate_common_impl):
|
||
* libpas/src/libpas/pas_try_allocate_intrinsic_primitive.h: Added.
|
||
(pas_try_allocate_intrinsic_primitive_impl_medium_slow_case):
|
||
(pas_try_allocate_intrinsic_primitive_impl):
|
||
* libpas/src/libpas/pas_try_allocate_primitive.h: Added.
|
||
(pas_try_allocate_primitive_impl):
|
||
* libpas/src/libpas/pas_try_reallocate.h: Added.
|
||
(pas_try_allocate_for_reallocate_and_copy):
|
||
(pas_try_reallocate_table_segregated_case):
|
||
(pas_try_reallocate_table_bitfit_case):
|
||
(pas_try_reallocate):
|
||
(pas_try_reallocate_intrinsic_primitive_allocate_callback):
|
||
(pas_try_reallocate_intrinsic_primitive):
|
||
(pas_try_reallocate_single_allocate_callback):
|
||
(pas_try_reallocate_single):
|
||
(pas_try_reallocate_array_allocate_callback):
|
||
(pas_try_reallocate_array):
|
||
(pas_try_reallocate_primitive_allocate_callback):
|
||
(pas_try_reallocate_primitive):
|
||
* libpas/src/libpas/pas_try_shrink.h: Added.
|
||
(pas_try_shrink):
|
||
* libpas/src/libpas/pas_typed_allocation_result.h: Added.
|
||
(pas_typed_allocation_result_create_empty):
|
||
(pas_typed_allocation_result_create):
|
||
(pas_typed_allocation_result_as_intrinsic_allocation_result):
|
||
(pas_typed_allocation_result_create_with_intrinsic_allocation_result):
|
||
(pas_typed_allocation_result_zero):
|
||
(pas_typed_allocation_result_set_errno):
|
||
(pas_typed_allocation_result_crash_on_error):
|
||
* libpas/src/libpas/pas_utility_heap.c: Added.
|
||
(pas_utility_heap_allocate_with_alignment):
|
||
(pas_utility_heap_try_allocate):
|
||
(pas_utility_heap_allocate):
|
||
(pas_utility_heap_deallocate):
|
||
(pas_utility_heap_get_num_free_bytes):
|
||
(for_each_live_object_small_object_callback):
|
||
(pas_utility_heap_for_each_live_object):
|
||
(pas_utility_heap_for_all_allocators):
|
||
* libpas/src/libpas/pas_utility_heap.h: Added.
|
||
(pas_utility_heap_allocate_with_asserted_kind):
|
||
(pas_utility_heap_deallocate_with_ignored_size_and_asserted_kind):
|
||
* libpas/src/libpas/pas_utility_heap_config.c: Added.
|
||
(pas_utility_heap_allocate_page):
|
||
(pas_utility_heap_config_for_each_shared_page_directory):
|
||
* libpas/src/libpas/pas_utility_heap_config.h: Added.
|
||
(pas_utility_heap_page_header_for_boundary):
|
||
(pas_utility_heap_boundary_for_page_header):
|
||
(pas_utility_heap_shared_page_directory_selector):
|
||
(pas_utility_heap_create_page_header):
|
||
(pas_utility_heap_destroy_page_header):
|
||
(pas_heap_config_is_utility):
|
||
(pas_heap_config_heap_lock_hold_mode):
|
||
* libpas/src/libpas/pas_utility_heap_support.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
* libpas/src/libpas/pas_utils.c: Added.
|
||
(pas_panic):
|
||
(pas_assertion_failed):
|
||
(pas_deallocation_did_fail):
|
||
(pas_set_deallocation_did_fail_callback):
|
||
(pas_reallocation_did_fail):
|
||
(pas_set_reallocation_did_fail_callback):
|
||
* libpas/src/libpas/pas_utils.h: Added.
|
||
(pas_is_power_of_2):
|
||
(pas_round_down_to_power_of_2):
|
||
(pas_round_down):
|
||
(pas_round_up_to_power_of_2):
|
||
(pas_round_up):
|
||
(pas_modulo_power_of_2):
|
||
(pas_is_aligned):
|
||
(pas_reverse):
|
||
(pas_reverse64):
|
||
(pas_make_mask64):
|
||
(pas_compare_and_swap_uintptr_weak):
|
||
(pas_compare_and_swap_uintptr_strong):
|
||
(pas_compare_and_swap_bool_weak):
|
||
(pas_compare_and_swap_bool_strong):
|
||
(pas_compare_and_swap_uint16_weak):
|
||
(pas_compare_and_swap_uint32_weak):
|
||
(pas_compare_and_swap_uint32_strong):
|
||
(pas_compare_and_swap_uint64_weak):
|
||
(pas_compare_and_swap_uint64_strong):
|
||
(pas_compare_and_swap_ptr_weak):
|
||
(pas_compare_and_swap_ptr_strong):
|
||
(pas_fence_after_load):
|
||
(pas_store_store_fence):
|
||
(pas_opaque):
|
||
(pas_pair_create):
|
||
(pas_compare_and_swap_pair_weak):
|
||
(pas_compare_and_swap_pair_strong):
|
||
(pas_atomic_load_pair):
|
||
(pas_atomic_store_pair):
|
||
(pas_hash32):
|
||
(pas_hash64):
|
||
(pas_hash_intptr):
|
||
(pas_hash_ptr):
|
||
(pas_log2):
|
||
(pas_log2_rounded_up):
|
||
(pas_log2_rounded_up_safe):
|
||
(pas_non_empty_ranges_overlap):
|
||
(pas_ranges_overlap):
|
||
(pas_xorshift32):
|
||
(pas_large_object_hash):
|
||
(pas_is_divisible_by):
|
||
* libpas/src/libpas/pas_utils_prefix.h: Added.
|
||
(__pas_round_up_to_power_of_2):
|
||
(__pas_compiler_fence):
|
||
(__pas_fence):
|
||
(__pas_depend_impl):
|
||
(__pas_depend):
|
||
(__pas_depend_cpu_only):
|
||
(__pas_memcpy):
|
||
* libpas/src/libpas/pas_versioned_field.c: Added.
|
||
(pas_versioned_field_minimize):
|
||
(pas_versioned_field_maximize):
|
||
(pas_versioned_field_minimize_watched):
|
||
(pas_versioned_field_maximize_watched):
|
||
* libpas/src/libpas/pas_versioned_field.h: Added.
|
||
(pas_versioned_field_construct):
|
||
(pas_versioned_field_create):
|
||
(pas_versioned_field_create_with_invalid_version):
|
||
(pas_versioned_field_create_empty):
|
||
(pas_versioned_field_with_version):
|
||
(pas_versioned_field_with_value):
|
||
(pas_versioned_field_as_pair):
|
||
(pas_versioned_field_from_pair):
|
||
(pas_versioned_field_equals):
|
||
(pas_versioned_field_read_torn):
|
||
(pas_versioned_field_read):
|
||
(pas_versioned_field_weak_cas):
|
||
(pas_versioned_field_strong_cas):
|
||
(pas_versioned_field_read_to_watch):
|
||
(pas_versioned_field_try_write):
|
||
(pas_versioned_field_try_write_watched):
|
||
(pas_versioned_field_write):
|
||
* libpas/src/libpas/pas_virtual_range.c: Copied from Source/WTF/wtf/FastTLS.h.
|
||
* libpas/src/libpas/pas_virtual_range.h: Added.
|
||
(pas_virtual_range_create):
|
||
(pas_virtual_range_create_empty):
|
||
(pas_virtual_range_get_range):
|
||
(pas_virtual_range_is_empty):
|
||
(pas_virtual_range_size):
|
||
(pas_virtual_range_overlaps):
|
||
* libpas/src/libpas/pas_virtual_range_min_heap.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
(pas_virtual_range_compare_begin):
|
||
(pas_virtual_range_get_index):
|
||
(pas_virtual_range_set_index):
|
||
* libpas/src/libpas/pas_zero_mode.h: Added.
|
||
(pas_zero_mode_get_string):
|
||
(pas_zero_mode_validate):
|
||
(pas_zero_mode_merge):
|
||
* libpas/src/libpas/thingy_heap.c: Added.
|
||
(thingy_try_allocate_primitive):
|
||
(thingy_try_allocate_primitive_zeroed):
|
||
(thingy_try_allocate_primitive_with_alignment):
|
||
(thingy_try_reallocate_primitive):
|
||
(thingy_try_allocate):
|
||
(thingy_try_allocate_zeroed):
|
||
(thingy_try_allocate_array):
|
||
(thingy_try_allocate_zeroed_array):
|
||
(thingy_get_allocation_size):
|
||
(thingy_try_reallocate_array):
|
||
(thingy_deallocate):
|
||
(thingy_heap_ref_get_heap):
|
||
(thingy_utility_heap_allocate):
|
||
* libpas/src/libpas/thingy_heap.h: Added.
|
||
* libpas/src/libpas/thingy_heap_config.c: Copied from Source/WTF/wtf/FastTLS.h.
|
||
* libpas/src/libpas/thingy_heap_config.h: Added.
|
||
* libpas/src/libpas/thingy_heap_prefix.h: Added.
|
||
* libpas/src/mbmalloc/mbmalloc_bmalloc.c: Copied from Source/WTF/wtf/FastTLS.h.
|
||
(mbmalloc):
|
||
(mbmemalign):
|
||
(mbrealloc):
|
||
(mbfree):
|
||
(mbscavenge):
|
||
* libpas/src/mbmalloc/mbmalloc_hotbit.c: Copied from Source/WTF/wtf/FastTLS.h.
|
||
(mbmalloc):
|
||
(mbmemalign):
|
||
(mbrealloc):
|
||
(mbfree):
|
||
(mbscavenge):
|
||
* libpas/src/mbmalloc/mbmalloc_iso_common_primitive.c: Added.
|
||
(install_verifier_if_necessary):
|
||
(mbmalloc):
|
||
(mbmemalign):
|
||
(mbrealloc):
|
||
(mbfree):
|
||
(dump_summary):
|
||
(mbscavenge):
|
||
* libpas/src/mbmalloc/mbmalloc_lineref.c: Copied from Source/WTF/wtf/FastTLS.h.
|
||
(mbmalloc):
|
||
(mbmemalign):
|
||
(mbrealloc):
|
||
(mbfree):
|
||
(mbscavenge):
|
||
* libpas/src/test/BitfieldVectorTests.cpp: Added.
|
||
(std::testBitfieldNumWords):
|
||
(std::testBitfieldNumFields):
|
||
(std::testBitfieldWordIndex):
|
||
(std::testBitfieldFieldIndex):
|
||
(std::testBitfieldFieldShift):
|
||
(std::BitfieldVector::BitfieldVector):
|
||
(std::BitfieldVector::get const):
|
||
(std::BitfieldVector::set):
|
||
(std::testBitfieldVectorForward):
|
||
(std::testBitfieldVectorBackward):
|
||
(addBitfieldVectorTests):
|
||
* libpas/src/test/BitvectorTests.cpp: Added.
|
||
(std::testFindFirst):
|
||
(addBitvectorTests):
|
||
* libpas/src/test/CartesianTreeTests.cpp: Added.
|
||
(std::TestNode::TestNode):
|
||
(std::TestNode::compare):
|
||
(std::TestNode::getXKey):
|
||
(std::TestNode::getYKey):
|
||
(std::Pair::Pair):
|
||
(std::Pair::operator== const):
|
||
(std::Pair::operator< const):
|
||
(std::findExact):
|
||
(std::remove):
|
||
(std::findLeastGreaterThanOrEqual):
|
||
(std::assertFoundAndRemove):
|
||
(std::assertEqual):
|
||
(std::assertSameValuesForKey):
|
||
(std::cartesianTreePrintWidth):
|
||
(std::printSpaces):
|
||
(std::dumpCartesianTree):
|
||
(std::dumpVector):
|
||
(std::testDriver):
|
||
(std::randomTestDriver):
|
||
(addCartesianTreeTests):
|
||
* libpas/src/test/CoalignTests.cpp: Added.
|
||
(std::testCoalignOneSided):
|
||
(std::testCoalignOneSidedError):
|
||
(std::testCoalign):
|
||
(addCoalignTests):
|
||
* libpas/src/test/ExtendedGCDTests.cpp: Copied from Source/WTF/wtf/FastTLS.h.
|
||
(std::testExtendedGCD):
|
||
(addExtendedGCDTests):
|
||
* libpas/src/test/HashtableTests.cpp: Added.
|
||
(std::hashtableForEachEntryCallback):
|
||
(std::hashtableForEachEntry):
|
||
(std::Key::Key):
|
||
(std::CollidingEntry_create_empty):
|
||
(std::CollidingEntry_create_deleted):
|
||
(std::CollidingEntry_is_empty_or_deleted):
|
||
(std::CollidingEntry_is_empty):
|
||
(std::CollidingEntry_is_deleted):
|
||
(std::CollidingEntry_get_key):
|
||
(std::CollidingKey_get_hash):
|
||
(std::CollidingKey_is_equal):
|
||
(std::testEmptyCollidingHashtable):
|
||
(std::testCollidingHashtableAddFindTakeImpl):
|
||
(std::testCollidingHashtableAddFindTake):
|
||
(std::testCollidingHashtableAddAddTakeSet):
|
||
(std::testCollidingHashtableAddAddAddTakeTakeSet):
|
||
(std::testCollidingHashtableAddAddAddTakeTakeAddSet):
|
||
(std::OutOfLineEntry_create_empty):
|
||
(std::OutOfLineEntry_create_deleted):
|
||
(std::OutOfLineEntry_is_empty_or_deleted):
|
||
(std::OutOfLineEntry_is_empty):
|
||
(std::OutOfLineEntry_is_deleted):
|
||
(std::OutOfLineEntry_get_key):
|
||
(std::OutOfLineKey_get_hash):
|
||
(std::OutOfLineKey_is_equal):
|
||
(std::testOutOfLineHashtable):
|
||
(addHashtableTests):
|
||
* libpas/src/test/HeapLocker.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
(HeapLocker::HeapLocker):
|
||
(HeapLocker::~HeapLocker):
|
||
* libpas/src/test/IsoDynamicPrimitiveHeapTests.cpp: Added.
|
||
(std::allocate42):
|
||
(std::allocate42WithAlignment):
|
||
(std::allocate42Zeroed):
|
||
(std::reallocate42):
|
||
(std::testManySizesAndKeys):
|
||
(std::testManySizesAndKeysInTandem):
|
||
(addIsoDynamicPrimitiveHeapTests):
|
||
* libpas/src/test/IsoHeapChaosTests.cpp: Added.
|
||
(std::flushDeallocationLogAndStopAllocators):
|
||
(std::verifyObjectSet):
|
||
(std::scavengerDidStart):
|
||
(std::scavengerWillShutDown):
|
||
(std::PageRange::PageRange):
|
||
(std::PageRange::operator< const):
|
||
(std::PageRange::end const):
|
||
(std::addPageRange):
|
||
(std::addPageRangeCallback):
|
||
(std::RecordedRange::RecordedRange):
|
||
(std::RecordedRange::operator< const):
|
||
(std::RecordedRange::end const):
|
||
(std::ReaderRange::ReaderRange):
|
||
(std::ReaderRange::operator< const):
|
||
(std::enumeratorReader):
|
||
(std::enumeratorRecorder):
|
||
(std::testAllocationChaos):
|
||
(std::addTheTests):
|
||
(std::addSpotTests):
|
||
(addIsoHeapChaosTests):
|
||
* libpas/src/test/IsoHeapPageSharingTests.cpp: Added.
|
||
(std::testTakePages):
|
||
(std::testTakePagesFromCorrectHeap):
|
||
(std::testLargeHeapTakesPagesFromCorrectSmallHeap):
|
||
(std::testLargeHeapTakesPagesFromCorrectSmallHeapAllocateAfterFree):
|
||
(std::testLargeHeapTakesPagesFromCorrectSmallHeapWithFancyOrder):
|
||
(std::testLargeHeapTakesPagesFromCorrectLargeHeap):
|
||
(std::testLargeHeapTakesPagesFromCorrectLargeHeapAllocateAfterFreeOnSmallHeap):
|
||
(std::testLargeHeapTakesPagesFromCorrectLargeHeapAllocateAfterFreeOnAnotherLargeHeap):
|
||
(std::testLargeHeapTakesPagesFromCorrectLargeHeapWithFancyOrder):
|
||
(std::testSmallHeapTakesPagesFromCorrectLargeHeap):
|
||
(std::testSmallHeapTakesPagesFromCorrectLargeHeapWithFancyOrder):
|
||
(std::testSmallHeapTakesPagesFromCorrectLargeHeapAllocateAfterFreeOnSmallHeap):
|
||
(std::testSmallHeapTakesPagesFromCorrectLargeHeapAllocateAfterFreeOnAnotherLargeHeap):
|
||
(std::thingyName):
|
||
(std::forEachThingyKind):
|
||
(std::setupThingy):
|
||
(std::cleanThingy):
|
||
(std::checkThingy):
|
||
(std::addObject):
|
||
(std::deleteThingy):
|
||
(std::allocationKindName):
|
||
(std::allocateThingiesImpl):
|
||
(std::allocateThingies):
|
||
(std::heapSummaryFor):
|
||
(std::assertOnlyDecommitted):
|
||
(std::testFullVdirToVdirObvious):
|
||
(std::testFullVdirToVdirObviousBackwards):
|
||
(std::testFullVdirToVdirOpportunistic):
|
||
(std::testFullVdirToVdirOpportunisticBackwards):
|
||
(std::testFullVdirToVdirNewAllocation):
|
||
(std::testFullVdirToVdirNewLateAllocation):
|
||
(std::testFullVdirToVdirNewDirAllocation):
|
||
(std::testFullVdirToVdirNewLateDirAllocation):
|
||
(std::testFullVdirToVdirNewLargeAllocation):
|
||
(std::testFullVdirToVdirNewLateLargeAllocation):
|
||
(std::testFullVdirToDir):
|
||
(std::testFullVdirToDirBackwardsTarget):
|
||
(std::testFullVdirToDirBackwardsSource):
|
||
(std::testFullVdirToDirNewAllocation):
|
||
(std::testFullVdirToDirNewLateAllocation):
|
||
(std::testFullVdirToDirNewDirAllocation):
|
||
(std::testFullVdirToDirNewLateDirAllocation):
|
||
(std::testFullVdirToDirNewLargeAllocation):
|
||
(std::testFullNotVdirButLargeToDirNewLargeAllocation):
|
||
(std::testFullVdirToDirNewLateLargeAllocation):
|
||
(std::testFullVdirToDirNewAllocationAlsoPhysical):
|
||
(std::testFullVdirToDirNewLateAllocationAlsoPhysical):
|
||
(std::testFullVdirToLarge):
|
||
(std::testFullVdirToLargeBackwardsTarget):
|
||
(std::testFullVdirToLargeBackwardsSource):
|
||
(std::testFullVdirToLargeNewAllocation):
|
||
(std::testFullVdirToLargeNewLateAllocation):
|
||
(std::testFullVdirToLargeNewDirAllocation):
|
||
(std::testFullVdirToLargeNewLateDirAllocation):
|
||
(std::testFullVdirToLargeNewLargeAllocation):
|
||
(std::testFullNotVdirButLargeToLargeNewLargeAllocation):
|
||
(std::testFullVdirToLargeNewLateLargeAllocation):
|
||
(std::testFullVdirToLargeNewAllocationAlsoPhysical):
|
||
(std::testFullVdirToLargeNewLateAllocationAlsoPhysical):
|
||
(std::testFullDirToVdir):
|
||
(std::testFullDirToVdirBackwards):
|
||
(std::testFullDirToVdirNewAllocation):
|
||
(std::testFullDirToVdirNewLateAllocation):
|
||
(std::testFullDirToDir):
|
||
(std::testFullDirToDirBackwards):
|
||
(std::testFullDirToDirWithThree):
|
||
(std::testFullDirToDirWithThreeBackwards):
|
||
(std::testFullDirToDirWithThreeNewAllocation):
|
||
(std::testFullDirToDirWithThreeNewLateAllocation):
|
||
(std::testFullDirToDirWithThreeNewVdirAllocation):
|
||
(std::testFullDirToDirWithThreeNewLateVdirAllocation):
|
||
(std::testFullDirToLarge):
|
||
(std::testFullDirToLargeNewAllocation):
|
||
(std::testFullDirToLargeNewLateAllocation):
|
||
(std::testFullDirToLargeNewVdirAllocation):
|
||
(std::testFullDirToLargeNewLateVdirAllocation):
|
||
(std::testFullLargeToVdirForwardMinEpoch):
|
||
(std::testFullNotLargeButDirToVdirCombinedUseEpoch):
|
||
(std::testFullLargeToVdirCombinedUseEpoch):
|
||
(std::testFullLargeToVdirBackwards):
|
||
(std::testFullLargeToVdirNewAllocation):
|
||
(std::testFullLargeToVdirNewLateAllocation):
|
||
(std::testFullLargeToVdirNewDirAllocationForwardMinEpoch):
|
||
(std::testFullLargeToVdirNewDirAllocationCombinedUseEpoch):
|
||
(std::testFullLargeToVdirNewLateDirAllocationForwardMinEpoch):
|
||
(std::testFullLargeToVdirNewLateDirAllocationCombinedUseEpoch):
|
||
(std::testFullLargeToDirForwardMinEpoch):
|
||
(std::testFullLargeToDirCombinedUseEpoch):
|
||
(std::testFullLargeToDirBackwardsSource):
|
||
(std::testFullLargeToDirBackwardsTarget):
|
||
(std::testFullLargeToDirBackwardsSourceAndTarget):
|
||
(std::testFullLargeToDirNewAllocation):
|
||
(std::testFullLargeToDirNewLateAllocation):
|
||
(std::testFullLargeToDirNewVdirAllocationForwardMinEpoch):
|
||
(std::testFullLargeToDirNewVdirAllocationCombinedUseEpoch):
|
||
(std::testFullLargeToDirNewLateVdirAllocationForwardMinEpoch):
|
||
(std::testFullLargeToDirNewLateVdirAllocationCombinedUseEpoch):
|
||
(std::testFullLargeToDirNewDirAllocationForwardMinEpoch):
|
||
(std::testFullLargeToDirNewDirAllocationCombinedUseEpoch):
|
||
(std::testFullLargeToDirNewLateDirAllocationForwardMinEpoch):
|
||
(std::testFullLargeToDirNewLateDirAllocationCombinedUseEpoch):
|
||
(std::testFullLargeToLargeForwardMinEpoch):
|
||
(std::testFullLargeToLargeCombinedUseEpoch):
|
||
(std::testFullLargeToLargeReverse):
|
||
(std::testFullLargeToLargeNewAllocation):
|
||
(std::testFullLargeToLargeNewLateAllocation):
|
||
(std::testFullLargeToLargeNewVdirAllocation):
|
||
(std::testFullLargeToLargeNewLateVdirAllocation):
|
||
(std::testFullLargeToLargeNewDirAllocation):
|
||
(std::testFullLargeToLargeNewLateDirAllocation):
|
||
(std::testNewEligibleHasOlderEpoch):
|
||
(std::scavengerCompletionCallback):
|
||
(std::testScavengerEventuallyReturnsMemory):
|
||
(std::testScavengerEventuallyReturnsMemoryEvenWithoutManualShrink):
|
||
(std::scavengerShutDownCallback):
|
||
(std::testScavengerShutsDownEventually):
|
||
(std::addAllTests):
|
||
(addIsoHeapPageSharingTests):
|
||
* libpas/src/test/IsoHeapPartialAndBaselineTests.cpp: Added.
|
||
(std::FreeOrder::setCount const):
|
||
(std::FreeOrder::didSetCount const):
|
||
(std::forEachSharedPageDirectoryCallbackAdaptor):
|
||
(std::forEachSharedPageDirectory):
|
||
(std::numSharedPages):
|
||
(std::numCommittedSharedPages):
|
||
(std::testSimplePartialAllocations):
|
||
(std::testFreeAroundPrimordialStop):
|
||
(std::testFreeInterleavedAroundPrimordialStop):
|
||
(std::PartialProgram::PartialProgram):
|
||
(std::testMultiplePartialsFromDifferentHeapsPerShared):
|
||
(std::addMultiplePartialsFromDifferentHeapsPerSharedTests):
|
||
(std::testMultiplePartialsFromDifferentThreadsPerShared):
|
||
(std::incrementalRandom):
|
||
(std::zeroRandom):
|
||
(std::testTwoBaselinesEvictions):
|
||
(std::addScavengerDependentTests):
|
||
(addIsoHeapPartialAndBaselineTests):
|
||
* libpas/src/test/IsoHeapReservedMemoryTests.cpp: Added.
|
||
(std::testSizeProgression):
|
||
(addIsoHeapReservedMemoryTests):
|
||
* libpas/src/test/IsoHeapTablingTests.cpp: Added.
|
||
(std::testTabling):
|
||
(std::testUntabling):
|
||
(std::addTablingTests):
|
||
(addIsoHeapTablingTests):
|
||
* libpas/src/test/JITHeapTests.cpp: Added.
|
||
(std::testAllocateShrinkAndAllocate):
|
||
(addJITHeapTests):
|
||
* libpas/src/test/LargeFreeHeapTests.cpp: Added.
|
||
(std::alignSimple):
|
||
(std::trappingAllocator):
|
||
(std::trappingDeallocator):
|
||
(std::failingAllocator):
|
||
(std::Action::allocate):
|
||
(std::Action::deallocate):
|
||
(std::Free::Free):
|
||
(std::Free::operator== const):
|
||
(std::Free::operator< const):
|
||
(std::Allocation::Allocation):
|
||
(std::function<pas_aligned_allocation_result):
|
||
(std::function<void):
|
||
(std::operator<<):
|
||
(std::allocatorAdapter):
|
||
(std::deallocatorAdapter):
|
||
(std::iterateAdapter):
|
||
(std::iterateHeap):
|
||
(std::testLargeFreeHeapImpl):
|
||
(std::testSimpleLargeFreeHeap):
|
||
(std::testFastLargeFreeHeap):
|
||
(std::testBootstrapHeap):
|
||
(std::freeListSize):
|
||
(addLargeFreeHeapTests):
|
||
* libpas/src/test/LargeSharingPoolDump.cpp: Copied from Source/WTF/wtf/FastTLS.h.
|
||
(std::forEachAdapter):
|
||
(forEachLargeSharingPoolNode):
|
||
(dumpLargeSharingPool):
|
||
* libpas/src/test/LargeSharingPoolDump.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
* libpas/src/test/LargeSharingPoolTests.cpp: Added.
|
||
(std::Range::Range):
|
||
(std::assertState):
|
||
(std::testBadCoalesceEpochUpdate):
|
||
(addLargeSharingPoolTests):
|
||
* libpas/src/test/LockFreeReadPtrPtrHashtableTests.cpp: Added.
|
||
(std::hashFunction):
|
||
(std::testChaos):
|
||
(addLockFreeReadPtrPtrHashtableTests):
|
||
* libpas/src/test/MinHeapTests.cpp: Added.
|
||
(std::Element::Element):
|
||
(std::asChar):
|
||
(std::compare):
|
||
(std::getIndex):
|
||
(std::setIndex):
|
||
(std::dumpHeap):
|
||
(std::testMinHeap):
|
||
(std::minHeapChaos):
|
||
(std::testMinHeapChaos):
|
||
(std::makeBootstrapAllocationConfig):
|
||
(std::testBootstrapMinHeapChaos):
|
||
(addMinHeapTests):
|
||
* libpas/src/test/RaceTests.cpp: Added.
|
||
(std::hookCallbackAdapter):
|
||
(std::willLockCallback):
|
||
(std::recordLockAcquisition):
|
||
(std::didLockCallback):
|
||
(std::didTryLockCallback):
|
||
(std::willUnlockCallback):
|
||
(std::isHoldingContendedLocks):
|
||
(std::InstallRaceHooks::InstallRaceHooks):
|
||
(std::testLocalAllocatorStopRace):
|
||
(std::testLocalAllocatorStopRaceAgainstScavenge):
|
||
(addRaceTests):
|
||
* libpas/src/test/RedBlackTreeTests.cpp: Added.
|
||
(std::TestNode::TestNode):
|
||
(std::TestNode::compare):
|
||
(std::TestNode::compareKey):
|
||
(std::Pair::Pair):
|
||
(std::Pair::operator== const):
|
||
(std::Pair::operator< const):
|
||
(std::findExact):
|
||
(std::remove):
|
||
(std::findLeastGreaterThanOrEqual):
|
||
(std::assertFoundAndRemove):
|
||
(std::assertEqual):
|
||
(std::assertSameValuesForKey):
|
||
(std::push):
|
||
(std::validateEnumerable):
|
||
(std::testDriver):
|
||
(std::randomTestDriver):
|
||
(addRedBlackTreeTests):
|
||
* libpas/src/test/SkipListTests.cpp: Added.
|
||
(std::TestNode::TestNode):
|
||
(std::TestNode::fromSkipListNode):
|
||
(std::TestNode::compareKey):
|
||
(std::TestNode::compareKeyLeastGreaterThanOrEqual):
|
||
(std::TestNode::compareKeyForInsert):
|
||
(std::TestNode::compare):
|
||
(std::Pair::Pair):
|
||
(std::Pair::operator== const):
|
||
(std::Pair::operator< const):
|
||
(std::findExact):
|
||
(std::remove):
|
||
(std::findLeastGreaterThanOrEqual):
|
||
(std::assertFoundAndRemove):
|
||
(std::assertEqual):
|
||
(std::assertSameValuesForKey):
|
||
(std::dumpSkipList):
|
||
(std::dumpVector):
|
||
(std::testDriver):
|
||
(std::randomTestDriver):
|
||
(addSkipListTests):
|
||
* libpas/src/test/SuspendScavenger.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
(SuspendScavenger::SuspendScavenger):
|
||
(SuspendScavenger::~SuspendScavenger):
|
||
* libpas/src/test/TSDTests.cpp: Added.
|
||
(std::destructor):
|
||
(std::testTSD):
|
||
(addTSDTests):
|
||
* libpas/src/test/TestHarness.cpp: Added.
|
||
(allocationConfigAllocate):
|
||
(allocationConfigDeallocate):
|
||
(TestScopeImpl::TestScopeImpl):
|
||
(RuntimeConfigTestScope::RuntimeConfigTestScope):
|
||
(ForceExclusives::ForceExclusives):
|
||
(ForceTLAs::ForceTLAs):
|
||
(ForceBitfit::ForceBitfit):
|
||
(DisableBitfit::DisableBitfit):
|
||
(ForcePartials::ForcePartials):
|
||
(ForceBaselines::ForceBaselines):
|
||
(RunScavengerFully::RunScavengerFully):
|
||
(RunScavengerOnNonRemoteCaches::RunScavengerOnNonRemoteCaches):
|
||
(SuspendScavengerScope::SuspendScavengerScope):
|
||
(VerifyGranules::VerifyGranules):
|
||
(InstallVerifier::InstallVerifier):
|
||
(DisableExplosion::DisableExplosion):
|
||
(ForceExplosion::ForceExplosion):
|
||
(ForceOneMagazine::ForceOneMagazine):
|
||
(EpochIsCounter::EpochIsCounter):
|
||
(BootJITHeap::BootJITHeap):
|
||
(testSucceeded):
|
||
(deterministicRandomNumber):
|
||
(addTest):
|
||
(TestScope::TestScope):
|
||
(TestScope::~TestScope):
|
||
(hasScope):
|
||
(dumpObjectSet):
|
||
(dumpObjectsInHeap):
|
||
(dumpObjectsInHeaps):
|
||
(forEachLiveObject):
|
||
(verifyMinimumObjectDistance):
|
||
(verifyExactObjectDistance):
|
||
(runOneTest):
|
||
(runForkedTest):
|
||
(runTests):
|
||
(runFilteredTests):
|
||
(main):
|
||
(scavenge):
|
||
(printStatusReport):
|
||
* libpas/src/test/TestHarness.h: Added.
|
||
(stringStreamConstruct):
|
||
(dumpToString):
|
||
(operator<<):
|
||
* libpas/src/test/ThingyAndUtilityHeapAllocationTests.cpp: Added.
|
||
(std::flushDeallocationLog):
|
||
(std::flushDeallocationLogAndStopAllocators):
|
||
(std::sizeClassFor):
|
||
(std::isLarge):
|
||
(std::forEachLiveUtilityObjectAdapter):
|
||
(std::forEachLiveUtilityObject):
|
||
(std::forEachCommittedViewAdapter):
|
||
(std::forEachCommittedView):
|
||
(std::numCommittedViews):
|
||
(std::numViews):
|
||
(std::verifyMinimumObjectDistance):
|
||
(std::verifyObjectSet):
|
||
(std::verifyUtilityObjectSet):
|
||
(std::verifyHeapEmpty):
|
||
(std::PrimitiveAllocator::PrimitiveAllocator):
|
||
(std::PrimitiveReallocAllocator::PrimitiveReallocAllocator):
|
||
(std::AlignedPrimitiveAllocator::AlignedPrimitiveAllocator):
|
||
(std::createIsolatedHeapRef):
|
||
(std::IsolatedHeapAllocator::IsolatedHeapAllocator):
|
||
(std::IsolatedHeapArrayAllocator::IsolatedHeapArrayAllocator):
|
||
(std::CounterScope::CounterScope):
|
||
(std::CounterScope::~CounterScope):
|
||
(std::testDeallocateNull):
|
||
(std::deallocationFailureCallback):
|
||
(std::DeallocationShouldFail::DeallocationShouldFail):
|
||
(std::DeallocationShouldFail::~DeallocationShouldFail):
|
||
(std::testDeallocateMalloc):
|
||
(std::testDeallocateStack):
|
||
(std::testSimpleAllocation):
|
||
(std::testSmallOrMediumAllocation):
|
||
(std::testSmallAllocation):
|
||
(std::testLargeAllocation):
|
||
(std::testAllocationWithInterleavedFragmentation):
|
||
(std::testFreeListRefillSpans):
|
||
(std::testInternalScavenge):
|
||
(std::testInternalScavengeFromCorrectDirectory):
|
||
(std::SizeClassProgram::SizeClassProgram):
|
||
(std::testSizeClassCreationImpl):
|
||
(std::testSizeClassCreation):
|
||
(std::testSpuriousEligibility):
|
||
(std::testBasicSizeClassNotSet):
|
||
(std::testSmallDoubleFree):
|
||
(std::testSmallFreeInner):
|
||
(std::testSmallFreeNextWithShrink):
|
||
(std::testSmallFreeNextWithoutShrink):
|
||
(std::testSmallFreeNextBeforeShrink):
|
||
(std::AllocationProgram::allocate):
|
||
(std::AllocationProgram::free):
|
||
(std::AllocationProgram::kind const):
|
||
(std::AllocationProgram::key const):
|
||
(std::AllocationProgram::isAllocate const):
|
||
(std::AllocationProgram::count const):
|
||
(std::AllocationProgram::alignment const):
|
||
(std::AllocationProgram::isFree const):
|
||
(std::IsolatedComplexAllocator::IsolatedComplexAllocator):
|
||
(std::IsolatedUnitComplexAllocator::IsolatedUnitComplexAllocator):
|
||
(std::checkObjectDistances):
|
||
(std::checkObjectBeginnings):
|
||
(std::addObjectAllocation):
|
||
(std::ExpectedBytes::exact):
|
||
(std::ExpectedBytes::upperBound):
|
||
(std::ExpectedBytes::check):
|
||
(std::testComplexLargeAllocationImpl):
|
||
(std::testComplexLargeAllocation):
|
||
(std::testAllocationCountProgression):
|
||
(std::testAllocationChaos):
|
||
(std::testUtilityAllocationChaos):
|
||
(std::testCombinedAllocationChaos):
|
||
(std::testLargeDoubleFree):
|
||
(std::testLargeOffsetFree):
|
||
(std::addDeallocationTests):
|
||
(std::testReallocatePrimitive):
|
||
(std::testReallocateArray):
|
||
(std::addSmallHeapTests):
|
||
(std::addLargeHeapTests):
|
||
(std::addMediumHeapTests):
|
||
(std::addLargerThanMediumHeapTests):
|
||
(std::addMargeBitfitTests):
|
||
(std::addLargerThanMargeBitfitTests):
|
||
(std::addCombinedHeapTests):
|
||
(std::addAllTestsImpl):
|
||
(std::addAllTests):
|
||
(addThingyAndUtilityHeapAllocationTests):
|
||
* libpas/src/test/UtilsTests.cpp: Added.
|
||
(std::testIsDivisibleBy3):
|
||
(addUtilsTests):
|
||
* libpas/src/toys/QuadraticSizeClass.cpp: Added.
|
||
(std::printSizeCell):
|
||
(std::printProgression):
|
||
(main):
|
||
* libpas/src/verifier/Verifier.cpp: Added.
|
||
(std::Locker::Locker):
|
||
(std::Locker::~Locker):
|
||
(std::Allocation::Allocation):
|
||
(std::allocationCallback):
|
||
(std::deallocationCallback):
|
||
(std::dumpStateForHeapKind):
|
||
(std::dumpStateHoldingLock):
|
||
(std::uninstall):
|
||
(pas_install_verifier):
|
||
(pas_dump_state):
|
||
* libpas/src/verifier/Verifier.h: Copied from Source/WTF/wtf/FastTLS.h.
|
||
* libpas/test-impl.sh: Added.
|
||
* libpas/test.sh: Added.
|
||
|
||
2021-07-06 Michael Saboff <msaboff@apple.com>
|
||
|
||
[bmalloc] Make adaptive scavenging more precise
|
||
https://bugs.webkit.org/show_bug.cgi?id=226237
|
||
|
||
Reviewed by Geoffrey Garen.
|
||
|
||
Reland the adaptive scavenger change for macOS with fix.
|
||
|
||
The bug happens when decommitting large ranges that don't have physical pages.
|
||
We'd call Heap::decommitLargeRange(), but would only add the range to the
|
||
decommitter list if there were physical pages associated with the range.
|
||
We would still perform all the other processing of a decommitted range,
|
||
including setting the range as not elgible for allocation or merging.
|
||
Had the range been added to the decommitter list, we would have set the
|
||
range as elgible after the physical pages were released to the OS.
|
||
The result is that the range could never be allocated, either by itself or as a
|
||
larger range merged with adjacent ranges.
|
||
|
||
The fix is to only perform decommit processing of large ranges if they have
|
||
physical pages. We now check for physical pages before calling Heap::decommitLargeRange().
|
||
For ranges that don't have physical pages, they can stay on the free list as
|
||
elgible without having to round trip through decommit processing.
|
||
|
||
Made a minor change to the calculation of the physical end of the LargeRange created
|
||
and added to the free list in Heap::deallocateSmallChunk. If the last page in the chunk
|
||
has a physical page, we set the physical end of the range to the end of the chunk.
|
||
This is for the case where there is an unusable partial small page at the end of the chunk.
|
||
|
||
* bmalloc/BPlatform.h:
|
||
* bmalloc/Heap.cpp:
|
||
(bmalloc::Heap::decommitLargeRange):
|
||
(bmalloc::Heap::scavenge):
|
||
(bmalloc::Heap::allocateSmallChunk):
|
||
(bmalloc::Heap::deallocateSmallChunk):
|
||
(bmalloc::Heap::allocateSmallPage):
|
||
(bmalloc::Heap::splitAndAllocate):
|
||
(bmalloc::Heap::allocateLarge):
|
||
(bmalloc::Heap::tryAllocateLargeChunk):
|
||
(bmalloc::Heap::shrinkLarge):
|
||
(bmalloc::Heap::deallocateLarge):
|
||
(bmalloc::Heap::scavengeToHighWatermark): Deleted.
|
||
* bmalloc/Heap.h:
|
||
* bmalloc/IsoDirectory.h:
|
||
* bmalloc/IsoDirectoryInlines.h:
|
||
(bmalloc::passedNumPages>::takeFirstEligible):
|
||
(bmalloc::passedNumPages>::scavenge):
|
||
(bmalloc::passedNumPages>::scavengeToHighWatermark): Deleted.
|
||
* bmalloc/IsoHeapImpl.h:
|
||
* bmalloc/IsoHeapImplInlines.h:
|
||
(bmalloc::IsoHeapImpl<Config>::scavengeToHighWatermark): Deleted.
|
||
* bmalloc/IsoSharedHeapInlines.h:
|
||
(bmalloc::IsoSharedHeap::allocateSlow):
|
||
* bmalloc/LargeMap.cpp:
|
||
(bmalloc::LargeMap::add):
|
||
* bmalloc/LargeRange.h:
|
||
(bmalloc::LargeRange::LargeRange):
|
||
(bmalloc::LargeRange::physicalEnd const):
|
||
(bmalloc::LargeRange::setPhysicalEnd):
|
||
(bmalloc::LargeRange::clearPhysicalEnd):
|
||
(bmalloc::LargeRange::setUsedSinceLastScavenge):
|
||
(bmalloc::merge):
|
||
(bmalloc::LargeRange::split const):
|
||
(): Deleted.
|
||
* bmalloc/Scavenger.cpp:
|
||
(bmalloc::Scavenger::Scavenger):
|
||
(bmalloc::Scavenger::scheduleIfUnderMemoryPressure):
|
||
(bmalloc::Scavenger::schedule):
|
||
(bmalloc::Scavenger::scavenge):
|
||
(bmalloc::Scavenger::threadRunLoop):
|
||
(bmalloc::Scavenger::didStartGrowing): Deleted.
|
||
(bmalloc::Scavenger::timeSinceLastPartialScavenge): Deleted.
|
||
(bmalloc::Scavenger::partialScavenge): Deleted.
|
||
* bmalloc/Scavenger.h:
|
||
* bmalloc/SmallPage.h:
|
||
(bmalloc::SmallPage::setUsedSinceLastScavenge):
|
||
|
||
2021-06-23 Mark Lam <mark.lam@apple.com>
|
||
|
||
Base Options::useWebAssemblyFastMemory's default value on Gigacage::hasCapacityToUseLargeGigacage.
|
||
https://bugs.webkit.org/show_bug.cgi?id=227328
|
||
|
||
Reviewed by Saam Barati.
|
||
|
||
Rename Gigacage::useLargeGigacage to Gigacage::hasCapacityToUseLargeGigacage.
|
||
|
||
* bmalloc/Gigacage.h:
|
||
|
||
2021-06-21 Kimmo Kinnunen <kkinnunen@apple.com>
|
||
|
||
makeUnique cannot be used to instantiate function-local classes
|
||
https://bugs.webkit.org/show_bug.cgi?id=227163
|
||
|
||
Reviewed by Antti Koivisto.
|
||
|
||
Make WTF_MAKE_FAST_ALLOCATED and similar macros work in function
|
||
local classes. Mark the typedef that is used to enforce semicolon
|
||
after the macro as unused to avoid unused typedef warning.
|
||
Fixes cases where the compiler is able to prove that it sees all the
|
||
use sites of the class and the typedef is not used.
|
||
|
||
* bmalloc/BCompiler.h:
|
||
* bmalloc/BMalloced.h:
|
||
* bmalloc/IsoHeap.h:
|
||
* bmalloc/IsoHeapInlines.h:
|
||
|
||
2021-06-17 Mark Lam <mark.lam@apple.com>
|
||
|
||
Define MacroAssemblerARM64E::numberOfPACBits based on OS_CONSTANT(EFFECTIVE_ADDRESS_WIDTH).
|
||
https://bugs.webkit.org/show_bug.cgi?id=227147
|
||
rdar://78785309
|
||
|
||
Reviewed by Saam Barati.
|
||
|
||
For OS(DARWIN), define BOS_EFFECTIVE_ADDRESS_WIDTH in terms of MACH_VM_MAX_ADDRESS,
|
||
which is provided by the SDK. This ensures that it is correct for each target
|
||
OS(DARWIN) platform.
|
||
|
||
* bmalloc/Algorithm.h:
|
||
(bmalloc::clzConstexpr):
|
||
(bmalloc::getMSBSetConstexpr):
|
||
* bmalloc/BPlatform.h:
|
||
* bmalloc/Gigacage.h:
|
||
* bmalloc/ObjectTypeTable.h:
|
||
* bmalloc/Packed.h:
|
||
|
||
2021-06-03 Michael Saboff <msaboff@apple.com>
|
||
|
||
Unreviewed, rolling out r278278.
|
||
https://bugs.webkit.org/show_bug.cgi?id=226237
|
||
|
||
Made some JSC mini mode and other tests flakey.
|
||
|
||
Reverted changeset:
|
||
|
||
[bmalloc] Make adaptive scavenging more precise
|
||
https://bugs.webkit.org/show_bug.cgi?id=226237
|
||
|
||
* bmalloc/BPlatform.h:
|
||
* bmalloc/Heap.cpp:
|
||
(bmalloc::Heap::decommitLargeRange):
|
||
(bmalloc::Heap::scavenge):
|
||
(bmalloc::Heap::scavengeToHighWatermark):
|
||
(bmalloc::Heap::allocateSmallChunk):
|
||
(bmalloc::Heap::deallocateSmallChunk):
|
||
(bmalloc::Heap::allocateSmallPage):
|
||
(bmalloc::Heap::splitAndAllocate):
|
||
(bmalloc::Heap::allocateLarge):
|
||
(bmalloc::Heap::tryAllocateLargeChunk):
|
||
(bmalloc::Heap::shrinkLarge):
|
||
(bmalloc::Heap::deallocateLarge):
|
||
* bmalloc/Heap.h:
|
||
* bmalloc/IsoDirectory.h:
|
||
* bmalloc/IsoDirectoryInlines.h:
|
||
(bmalloc::passedNumPages>::takeFirstEligible):
|
||
(bmalloc::passedNumPages>::scavenge):
|
||
(bmalloc::passedNumPages>::scavengeToHighWatermark):
|
||
* bmalloc/IsoHeapImpl.h:
|
||
* bmalloc/IsoHeapImplInlines.h:
|
||
(bmalloc::IsoHeapImpl<Config>::scavengeToHighWatermark):
|
||
* bmalloc/IsoSharedHeapInlines.h:
|
||
(bmalloc::IsoSharedHeap::allocateSlow):
|
||
* bmalloc/LargeMap.cpp:
|
||
(bmalloc::LargeMap::add):
|
||
* bmalloc/LargeRange.h:
|
||
(bmalloc::LargeRange::LargeRange):
|
||
(bmalloc::LargeRange::setTotalPhysicalSize):
|
||
(bmalloc::merge):
|
||
(bmalloc::LargeRange::split const):
|
||
(bmalloc::LargeRange::physicalEnd const): Deleted.
|
||
(bmalloc::LargeRange::setPhysicalEnd): Deleted.
|
||
(bmalloc::LargeRange::clearPhysicalEnd): Deleted.
|
||
* bmalloc/Scavenger.cpp:
|
||
(bmalloc::Scavenger::Scavenger):
|
||
(bmalloc::Scavenger::didStartGrowing):
|
||
(bmalloc::Scavenger::scheduleIfUnderMemoryPressure):
|
||
(bmalloc::Scavenger::schedule):
|
||
(bmalloc::Scavenger::timeSinceLastPartialScavenge):
|
||
(bmalloc::Scavenger::scavenge):
|
||
(bmalloc::Scavenger::partialScavenge):
|
||
(bmalloc::Scavenger::threadRunLoop):
|
||
* bmalloc/Scavenger.h:
|
||
* bmalloc/SmallPage.h:
|
||
|
||
2021-05-31 Michael Saboff <msaboff@apple.com>
|
||
|
||
[bmalloc] Make adaptive scavenging more precise
|
||
https://bugs.webkit.org/show_bug.cgi?id=226237
|
||
|
||
Reviewed by Geoffrey Garen.
|
||
|
||
This patch re-enables the adaptive scavenger for macOS.
|
||
It is much more precise when calling madvise() by keeping track of the
|
||
maximum extent of physically mapped memory in a LargeRange. For example,
|
||
in the prior code, we'd return a small range back to a Gigacage LargeRange
|
||
whithout any physical pages. When we scavenge that LargeRange, we madvise()
|
||
the whole Gigacage range. Although this didn't cause correctness issues,
|
||
it was quite wasteful.
|
||
|
||
In the tests I did with this patch compared to the earlier adaptive change,
|
||
the number of madvise calls we made drops by ~18% and the time spent in those
|
||
calls dropped ~13% on an Apple silicon mac and ~30% on an x86 mac.
|
||
This was measured using dtrace running JetStream2 from the command line.
|
||
RAMification results improved ~1% over the adptive change.
|
||
|
||
There is the possible future optimization where we also keep track of the
|
||
first address of physically mapped memory in a LargeRange. Since bmalloc
|
||
allocates memory from lower addresses first, it is thought that the change
|
||
in this patch is sufficient to reduce not only the number of madvise calls,
|
||
but the time it takes to make those calls.
|
||
|
||
* bmalloc/BPlatform.h:
|
||
* bmalloc/Heap.cpp:
|
||
(bmalloc::Heap::decommitLargeRange):
|
||
(bmalloc::Heap::scavenge):
|
||
(bmalloc::Heap::allocateSmallChunk):
|
||
(bmalloc::Heap::deallocateSmallChunk):
|
||
(bmalloc::Heap::allocateSmallPage):
|
||
(bmalloc::Heap::splitAndAllocate):
|
||
(bmalloc::Heap::allocateLarge):
|
||
(bmalloc::Heap::tryAllocateLargeChunk):
|
||
(bmalloc::Heap::shrinkLarge):
|
||
(bmalloc::Heap::deallocateLarge):
|
||
(bmalloc::Heap::scavengeToHighWatermark): Deleted.
|
||
* bmalloc/Heap.h:
|
||
* bmalloc/IsoDirectory.h:
|
||
* bmalloc/IsoDirectoryInlines.h:
|
||
(bmalloc::passedNumPages>::takeFirstEligible):
|
||
(bmalloc::passedNumPages>::scavenge):
|
||
(bmalloc::passedNumPages>::scavengeToHighWatermark): Deleted.
|
||
* bmalloc/IsoHeapImpl.h:
|
||
* bmalloc/IsoHeapImplInlines.h:
|
||
(bmalloc::IsoHeapImpl<Config>::scavengeToHighWatermark): Deleted.
|
||
* bmalloc/LargeMap.cpp:
|
||
(bmalloc::LargeMap::add):
|
||
* bmalloc/LargeRange.h:
|
||
(bmalloc::LargeRange::LargeRange):
|
||
(bmalloc::LargeRange::physicalExtent const):
|
||
(bmalloc::LargeRange::setPhysicalExtent):
|
||
(bmalloc::LargeRange::resetPhysicalExtent):
|
||
(bmalloc::LargeRange::setUsedSinceLastScavenge):
|
||
(bmalloc::merge):
|
||
(bmalloc::LargeRange::split const):
|
||
(): Deleted.
|
||
* bmalloc/Scavenger.cpp:
|
||
(bmalloc::Scavenger::Scavenger):
|
||
(bmalloc::Scavenger::scavenge):
|
||
(bmalloc::Scavenger::threadRunLoop):
|
||
(bmalloc::Scavenger::timeSinceLastPartialScavenge): Deleted.
|
||
(bmalloc::Scavenger::partialScavenge): Deleted.
|
||
* bmalloc/Scavenger.h:
|
||
* bmalloc/SmallPage.h:
|
||
(bmalloc::SmallPage::setUsedSinceLastScavenge):
|
||
|
||
2021-05-21 Michael Saboff <msaboff@apple.com>
|
||
|
||
[bmalloc] Rollout r276266 because WebKit processes are spending much more time in madvise
|
||
https://bugs.webkit.org/show_bug.cgi?id=226122
|
||
|
||
Unreviewed rollout.
|
||
|
||
Rolling out r276266 to do some automated testing. At the same time, we'll work on changing the madvise() decommitting to be more precise.
|
||
|
||
* bmalloc/BPlatform.h:
|
||
* bmalloc/Heap.cpp:
|
||
(bmalloc::Heap::scavenge):
|
||
(bmalloc::Heap::scavengeToHighWatermark):
|
||
(bmalloc::Heap::allocateSmallChunk):
|
||
(bmalloc::Heap::allocateSmallPage):
|
||
(bmalloc::Heap::allocateLarge):
|
||
* bmalloc/Heap.h:
|
||
* bmalloc/IsoDirectory.h:
|
||
* bmalloc/IsoDirectoryInlines.h:
|
||
(bmalloc::passedNumPages>::takeFirstEligible):
|
||
(bmalloc::passedNumPages>::scavenge):
|
||
(bmalloc::passedNumPages>::scavengeToHighWatermark):
|
||
* bmalloc/IsoHeapImpl.h:
|
||
* bmalloc/IsoHeapImplInlines.h:
|
||
(bmalloc::IsoHeapImpl<Config>::scavengeToHighWatermark):
|
||
* bmalloc/LargeMap.cpp:
|
||
(bmalloc::LargeMap::add):
|
||
* bmalloc/LargeRange.h:
|
||
(bmalloc::LargeRange::LargeRange):
|
||
(bmalloc::merge):
|
||
* bmalloc/Scavenger.cpp:
|
||
(bmalloc::Scavenger::Scavenger):
|
||
(bmalloc::Scavenger::timeSinceLastPartialScavenge):
|
||
(bmalloc::Scavenger::scavenge):
|
||
(bmalloc::Scavenger::partialScavenge):
|
||
(bmalloc::Scavenger::threadRunLoop):
|
||
* bmalloc/Scavenger.h:
|
||
* bmalloc/SmallPage.h:
|
||
|
||
2021-04-19 Michael Saboff <msaboff@apple.com>
|
||
|
||
[bmalloc] Enable Adaptive Scavenger for Mac
|
||
https://bugs.webkit.org/show_bug.cgi?id=224706
|
||
|
||
Reviewed by Filip Pizlo.
|
||
|
||
Enabled the adaptive scavenger code paths for macOS.
|
||
The original reason that the partial scavenging paths were kept for macOS was due
|
||
to regression on power tests. To alleviate the power regression, this patch splits
|
||
out the adaptive scavenger parameters with macOS specific values.
|
||
|
||
The parameters are:
|
||
The multiplier used to compute the next scavenging wait time based on the
|
||
time needed for the prior scavenging.
|
||
Minimum wait time between scavenging.
|
||
Maximum wait time between scavenging.
|
||
|
||
The values in the current code are:
|
||
Wait time Multiplier: 150
|
||
Minimum wait time: 100ms
|
||
Maximum wait time: 10,000ms (10 seconds)
|
||
|
||
The proposed values for macOS, determined using empirical testing.
|
||
Wait time Multiplier: 300
|
||
Minimum wait time: 750ms
|
||
Maximum wait time: 20,000ms (20 seconds)
|
||
|
||
When tested on various mac variants, this change:
|
||
* Provides a 3-5% reduction in memory use on RAMification.
|
||
* It is neutral on JetStream2.
|
||
* It is neutral to a slight regression on Speedometer2, but there is some
|
||
variability in those results.
|
||
|
||
Since macOS was the only platform still using the partial scavenging code path,
|
||
the partial scavenging code paths were deleted.
|
||
|
||
* bmalloc/BPlatform.h:
|
||
* bmalloc/Heap.cpp:
|
||
(bmalloc::Heap::scavenge):
|
||
(bmalloc::Heap::allocateSmallChunk):
|
||
(bmalloc::Heap::allocateSmallPage):
|
||
(bmalloc::Heap::allocateLarge):
|
||
(bmalloc::Heap::scavengeToHighWatermark): Deleted.
|
||
* bmalloc/Heap.h:
|
||
* bmalloc/IsoDirectory.h:
|
||
* bmalloc/IsoDirectoryInlines.h:
|
||
(bmalloc::passedNumPages>::takeFirstEligible):
|
||
(bmalloc::passedNumPages>::scavenge):
|
||
(bmalloc::passedNumPages>::scavengeToHighWatermark): Deleted.
|
||
* bmalloc/IsoHeapImpl.h:
|
||
* bmalloc/IsoHeapImplInlines.h:
|
||
(bmalloc::IsoHeapImpl<Config>::scavengeToHighWatermark): Deleted.
|
||
* bmalloc/LargeMap.cpp:
|
||
(bmalloc::LargeMap::add):
|
||
* bmalloc/LargeRange.h:
|
||
(bmalloc::LargeRange::LargeRange):
|
||
(bmalloc::LargeRange::setUsedSinceLastScavenge):
|
||
(bmalloc::merge):
|
||
(): Deleted.
|
||
* bmalloc/Scavenger.cpp:
|
||
(bmalloc::Scavenger::Scavenger):
|
||
(bmalloc::Scavenger::scavenge):
|
||
(bmalloc::Scavenger::threadRunLoop):
|
||
(bmalloc::Scavenger::timeSinceLastPartialScavenge): Deleted.
|
||
(bmalloc::Scavenger::partialScavenge): Deleted.
|
||
* bmalloc/Scavenger.h:
|
||
* bmalloc/SmallPage.h:
|
||
(bmalloc::SmallPage::setUsedSinceLastScavenge):
|
||
|
||
2021-04-19 Kimmo Kinnunen <kkinnunen@apple.com>
|
||
|
||
Enable -Wthread-safety, add attributes to custom lock classes, and provide macros to declare guards
|
||
https://bugs.webkit.org/show_bug.cgi?id=221614
|
||
<rdar://problem/74396781>
|
||
|
||
Reviewed by David Kilzer.
|
||
|
||
Add -Wthread-safety to compile flags.
|
||
|
||
* Configurations/Base.xcconfig:
|
||
|
||
2021-03-26 Jessie Berlin <jberlin@webkit.org>
|
||
|
||
Update the BEFORE/SINCE, SYSTEM_VERSION_PREFIX, and MACOSX_DEPLOYMENT_TARGET flags
|
||
https://bugs.webkit.org/show_bug.cgi?id=223779
|
||
|
||
Reviewed by Tim Horton.
|
||
|
||
* Configurations/DebugRelease.xcconfig:
|
||
|
||
2021-03-25 Jessie Berlin <jberlin@webkit.org>
|
||
|
||
Remove 10.13 DEPLOYMENT_TARGETs and SYSTEM_VERSION_PREFIXs
|
||
https://bugs.webkit.org/show_bug.cgi?id=223765
|
||
|
||
Reviewed by Tim Horton.
|
||
|
||
* Configurations/Base.xcconfig:
|
||
* Configurations/DebugRelease.xcconfig:
|
||
|
||
2021-02-15 Michael Saboff <msaboff@apple.com>
|
||
|
||
[ARM64] Change break instruction comment to indicate possible security failure
|
||
https://bugs.webkit.org/show_bug.cgi?id=221936
|
||
|
||
Reviewed by Mark Lam.
|
||
|
||
Change the ASSERT break comment immediate to the same value the C++ compiler uses.
|
||
|
||
* bmalloc/BAssert.h:
|
||
|
||
2020-12-07 Don Olmstead <don.olmstead@sony.com>
|
||
|
||
[CMake] Remove WEBKIT_WRAP_SOURCELIST
|
||
https://bugs.webkit.org/show_bug.cgi?id=196916
|
||
|
||
Reviewed by Michael Catanzaro.
|
||
|
||
* CMakeLists.txt:
|
||
|
||
2020-12-04 Adam Roben <aroben@apple.com>
|
||
|
||
More FALLBACK_PLATFORM adoption
|
||
https://bugs.webkit.org/show_bug.cgi?id=219545
|
||
|
||
Reviewed by Tim Horton.
|
||
|
||
* Configurations/SDKVariant.xcconfig:
|
||
WK_EMPTY_$(THIS_IS_NOT_EMPTY) evaluates to the empty string, not to
|
||
NO.
|
||
|
||
2020-12-03 Adam Roben <aroben@apple.com>
|
||
|
||
Adopt FALLBACK_PLATFORM
|
||
https://bugs.webkit.org/show_bug.cgi?id=219504
|
||
|
||
Reviewed by Tim Horton.
|
||
|
||
* Configurations/SDKVariant.xcconfig: Use FALLBACK_PLATFORM it if it's
|
||
defined, otherwise use PLATFORM_NAME as before.
|
||
|
||
2020-11-04 David Kilzer <ddkilzer@apple.com>
|
||
|
||
Remove bmalloc::logVMFailure() and BSoftLinking.h
|
||
<https://webkit.org/b/218571>
|
||
<rdar://problem/69813381>
|
||
|
||
Reviewed by Geoffrey Garen.
|
||
|
||
* PlatformMac.cmake:
|
||
* bmalloc.xcodeproj/project.pbxproj:
|
||
- Remove BSoftLinking.h from project files.
|
||
|
||
* bmalloc/Logging.cpp:
|
||
* bmalloc/Logging.h:
|
||
(bmalloc::logVMFailure): Delete.
|
||
|
||
* bmalloc/darwin/BSoftLinking.h: Remove.
|
||
|
||
2020-11-04 David Kilzer <ddkilzer@apple.com>
|
||
|
||
WebKit should remove unused debug variant support
|
||
<https://webkit.org/b/218315>
|
||
<rdar://problem/70785369>
|
||
|
||
Reviewed by Darin Adler.
|
||
|
||
Remove support for building the debug variant since it is
|
||
currently unused. We now set default values for the
|
||
DEAD_CODE_STRIPPING, DEBUG_DEFINES, GCC_OPTIMIZATION_LEVEL and
|
||
STRIP_INSTALLED_PRODUCT variables.
|
||
|
||
Also move these values out of the Xcode project into
|
||
Base.xcconfig files using the [config=Debug] specifier so that
|
||
these overrides are next to the definitions.
|
||
|
||
* Configurations/Base.xcconfig:
|
||
* bmalloc.xcodeproj/project.pbxproj:
|
||
|
||
2020-11-02 David Kilzer <ddkilzer@apple.com>
|
||
|
||
Sort Xcode project file
|
||
|
||
* bmalloc.xcodeproj/project.pbxproj:
|
||
|
||
2020-10-13 Keith Rollin <krollin@apple.com>
|
||
|
||
Remove leftover MACOSX_DEPLOYMENT_TARGET_macosx support
|
||
https://bugs.webkit.org/show_bug.cgi?id=217649
|
||
<rdar://problem/70236877>
|
||
|
||
Reviewed by Darin Adler.
|
||
|
||
Bug 42796 introduced MACOSX_DEPLOYMENT_TARGET_<PLATFORM> as "support
|
||
for compiling WebKit against iOS SDKs". Support for the iOS part of
|
||
this feature was later removed in several changes, including Bug
|
||
139212, Bug 139463 and Bug 144762. However, vestiges have remained for
|
||
five or six years in the form of MACOSX_DEPLOYMENT_TARGET_macosx. The
|
||
inclusion of the platform in MACOSX_DEPLOYMENT_TARGET is no longer
|
||
needed and can be removed.
|
||
|
||
This changes brings most projects in conformance with other projects
|
||
that don't support including the platform in MACOSX_DEPLOYMENT_TARGET,
|
||
including WebEditingTester, gtest, WebKitTestRunner, MiniBrowser, and
|
||
TestWebKitAPI.
|
||
|
||
Along the way, remove a couple of left-over references to macOS 10.16,
|
||
and a couple of places where [sdk=macosx*] was still being used.
|
||
|
||
With this change, initialization of MACOSX_DEPLOYMENT_TARGET should be
|
||
consistent across all projects, with two exceptions: WebKitLauncher
|
||
(which hardcodes it to 10.12) and libwebrtc's copy of googletest
|
||
(which hardcodes it to 10.4). The reasons for these hard-coded values
|
||
is not apparent, so leave them be.
|
||
|
||
* Configurations/DebugRelease.xcconfig:
|
||
|
||
2020-10-12 Luming Yin <luming_yin@apple.com>
|
||
|
||
[macOS] Workaround for MAC_OS_X_VERSION_MAJOR incorrectly including minor version when building
|
||
with Xcode 12 on macOS Big Sur SUs
|
||
https://bugs.webkit.org/show_bug.cgi?id=217602
|
||
rdar://70194453
|
||
|
||
Reviewed by Darin Adler.
|
||
|
||
The previous workaround turns out to be ineffective because we can't set the value of
|
||
TARGET_MAC_OS_X_VERSION_MAJOR based on a previous value of itself. Introduce a new
|
||
variable TARGET_MAC_OS_X_VERSION_MAJOR to determine whether we need to explicitly
|
||
adjust MAC_OS_X_VERSION_MAJOR to 110000.
|
||
|
||
* Configurations/DebugRelease.xcconfig:
|
||
|
||
2020-10-12 Luming Yin <luming_yin@apple.com>
|
||
|
||
[macOS] Workaround for MAC_OS_X_VERSION_MAJOR incorrectly including minor version when building
|
||
with Xcode 12 on macOS Big Sur SUs
|
||
https://bugs.webkit.org/show_bug.cgi?id=217602
|
||
rdar://70194453
|
||
|
||
Reviewed by Darin Adler.
|
||
|
||
Due to a bug in Xcode (rdar://70185899), Xcode 12.0 and Xcode 12.1 Beta incorrectly includes the
|
||
minor release number in MAC_OS_X_VERSION_MAJOR, which causes Debug and Release builds of WebKit
|
||
to be misconfigured when building on macOS Big Sur SUs, leading to webpages failing to load.
|
||
|
||
To work around the Xcode bug, when the MAC_OS_X_VERSION_MAJOR includes the minor version number,
|
||
drop the minor version number by explicitly setting TARGET_MAC_OS_X_VERSION_MAJOR to 110000.
|
||
|
||
Note: This change should be reverted after <rdar://70185899> is resolved.
|
||
|
||
* Configurations/DebugRelease.xcconfig:
|
||
|
||
2020-10-11 Luming Yin <luming_yin@apple.com>
|
||
|
||
Strip patch version from TARGET_MAC_OS_X_VERSION_MAJOR when building for macOS Big Sur
|
||
or later
|
||
https://bugs.webkit.org/show_bug.cgi?id=217594
|
||
rdar://70188497
|
||
|
||
Reviewed by Darin Adler.
|
||
|
||
To ensure successful Mac Catalyst WebKit builds, strip the patch version from
|
||
TARGET_MAC_OS_X_VERSION_MAJOR by using two `base:`s on MACOSX_DEPLOYMENT_TARGET.
|
||
|
||
* Configurations/Base.xcconfig:
|
||
|
||
2020-10-11 Luming Yin <luming_yin@apple.com>
|
||
|
||
Ignore deployment suffix and identifier when computing major OS version for macOS
|
||
Big Sur and newer
|
||
https://bugs.webkit.org/show_bug.cgi?id=217584
|
||
rdar://70168426
|
||
|
||
Reviewed by Darin Adler.
|
||
|
||
Stop using MACOSX_DEPLOYMENT_TARGET:suffix:identifier to compute major OS versions.
|
||
Only use the deployment target base for macOS Big Sur and newer. Keep the manual
|
||
definitions for legacy versions of macOS.
|
||
|
||
* Configurations/Base.xcconfig:
|
||
|
||
2020-09-21 Mark Lam <mark.lam@apple.com>
|
||
|
||
Move some LLInt globals into JSC::Config.
|
||
https://bugs.webkit.org/show_bug.cgi?id=216685
|
||
rdar://68964544
|
||
|
||
Reviewed by Keith Miller.
|
||
|
||
Introduce ConfigAlignment to match WTFConfig.h.
|
||
Added BENABLE(UNIFIED_AND_FREEZABLE_CONFIG_RECORD) support to match WTF.
|
||
|
||
* bmalloc/BPlatform.h:
|
||
* bmalloc/Gigacage.cpp:
|
||
(Gigacage::ensureGigacage):
|
||
* bmalloc/GigacageConfig.h:
|
||
* bmalloc/mbmalloc.cpp:
|
||
|
||
2020-08-07 Saam Barati <sbarati@apple.com>
|
||
|
||
Use thread_switch instead of switch_pri to drop priority to zero for 1ms instead of 10
|
||
https://bugs.webkit.org/show_bug.cgi?id=215248
|
||
|
||
Reviewed by Yusuke Suzuki.
|
||
|
||
* bmalloc/Mutex.cpp:
|
||
(bmalloc::yield):
|
||
(bmalloc::Mutex::lockSlowCase):
|
||
|
||
2020-08-05 Tim Horton <timothy_horton@apple.com>
|
||
|
||
Remove all references to non-existent 10.16
|
||
https://bugs.webkit.org/show_bug.cgi?id=215202
|
||
|
||
Reviewed by Wenson Hsieh.
|
||
|
||
* Configurations/Base.xcconfig:
|
||
* Configurations/DebugRelease.xcconfig:
|
||
|
||
2020-07-22 Conrad Shultz <conrad_shultz@apple.com>
|
||
|
||
Update macOS Version macros
|
||
https://bugs.webkit.org/show_bug.cgi?id=214653
|
||
|
||
Reviewed by Tim Horton.
|
||
|
||
* Configurations/Base.xcconfig:
|
||
* Configurations/DebugRelease.xcconfig:
|
||
|
||
2020-07-03 Sam Weinig <weinig@apple.com>
|
||
|
||
Add "-Wliteral-conversion" warning to Xcode based builds and fix the issues it finds
|
||
https://bugs.webkit.org/show_bug.cgi?id=213931
|
||
|
||
Reviewed by Darin Adler.
|
||
|
||
* Configurations/Base.xcconfig:
|
||
Add -Wliteral-conversion.
|
||
|
||
2020-06-30 Andy Estes <aestes@apple.com>
|
||
|
||
[Xcode] Enable the "My Mac (Mac Catalyst)" destination in WebKit Xcode projects
|
||
https://bugs.webkit.org/show_bug.cgi?id=213740
|
||
|
||
Reviewed by Darin Adler.
|
||
|
||
* Configurations/Base.xcconfig: Set SUPPORTS_MACCATALYST to YES to tell Xcode that this
|
||
project supports building for Mac Catalyst.
|
||
|
||
2020-06-22 Saam Barati <sbarati@apple.com>
|
||
|
||
Only use vm_kernel_page_size on arm architectures
|
||
https://bugs.webkit.org/show_bug.cgi?id=213474
|
||
|
||
Reviewed by Sam Weinig.
|
||
|
||
We don't want to use this under catalyst.
|
||
|
||
* bmalloc/VMAllocate.h:
|
||
(bmalloc::vmPageSizePhysical):
|
||
|
||
2020-06-22 Tim Horton <timothy_horton@apple.com>
|
||
|
||
Update macOS version macros
|
||
https://bugs.webkit.org/show_bug.cgi?id=213484
|
||
|
||
Reviewed by Alexey Proskuryakov.
|
||
|
||
* Configurations/Base.xcconfig:
|
||
* Configurations/DebugRelease.xcconfig:
|
||
|
||
2020-06-19 Yusuke Suzuki <ysuzuki@apple.com>
|
||
|
||
[JSC] Check Gigacage usage before launching VM
|
||
https://bugs.webkit.org/show_bug.cgi?id=213410
|
||
|
||
Reviewed by Mark Lam.
|
||
|
||
Add Gigacage::footprint and Gigacage::size functions to expose usage to API users.
|
||
Rename Gigacage::size to Gigacage::maxSize.
|
||
|
||
* bmalloc/Gigacage.cpp:
|
||
(Gigacage::ensureGigacage):
|
||
(Gigacage::size):
|
||
(Gigacage::footprint):
|
||
* bmalloc/Gigacage.h:
|
||
(Gigacage::maxSize):
|
||
(Gigacage::alignment):
|
||
(Gigacage::mask):
|
||
(Gigacage::size):
|
||
(Gigacage::footprint):
|
||
* bmalloc/Heap.cpp:
|
||
(bmalloc::Heap::gigacageSize):
|
||
* bmalloc/Heap.h:
|
||
|
||
2020-06-04 Tim Horton <timothy_horton@apple.com>
|
||
|
||
Work around broken system version macro
|
||
https://bugs.webkit.org/show_bug.cgi?id=212726
|
||
|
||
Reviewed by Dan Bernstein.
|
||
|
||
* Configurations/DebugRelease.xcconfig:
|
||
|
||
2020-06-02 Mark Lam <mark.lam@apple.com>
|
||
|
||
Change Gigacage::Config to use storage in WebConfig::g_config instead of its own.
|
||
https://bugs.webkit.org/show_bug.cgi?id=212585
|
||
<rdar://problem/63812487>
|
||
|
||
Reviewed by Yusuke Suzuki.
|
||
|
||
1. Gigacage::Config now reserves and expect space to be available in an external
|
||
WebConfig::g_config buffer. Gigacage does not allocate that buffer.
|
||
|
||
2. Moved Gigacage::Config to GigacageConfig.h.
|
||
This allows WTFConfig.h to include GigacageConfig.h instead of all of Gigacage.h.
|
||
|
||
3. Moved Gigacage::Kind to GigacageKind.h.
|
||
Otherwise, Gigacage::Kind would need to move to GigacageConfig.h which is a
|
||
weird place to put it.
|
||
|
||
4. Removed freezeGigacageConfig(), unfreezeGigacageConfig(), and
|
||
permanentlyFreezeGigacageConfig().
|
||
|
||
It is no longer possible to temporarily freeze and unfreeze the Gigacage::Config
|
||
because it now share the same memory page with higher level Configs.
|
||
|
||
permanentlyFreezeGigacageConfig() is no longer needed because it is subsumed
|
||
by WTF::Config::permanentlyFreeze(), which will freeze the entire WebConfig::g_config
|
||
buffer.
|
||
|
||
One difference in behavior here is that Gigacage::Config data used to be permanently
|
||
frozen as soon as forbidDisablingPrimitiveGigacage() is called. Now, it isn't
|
||
permanently frozen until the end of the construction of the first JSC::VM instance
|
||
in the process (just like the other Config records). This still guarantees that
|
||
it is frozen before any JS script can run.
|
||
|
||
5. Previously, disablePrimitiveGigacage() works by nullifying the Primitive gigacage
|
||
base pointer. We can no longer do that because the base pointer will be frozen
|
||
on VM instantiation. Instead, if not forbidden, we now disable the Primitive
|
||
gigacage by setting a disablePrimitiveGigacageRequested bool variable that is
|
||
not frozen in the Gigacage::Config.
|
||
|
||
To check if the Primitive gigacage is enabled, the LLInt, AssemblyHelpers::cageConditionally(),
|
||
and runtime functions will check the following conditions:
|
||
|
||
g_gigacageConfig.basePtr(Primitive) &&
|
||
(disablingPrimitiveGigacageIsForbidden() || disableNotRequestedForPrimitiveGigacage())
|
||
|
||
The base pointer being null means the gigacage was never set up.
|
||
|
||
If disablingPrimitiveGigacageIsForbidden() is true, then we don't care
|
||
whether a disable request has been received.
|
||
|
||
Otherwise, the gigacage is only enabled if it has been set up, and a
|
||
disable request has not been received.
|
||
|
||
Note that the first 2 terms are frozen in the Gigacage::Config. Only the
|
||
last term is a runtime variable. If disabling is forbidden, then the
|
||
runtime variable never comes into play.
|
||
|
||
The FTL does not rely on a runtime check for whether the Primitive gigacage is
|
||
enabled or not. Instead, it relies on a watchpoint to handle this. So, it
|
||
just works, and there's no performance penalty with adding the 2 extra terms
|
||
to check.
|
||
|
||
Note also that the jsc shell and the WebProcess will forbid disabling of the
|
||
Primitive gigacage. This means the AssemblyHelpers::cageConditionally() will
|
||
also not generate the runtime checks for the 2 extra terms.
|
||
|
||
Only the LLInt and runtime functions will have to do work to check the 2 extra
|
||
terms. But because these are not in perf critical paths, this is ok.
|
||
|
||
Note that we're deliberately gating the disablePrimitiveGigacageRequested
|
||
variable check on disablingPrimitiveGigacageIsForbidden though, logically, the
|
||
isEnable check does not really depend on whether disabling is forbidden or not.
|
||
We do this because disablingPrimitiveGigacageIsForbidden is frozen in the Config,
|
||
and it is, therefore, a stronger guarantee of correctness whereas the variable
|
||
can be corrupted.
|
||
|
||
6. Replaced isDisablingPrimitiveGigacageForbidden(), canPrimitiveGigacageBeDisabled(),
|
||
and isPrimitiveGigacagePermanentlyEnabled() with disablingPrimitiveGigacageIsForbidden().
|
||
|
||
* CMakeLists.txt:
|
||
* bmalloc.xcodeproj/project.pbxproj:
|
||
* bmalloc/Gigacage.cpp:
|
||
(Gigacage::ensureGigacage):
|
||
(Gigacage::disablePrimitiveGigacage):
|
||
(Gigacage::forbidDisablingPrimitiveGigacage):
|
||
(Gigacage::bmalloc::freezeGigacageConfig): Deleted.
|
||
(Gigacage::bmalloc::unfreezeGigacageConfig): Deleted.
|
||
(Gigacage::bmalloc::permanentlyFreezeGigacageConfig): Deleted.
|
||
(Gigacage::bmalloc::UnfreezeGigacageConfigScope::UnfreezeGigacageConfigScope): Deleted.
|
||
(Gigacage::bmalloc::UnfreezeGigacageConfigScope::~UnfreezeGigacageConfigScope): Deleted.
|
||
(Gigacage::isDisablingPrimitiveGigacageForbidden): Deleted.
|
||
* bmalloc/Gigacage.h:
|
||
(Gigacage::disablingPrimitiveGigacageIsForbidden):
|
||
(Gigacage::disableNotRequestedForPrimitiveGigacage):
|
||
(Gigacage::isEnabled):
|
||
(Gigacage::basePtr):
|
||
(Gigacage::caged):
|
||
(Gigacage::forbidDisablingPrimitiveGigacage):
|
||
(): Deleted.
|
||
(Gigacage::Config::basePtr const): Deleted.
|
||
(Gigacage::Config::setBasePtr): Deleted.
|
||
(Gigacage::isPrimitiveGigacagePermanentlyEnabled): Deleted.
|
||
(Gigacage::canPrimitiveGigacageBeDisabled): Deleted.
|
||
* bmalloc/GigacageConfig.h: Added.
|
||
(Gigacage::Config::basePtr const):
|
||
(Gigacage::Config::setBasePtr):
|
||
* bmalloc/GigacageKind.h: Added.
|
||
* bmalloc/Heap.cpp:
|
||
(bmalloc::Heap::usingGigacage):
|
||
* bmalloc/mbmalloc.cpp:
|
||
|
||
2020-05-23 Caio Lima <ticaiolima@gmail.com>
|
||
|
||
[bmalloc] Fix OOM errors on MIPS after r261667
|
||
https://bugs.webkit.org/show_bug.cgi?id=212016
|
||
|
||
Reviewed by Yusuke Suzuki.
|
||
|
||
The way we were calculating `newBegin` and `newEnd` on
|
||
`ObjectTypeTable::set` when index is out of bounds didn't consider
|
||
cases where `bits->begin() - bits->count()` or `index - ObjectTypeTable::Bits::bitCountPerWord * 4`
|
||
can underflow and `bits->end() + bits->count()` can overflow.
|
||
Given that, the value used is going to be `index` or `index + 1`.
|
||
Since we extend the size of bitvector everytime we have an OOB, this can cause a pathological case
|
||
that memory will keep extending quite often until systems reachs OOM.
|
||
It is reproducible on ARMv7 and MIPS architectures on
|
||
`stress/array-buffer-view-watchpoint-can-be-fired-in-really-add-in-dfg.js`,
|
||
`stress/big-int-mod-memory-stress.js` and some other tests.
|
||
This patch is including a verification if those operations are going
|
||
to overflow/underflow, and properly set `newBegin` to 0 and `newEnd`
|
||
to UINT_MAX when we observe such behavior.
|
||
|
||
* bmalloc/ObjectTypeTable.cpp:
|
||
(bmalloc::ObjectTypeTable::set):
|
||
|
||
2020-05-18 Mark Lam <mark.lam@apple.com>
|
||
|
||
Implement a faster findBitInWord() using the hardware ctz instruction.
|
||
https://bugs.webkit.org/show_bug.cgi?id=212032
|
||
<rdar://problem/63348086>
|
||
|
||
Reviewed by Saam Barati.
|
||
|
||
Apply same changes to bmalloc's copy of findBitInWord().
|
||
|
||
* bmalloc/Algorithm.h:
|
||
(bmalloc::ctz):
|
||
(bmalloc::findBitInWord):
|
||
|
||
2020-05-13 Yusuke Suzuki <ysuzuki@apple.com>
|
||
|
||
[bmalloc] Introduce lock-less ObjectType query
|
||
https://bugs.webkit.org/show_bug.cgi?id=211809
|
||
|
||
Reviewed by Mark Lam.
|
||
|
||
This patch introduces ObjectTypeTable, which allows lock-less ObjectType query for Chunk*.
|
||
It has bit-vector to store ObjectType per address. And each bit represents 1MB of VA region since
|
||
Chunk*'s size is at least 1MB and ObjectType is 1bit data. Every time we extend this bit-vector
|
||
to support larger VA region, we do not free the old bit-vector. Since we always allocate power-of-2
|
||
sized bit-vector, # of extension is limited and it does not waste much memory because Chunk's size
|
||
is enough large (1MB). Since each 4KB page on macOS can represent a bit-vector for 32GB VA region,
|
||
in practice, this extension almost never happens. I verified that 4KB page can handle memory
|
||
allocations in JetStream2 and Gmail.
|
||
|
||
* CMakeLists.txt:
|
||
* bmalloc.xcodeproj/project.pbxproj:
|
||
* bmalloc/Algorithm.h:
|
||
(bmalloc::roundUpToPowerOfTwo):
|
||
* bmalloc/Deallocator.cpp:
|
||
(bmalloc::Deallocator::deallocateSlowCase):
|
||
* bmalloc/Heap.cpp:
|
||
(bmalloc::Heap::freeableMemory):
|
||
(bmalloc::Heap::decommitLargeRange):
|
||
(bmalloc::Heap::scavenge):
|
||
(bmalloc::Heap::scavengeToHighWatermark):
|
||
(bmalloc::Heap::allocateSmallChunk):
|
||
(bmalloc::Heap::deallocateSmallChunk):
|
||
(bmalloc::Heap::deallocateSmallLine):
|
||
(bmalloc::Heap::splitAndAllocate):
|
||
(bmalloc::Heap::isLarge): Deleted.
|
||
* bmalloc/Heap.h:
|
||
(bmalloc::Heap::isLarge):
|
||
* bmalloc/ObjectType.cpp:
|
||
(bmalloc::objectType):
|
||
* bmalloc/ObjectTypeTable.cpp: Added.
|
||
(bmalloc::ObjectTypeTable::set):
|
||
* bmalloc/ObjectTypeTable.h: Added.
|
||
(bmalloc::ObjectTypeTable::convertToIndex):
|
||
(bmalloc::ObjectTypeTable::Bits::Bits):
|
||
(bmalloc::ObjectTypeTable::Bits::previous const):
|
||
(bmalloc::ObjectTypeTable::Bits::begin const):
|
||
(bmalloc::ObjectTypeTable::Bits::end const):
|
||
(bmalloc::ObjectTypeTable::Bits::count const):
|
||
(bmalloc::ObjectTypeTable::Bits::sizeInBytes const):
|
||
(bmalloc::ObjectTypeTable::Bits::words const):
|
||
(bmalloc::ObjectTypeTable::Bits::words):
|
||
(bmalloc::ObjectTypeTable::ObjectTypeTable):
|
||
(bmalloc::ObjectTypeTable::get):
|
||
(bmalloc::ObjectTypeTable::Bits::get):
|
||
(bmalloc::ObjectTypeTable::Bits::set):
|
||
(bmalloc::ObjectTypeTable::Bits::wordForIndex):
|
||
* bmalloc/Scavenger.cpp:
|
||
(bmalloc::Scavenger::scavenge):
|
||
(bmalloc::Scavenger::partialScavenge):
|
||
(bmalloc::Scavenger::freeableMemory):
|
||
|
||
2020-05-11 Basuke Suzuki <basuke.suzuki@sony.com>
|
||
|
||
[bmalloc][WTF] Add computing memory size implementation for FreeBSD
|
||
https://bugs.webkit.org/show_bug.cgi?id=211749
|
||
|
||
Reviewed by David Kilzer.
|
||
|
||
* bmalloc/AvailableMemory.cpp:
|
||
(bmalloc::computeAvailableMemory):
|
||
|
||
2020-05-09 Don Olmstead <don.olmstead@sony.com>
|
||
|
||
[CMake] Use WEBKIT_EXECUTABLE in MallocBench
|
||
https://bugs.webkit.org/show_bug.cgi?id=211665
|
||
|
||
Reviewed by Yusuke Suzuki.
|
||
|
||
Only build mbmalloc if its possible to run MallocBench.
|
||
|
||
* CMakeLists.txt:
|
||
|
||
2020-05-08 Basuke Suzuki <basuke.suzuki@sony.com>
|
||
|
||
[WTF] Share Linux's MemoryPressureHandler among other Unix ports
|
||
https://bugs.webkit.org/show_bug.cgi?id=208955
|
||
|
||
Reviewed by Yusuke Suzuki.
|
||
|
||
Added FreeBSD implementation of memoryFootprint().
|
||
|
||
* bmalloc/AvailableMemory.cpp:
|
||
(bmalloc::memoryStatus):
|
||
* bmalloc/AvailableMemory.h:
|
||
(bmalloc::isUnderMemoryPressure):
|
||
* bmalloc/bmalloc.h:
|
||
|
||
2020-05-05 Saam Barati <sbarati@apple.com>
|
||
|
||
Don't use the DebugHeap for catalyst
|
||
https://bugs.webkit.org/show_bug.cgi?id=211471
|
||
|
||
Reviewed by Tim Horton.
|
||
|
||
* bmalloc/BPlatform.h:
|
||
* bmalloc/Environment.cpp:
|
||
(bmalloc::Environment::computeIsDebugHeapEnabled):
|
||
* bmalloc/ProcessCheck.h:
|
||
(bmalloc::shouldProcessUnconditionallyUseBmalloc): Deleted.
|
||
* bmalloc/ProcessCheck.mm:
|
||
|
||
2020-05-01 Don Olmstead <don.olmstead@sony.com>
|
||
|
||
Use export macros on all platforms
|
||
https://bugs.webkit.org/show_bug.cgi?id=211293
|
||
|
||
Reviewed by Michael Catanzaro.
|
||
|
||
Always use export macros on all platforms. Remove BUSE_EXPORT_MACROS and always
|
||
define macros based on the platform. Allow overriding of BEXPORT if desired
|
||
otherwise use the defaults.
|
||
|
||
* bmalloc/BExport.h:
|
||
* bmalloc/BPlatform.h:
|
||
|
||
2020-04-29 Mark Lam <mark.lam@apple.com>
|
||
|
||
Freezing of Gigacage and JSC Configs should be thread safe.
|
||
https://bugs.webkit.org/show_bug.cgi?id=211201
|
||
<rdar://problem/62597619>
|
||
|
||
Reviewed by Yusuke Suzuki.
|
||
|
||
* bmalloc/Gigacage.cpp:
|
||
(Gigacage::bmalloc::permanentlyFreezeGigacageConfig):
|
||
|
||
2020-04-25 Darin Adler <darin@apple.com>
|
||
|
||
[Cocoa] Deal with another round of Xcode upgrade checks
|
||
https://bugs.webkit.org/show_bug.cgi?id=211027
|
||
|
||
Reviewed by Alexey Proskuryakov.
|
||
|
||
* bmalloc.xcodeproj/project.pbxproj: Bump the upgrade check version.
|
||
Add a harmless base localization; this project contains nothing localized.
|
||
|
||
2020-04-07 Saam Barati <sbarati@apple.com>
|
||
|
||
RAMification should have a way of gathering vmmaps for each test at the end of each run
|
||
https://bugs.webkit.org/show_bug.cgi?id=210060
|
||
|
||
Reviewed by Yusuke Suzuki.
|
||
|
||
* bmalloc/Scavenger.cpp:
|
||
(bmalloc::Scavenger::scavenge):
|
||
(bmalloc::Scavenger::partialScavenge):
|
||
* bmalloc/Scavenger.h:
|
||
(bmalloc::Scavenger::disable):
|
||
* bmalloc/bmalloc.cpp:
|
||
(bmalloc::api::disableScavenger):
|
||
* bmalloc/bmalloc.h:
|
||
|
||
2020-04-03 David Kilzer <ddkilzer@apple.com>
|
||
|
||
[Xcode] Replace ASAN_OTHER_CFLAGS and ASAN_OTHER_CPLUSPLUSFLAGS with $(inherited)
|
||
<https://webkit.org/b/209963>
|
||
<rdar://problem/61257504>
|
||
|
||
Reviewed by Alexey Proskuryakov.
|
||
|
||
* Configurations/Base.xcconfig:
|
||
- Remove ASAN_OTHER_CFLAGS, ASAN_OTHER_CPLUSPLUSFLAGS and
|
||
ASAN_OTHER_LDFLAGS.
|
||
|
||
2020-04-01 Michael Catanzaro <mcatanzaro@gnome.org>
|
||
|
||
Update check for aarch64
|
||
https://bugs.webkit.org/show_bug.cgi?id=209322
|
||
<rdar://problem/61135818>
|
||
|
||
Reviewed by Mark Lam.
|
||
|
||
Update BPlatform.h to follow the corresponding change in WTF's PlatformCPU.h.
|
||
|
||
* bmalloc/BPlatform.h:
|
||
|
||
2020-03-15 Yusuke Suzuki <ysuzuki@apple.com>
|
||
|
||
Should not use variable-length-array (VLA)
|
||
https://bugs.webkit.org/show_bug.cgi?id=209043
|
||
|
||
Reviewed by Mark Lam.
|
||
|
||
* Configurations/Base.xcconfig:
|
||
|
||
2020-03-13 Saam Barati <sbarati@apple.com>
|
||
|
||
configSizeToProtect should be 16KB
|
||
https://bugs.webkit.org/show_bug.cgi?id=209068
|
||
|
||
Reviewed by Keith Miller.
|
||
|
||
* bmalloc/Gigacage.h:
|
||
|
||
2020-03-09 Mike Gorse <mgorse@suse.com>
|
||
|
||
Build failure on ppc64le if __unix is undefined
|
||
https://bugs.webkit.org/show_bug.cgi?id=207906
|
||
|
||
Reviewed by Yusuke Suzuki.
|
||
|
||
* bmalloc/BPlatform.h: Check for __unix__ along with __unix.
|
||
|
||
2020-03-02 Alan Coon <alancoon@apple.com>
|
||
|
||
Add new Mac target numbers
|
||
https://bugs.webkit.org/show_bug.cgi?id=208398
|
||
|
||
Reviewed by Alexey Proskuryakov.
|
||
|
||
* Configurations/Base.xcconfig:
|
||
* Configurations/DebugRelease.xcconfig:
|
||
|
||
2020-02-29 Yusuke Suzuki <ysuzuki@apple.com>
|
||
|
||
Remove std::lock_guard
|
||
https://bugs.webkit.org/show_bug.cgi?id=206451
|
||
|
||
Reviewed by Anders Carlsson.
|
||
|
||
* bmalloc/Mutex.h:
|
||
|
||
2020-02-28 Saam Barati <sbarati@apple.com>
|
||
|
||
Clean up code with how we choose Gigacage sizes and whether or not to use Wasm fast memory
|
||
https://bugs.webkit.org/show_bug.cgi?id=208392
|
||
|
||
Reviewed by Yusuke Suzuki.
|
||
|
||
* bmalloc/Gigacage.h:
|
||
|
||
2020-02-26 Basuke Suzuki <basuke.suzuki@sony.com>
|
||
|
||
[bmalloc][PlayStation] Set Scavenger's thread name.
|
||
https://bugs.webkit.org/show_bug.cgi?id=208268
|
||
|
||
Reviewed by Alex Christensen.
|
||
|
||
We also need to have our thread with proper name.
|
||
|
||
* bmalloc/Scavenger.cpp:
|
||
(bmalloc::Scavenger::setThreadName):
|
||
|
||
2020-02-25 Saam Barati <sbarati@apple.com>
|
||
|
||
Update stale comment about PackedAlignedPtr
|
||
https://bugs.webkit.org/show_bug.cgi?id=208176
|
||
|
||
Reviewed by Yusuke Suzuki.
|
||
|
||
The comment was stale since cells aren't guaranteed anymore to be
|
||
aligned on 16 byte boundaries because of the GCs precise allocations.
|
||
|
||
* bmalloc/Packed.h:
|
||
|
||
2020-02-09 Keith Rollin <krollin@apple.com>
|
||
|
||
Re-enable LTO for ARM builds
|
||
https://bugs.webkit.org/show_bug.cgi?id=207402
|
||
<rdar://problem/49190767>
|
||
|
||
Reviewed by Sam Weinig.
|
||
|
||
Bug 190758 re-enabled LTO for Production builds for x86-family CPUs.
|
||
Enabling it for ARM was left out due to a compiler issue. That issue
|
||
has been fixed, and so now we can re-enable LTO for ARM.
|
||
|
||
* Configurations/Base.xcconfig:
|
||
|
||
2020-02-08 Basuke Suzuki <basuke.suzuki@sony.com>
|
||
|
||
[bmalloc] VMHeap can be merge into Heap
|
||
https://bugs.webkit.org/show_bug.cgi?id=207410
|
||
|
||
Reviewed by Yusuke Suzuki.
|
||
|
||
VMHeap has only one member function in it and Heap is the only client of that.
|
||
No member variable is defined. It does nothing special with its context as a class.
|
||
It is safe to merge the function into Heap.
|
||
|
||
* CMakeLists.txt:
|
||
* bmalloc.xcodeproj/project.pbxproj:
|
||
* bmalloc/Heap.cpp:
|
||
(bmalloc::Heap::allocateLarge):
|
||
(bmalloc::Heap::tryAllocateLargeChunk): Moved from VMHeap.
|
||
* bmalloc/Heap.h:
|
||
* bmalloc/VMHeap.cpp: Removed.
|
||
* bmalloc/VMHeap.h: Removed.
|
||
|
||
2020-02-05 Don Olmstead <don.olmstead@sony.com>
|
||
|
||
[bmalloc] Add declspec support for export macros
|
||
https://bugs.webkit.org/show_bug.cgi?id=207158
|
||
|
||
Reviewed by Yusuke Suzuki.
|
||
|
||
Add a check for __has_declspec_attribute which is a Clang extension for Microsoft
|
||
style __declspec attributes in BCompiler.h. Then use that check within BPlatform.h
|
||
to determine if __declspec or visibility("default") should be used for the export
|
||
macros.
|
||
|
||
The BExport.h file was then expanded to properly implement __declspec definitions
|
||
for export and import. Libraries that could be compiled statically in WebKit have
|
||
a STATICALLY_LINKED_WITH_${library} definition for when they're compiled statically
|
||
and later exposed through a different shared library. In practice though this is
|
||
really only needed when __declspec is used since this is the only case where the
|
||
import and export declarations differ.
|
||
|
||
The BEXPORT on StaticPerProcess resulted in undefined symbol errors when WebCore
|
||
was linked. Specifically IsoSharedHeap::getFastCase and IsoSharedHeap::getSlowCase
|
||
which is a subclass. It appears that Storage was the only one required to be
|
||
exported. This only appeared due to how __declspec requires exports and imports
|
||
to be defined differently.
|
||
|
||
Also added support for export macros on PlayStation in the BPlatform.h file and
|
||
fixed any exports for that platform when building.
|
||
|
||
* CMakeLists.txt:
|
||
* bmalloc/Allocator.h:
|
||
* bmalloc/AvailableMemory.h:
|
||
* bmalloc/BCompiler.h:
|
||
* bmalloc/BExport.h:
|
||
* bmalloc/BPlatform.h:
|
||
* bmalloc/Cache.h:
|
||
* bmalloc/IsoTLS.h:
|
||
* bmalloc/StaticPerProcess.h:
|
||
|
||
2020-01-21 Basuke Suzuki <basuke.suzuki@sony.com>
|
||
|
||
[bmalloc] Make use of LockHolder strict in some methods of Scavenger
|
||
https://bugs.webkit.org/show_bug.cgi?id=206460
|
||
|
||
Reviewed by Darin Adler.
|
||
|
||
For instance, Scavenger::runHoldingLock() assume the caller has lock and express that by its function name. This rule can be
|
||
strict by passing LockHolder and that's the way as other code do.
|
||
|
||
Same change to runSoonHoldingLock and scheduleIfUnderMemoryPressureHoldingLock.
|
||
|
||
* bmalloc/Scavenger.cpp:
|
||
(bmalloc::Scavenger::run):
|
||
(bmalloc::Scavenger::runSoon):
|
||
(bmalloc::Scavenger::scheduleIfUnderMemoryPressure):
|
||
(bmalloc::Scavenger::schedule):
|
||
(bmalloc::Scavenger::runHoldingLock): Renamed.
|
||
(bmalloc::Scavenger::runSoonHoldingLock): Renamed.
|
||
(bmalloc::Scavenger::scheduleIfUnderMemoryPressureHoldingLock): Renamed.
|
||
* bmalloc/Scavenger.h:
|
||
|
||
2020-01-17 Sam Weinig <weinig@apple.com>
|
||
|
||
Platform.h is out of control Part 8: Macros are used inconsistently
|
||
https://bugs.webkit.org/show_bug.cgi?id=206425
|
||
|
||
Reviewed by Darin Adler.
|
||
|
||
* bmalloc/BPlatform.h:
|
||
Update OS_EFFECTIVE_ADDRESS_WIDTH to match WTF definition, add needed OS macros.
|
||
|
||
2020-01-20 David Kilzer <ddkilzer@apple.com>
|
||
|
||
Fix missing header guards and clean up empty files in bmalloc, WTF, JavaScriptCore
|
||
<https://webkit.org/b/206481>
|
||
|
||
Reviewed by Darin Adler.
|
||
|
||
* bmalloc/PerThread.cpp: Remove empty file.
|
||
* bmalloc/ScopeExit.h: Add #pragma once.
|
||
|
||
2020-01-17 Basuke Suzuki <basuke.suzuki@sony.com>
|
||
|
||
[bmalloc] Define alias for std::lock_guard and std::unique_lock for better readability
|
||
https://bugs.webkit.org/show_bug.cgi?id=206443
|
||
|
||
Reviewed by Yusuke Suzuki.
|
||
|
||
There are two types of lock holder in bmalloc: std::lock_guard and std::unique_lock. Their names are relatively long
|
||
and a bit harder to distinguish them each other. Define simple type name for them, LockHolder and UniqueLockHolder.
|
||
|
||
* bmalloc/AllIsoHeaps.cpp:
|
||
(bmalloc::AllIsoHeaps::AllIsoHeaps):
|
||
(bmalloc::AllIsoHeaps::add):
|
||
(bmalloc::AllIsoHeaps::head):
|
||
* bmalloc/AllIsoHeaps.h:
|
||
* bmalloc/Allocator.cpp:
|
||
(bmalloc::Allocator::reallocateImpl):
|
||
(bmalloc::Allocator::refillAllocatorSlowCase):
|
||
(bmalloc::Allocator::allocateLarge):
|
||
* bmalloc/CryptoRandom.cpp:
|
||
(bmalloc::ARC4RandomNumberGenerator::ARC4RandomNumberGenerator):
|
||
(bmalloc::ARC4RandomNumberGenerator::randomValues):
|
||
* bmalloc/Deallocator.cpp:
|
||
(bmalloc::Deallocator::scavenge):
|
||
(bmalloc::Deallocator::processObjectLog):
|
||
(bmalloc::Deallocator::deallocateSlowCase):
|
||
* bmalloc/Deallocator.h:
|
||
(bmalloc::Deallocator::lineCache):
|
||
* bmalloc/DebugHeap.cpp:
|
||
(bmalloc::DebugHeap::DebugHeap):
|
||
(bmalloc::DebugHeap::memalignLarge):
|
||
(bmalloc::DebugHeap::freeLarge):
|
||
* bmalloc/DebugHeap.h:
|
||
* bmalloc/DeferredTrigger.h:
|
||
* bmalloc/DeferredTriggerInlines.h:
|
||
(bmalloc::DeferredTrigger<trigger>::didBecome):
|
||
(bmalloc::DeferredTrigger<trigger>::handleDeferral):
|
||
* bmalloc/Environment.cpp:
|
||
(bmalloc::Environment::Environment):
|
||
* bmalloc/Environment.h:
|
||
* bmalloc/Gigacage.cpp:
|
||
(bmalloc::PrimitiveDisableCallbacks::PrimitiveDisableCallbacks):
|
||
(Gigacage::disablePrimitiveGigacage):
|
||
(Gigacage::addPrimitiveDisableCallback):
|
||
(Gigacage::removePrimitiveDisableCallback):
|
||
* bmalloc/Heap.cpp:
|
||
(bmalloc::Heap::Heap):
|
||
(bmalloc::Heap::freeableMemory):
|
||
(bmalloc::Heap::markAllLargeAsEligibile):
|
||
(bmalloc::Heap::decommitLargeRange):
|
||
(bmalloc::Heap::scavenge):
|
||
(bmalloc::Heap::scavengeToHighWatermark):
|
||
(bmalloc::Heap::deallocateLineCache):
|
||
(bmalloc::Heap::allocateSmallChunk):
|
||
(bmalloc::Heap::allocateSmallPage):
|
||
(bmalloc::Heap::deallocateSmallLine):
|
||
(bmalloc::Heap::allocateSmallBumpRangesByMetadata):
|
||
(bmalloc::Heap::allocateSmallBumpRangesByObject):
|
||
(bmalloc::Heap::splitAndAllocate):
|
||
(bmalloc::Heap::allocateLarge):
|
||
(bmalloc::Heap::isLarge):
|
||
(bmalloc::Heap::largeSize):
|
||
(bmalloc::Heap::shrinkLarge):
|
||
(bmalloc::Heap::deallocateLarge):
|
||
(bmalloc::Heap::externalCommit):
|
||
(bmalloc::Heap::externalDecommit):
|
||
* bmalloc/Heap.h:
|
||
(bmalloc::Heap::allocateSmallBumpRanges):
|
||
(bmalloc::Heap::derefSmallLine):
|
||
* bmalloc/HeapConstants.cpp:
|
||
(bmalloc::HeapConstants::HeapConstants):
|
||
* bmalloc/HeapConstants.h:
|
||
* bmalloc/IsoAllocatorInlines.h:
|
||
(bmalloc::IsoAllocator<Config>::allocateSlow):
|
||
(bmalloc::IsoAllocator<Config>::scavenge):
|
||
* bmalloc/IsoDeallocatorInlines.h:
|
||
(bmalloc::IsoDeallocator<Config>::deallocate):
|
||
(bmalloc::IsoDeallocator<Config>::scavenge):
|
||
* bmalloc/IsoDirectory.h:
|
||
* bmalloc/IsoDirectoryInlines.h:
|
||
(bmalloc::passedNumPages>::takeFirstEligible):
|
||
(bmalloc::passedNumPages>::didBecome):
|
||
(bmalloc::passedNumPages>::didDecommit):
|
||
(bmalloc::passedNumPages>::scavengePage):
|
||
(bmalloc::passedNumPages>::scavenge):
|
||
(bmalloc::passedNumPages>::scavengeToHighWatermark):
|
||
(bmalloc::passedNumPages>::forEachCommittedPage):
|
||
* bmalloc/IsoHeapImpl.h:
|
||
* bmalloc/IsoHeapImplInlines.h:
|
||
(bmalloc::IsoHeapImpl<Config>::takeFirstEligible):
|
||
(bmalloc::IsoHeapImpl<Config>::didBecomeEligibleOrDecommited):
|
||
(bmalloc::IsoHeapImpl<Config>::scavenge):
|
||
(bmalloc::IsoHeapImpl<Config>::scavengeToHighWatermark):
|
||
(bmalloc::IsoHeapImpl<Config>::numLiveObjects):
|
||
(bmalloc::IsoHeapImpl<Config>::numCommittedPages):
|
||
(bmalloc::IsoHeapImpl<Config>::forEachDirectory):
|
||
(bmalloc::IsoHeapImpl<Config>::forEachCommittedPage):
|
||
(bmalloc::IsoHeapImpl<Config>::forEachLiveObject):
|
||
(bmalloc::IsoHeapImpl<Config>::allocateFromShared):
|
||
* bmalloc/IsoPage.h:
|
||
* bmalloc/IsoPageInlines.h:
|
||
(bmalloc::IsoPage<Config>::free):
|
||
(bmalloc::IsoPage<Config>::startAllocating):
|
||
(bmalloc::IsoPage<Config>::stopAllocating):
|
||
(bmalloc::IsoPage<Config>::forEachLiveObject):
|
||
* bmalloc/IsoSharedHeap.h:
|
||
(bmalloc::IsoSharedHeap::IsoSharedHeap):
|
||
* bmalloc/IsoSharedHeapInlines.h:
|
||
(bmalloc::IsoSharedHeap::allocateNew):
|
||
(bmalloc::IsoSharedHeap::allocateSlow):
|
||
* bmalloc/IsoSharedPage.h:
|
||
* bmalloc/IsoSharedPageInlines.h:
|
||
(bmalloc::IsoSharedPage::free):
|
||
(bmalloc::IsoSharedPage::startAllocating):
|
||
(bmalloc::IsoSharedPage::stopAllocating):
|
||
* bmalloc/IsoTLSDeallocatorEntry.h:
|
||
* bmalloc/IsoTLSDeallocatorEntryInlines.h:
|
||
(bmalloc::IsoTLSDeallocatorEntry<Config>::IsoTLSDeallocatorEntry):
|
||
* bmalloc/IsoTLSInlines.h:
|
||
(bmalloc::IsoTLS::ensureHeap):
|
||
* bmalloc/IsoTLSLayout.cpp:
|
||
(bmalloc::IsoTLSLayout::IsoTLSLayout):
|
||
(bmalloc::IsoTLSLayout::add):
|
||
* bmalloc/IsoTLSLayout.h:
|
||
* bmalloc/Mutex.h:
|
||
(bmalloc::sleep):
|
||
(bmalloc::waitUntilFalse):
|
||
* bmalloc/ObjectType.cpp:
|
||
(bmalloc::objectType):
|
||
* bmalloc/PerProcess.cpp:
|
||
(bmalloc::getPerProcessData):
|
||
* bmalloc/PerProcess.h:
|
||
(bmalloc::PerProcess::getSlowCase):
|
||
* bmalloc/Scavenger.cpp:
|
||
(bmalloc::Scavenger::Scavenger):
|
||
(bmalloc::Scavenger::run):
|
||
(bmalloc::Scavenger::runSoon):
|
||
(bmalloc::Scavenger::scheduleIfUnderMemoryPressure):
|
||
(bmalloc::Scavenger::schedule):
|
||
(bmalloc::Scavenger::timeSinceLastFullScavenge):
|
||
(bmalloc::Scavenger::timeSinceLastPartialScavenge):
|
||
(bmalloc::Scavenger::scavenge):
|
||
(bmalloc::Scavenger::partialScavenge):
|
||
(bmalloc::Scavenger::freeableMemory):
|
||
(bmalloc::Scavenger::threadRunLoop):
|
||
* bmalloc/Scavenger.h:
|
||
* bmalloc/SmallLine.h:
|
||
(bmalloc::SmallLine::refCount):
|
||
(bmalloc::SmallLine::ref):
|
||
(bmalloc::SmallLine::deref):
|
||
* bmalloc/SmallPage.h:
|
||
(bmalloc::SmallPage::refCount):
|
||
(bmalloc::SmallPage::hasFreeLines const):
|
||
(bmalloc::SmallPage::setHasFreeLines):
|
||
(bmalloc::SmallPage::ref):
|
||
(bmalloc::SmallPage::deref):
|
||
* bmalloc/StaticPerProcess.h:
|
||
* bmalloc/VMHeap.cpp:
|
||
(bmalloc::VMHeap::VMHeap):
|
||
* bmalloc/VMHeap.h:
|
||
* bmalloc/Zone.cpp:
|
||
(bmalloc::Zone::Zone):
|
||
* bmalloc/Zone.h:
|
||
* bmalloc/bmalloc.cpp:
|
||
(bmalloc::api::tryLargeZeroedMemalignVirtual):
|
||
(bmalloc::api::freeLargeVirtual):
|
||
(bmalloc::api::setScavengerThreadQOSClass):
|
||
|
||
2020-01-17 Yusuke Suzuki <ysuzuki@apple.com>
|
||
|
||
[bmalloc] Fix IsoHeapImpl's assertion introduced in r254708
|
||
https://bugs.webkit.org/show_bug.cgi?id=206440
|
||
<rdar://problem/58688794>
|
||
|
||
Reviewed by Mark Lam.
|
||
|
||
The assertion introduced in r254708 was wrong.
|
||
Fixing it to check the invariant we actually want to ensure.
|
||
This fixes TestWTF crash in Debug build.
|
||
|
||
* bmalloc/IsoHeapImplInlines.h:
|
||
(bmalloc::IsoHeapImpl<Config>::takeFirstEligible):
|
||
|
||
2020-01-16 Yusuke Suzuki <ysuzuki@apple.com>
|
||
|
||
[bmalloc] IsoHeap's initial setup should be small
|
||
https://bugs.webkit.org/show_bug.cgi?id=206214
|
||
|
||
Reviewed by Michael Saboff.
|
||
|
||
Keep IsoHeap related data structures small by using Packed technique. We start using IsoHeap for many classes,
|
||
then it is important that we keep metadata for IsoHeap small because these data persistently exists.
|
||
|
||
1. We pass IsoHeapImpl<> as a parameter instead of holding it unnecessarily.
|
||
2. We make some of pointers Packed so that we can keep sizeof(IsoHeapImpl<Config>) small.
|
||
3. One of the drawback of PackedPtr is that loading and storing are not atomic. And we pass `const std::lock_guard<Mutex>&`
|
||
to functions if functions need to be called with lock so that we ensure that PackedPtr are accessed only when lock is
|
||
held correctly.
|
||
|
||
* CMakeLists.txt:
|
||
* bmalloc.xcodeproj/project.pbxproj:
|
||
* bmalloc/Algorithm.h:
|
||
(bmalloc::ctzConstexpr):
|
||
(bmalloc::getLSBSetNonZeroConstexpr):
|
||
* bmalloc/BPlatform.h:
|
||
* bmalloc/DebugHeap.cpp:
|
||
(bmalloc::DebugHeap::DebugHeap):
|
||
* bmalloc/DebugHeap.h:
|
||
* bmalloc/DeferredTrigger.h:
|
||
* bmalloc/DeferredTriggerInlines.h:
|
||
(bmalloc::DeferredTrigger<trigger>::didBecome):
|
||
(bmalloc::DeferredTrigger<trigger>::handleDeferral):
|
||
* bmalloc/Environment.cpp:
|
||
(bmalloc::Environment::Environment):
|
||
* bmalloc/Environment.h:
|
||
* bmalloc/Gigacage.cpp:
|
||
(bmalloc::PrimitiveDisableCallbacks::PrimitiveDisableCallbacks):
|
||
* bmalloc/Heap.cpp:
|
||
(bmalloc::Heap::freeableMemory):
|
||
(bmalloc::Heap::markAllLargeAsEligibile):
|
||
(bmalloc::Heap::decommitLargeRange):
|
||
(bmalloc::Heap::scavenge):
|
||
(bmalloc::Heap::scavengeToHighWatermark):
|
||
* bmalloc/Heap.h:
|
||
* bmalloc/HeapConstants.cpp:
|
||
(bmalloc::HeapConstants::HeapConstants):
|
||
* bmalloc/HeapConstants.h:
|
||
* bmalloc/IsoAllocator.h:
|
||
* bmalloc/IsoAllocatorInlines.h:
|
||
(bmalloc::IsoAllocator<Config>::IsoAllocator):
|
||
(bmalloc::IsoAllocator<Config>::allocate):
|
||
(bmalloc::IsoAllocator<Config>::allocateSlow):
|
||
(bmalloc::IsoAllocator<Config>::scavenge):
|
||
* bmalloc/IsoDeallocatorInlines.h:
|
||
(bmalloc::IsoDeallocator<Config>::scavenge):
|
||
* bmalloc/IsoDirectory.h:
|
||
* bmalloc/IsoDirectoryInlines.h:
|
||
(bmalloc::passedNumPages>::IsoDirectory):
|
||
(bmalloc::passedNumPages>::takeFirstEligible):
|
||
(bmalloc::passedNumPages>::didBecome):
|
||
(bmalloc::passedNumPages>::didDecommit):
|
||
(bmalloc::passedNumPages>::scavengePage):
|
||
(bmalloc::passedNumPages>::scavenge):
|
||
(bmalloc::passedNumPages>::scavengeToHighWatermark):
|
||
(bmalloc::passedNumPages>::forEachCommittedPage):
|
||
* bmalloc/IsoHeapImpl.cpp:
|
||
(bmalloc::IsoHeapImplBase::IsoHeapImplBase):
|
||
* bmalloc/IsoHeapImpl.h:
|
||
* bmalloc/IsoHeapImplInlines.h:
|
||
(bmalloc::IsoHeapImpl<Config>::IsoHeapImpl):
|
||
(bmalloc::IsoHeapImpl<Config>::takeFirstEligible):
|
||
(bmalloc::IsoHeapImpl<Config>::didBecomeEligibleOrDecommited):
|
||
(bmalloc::IsoHeapImpl<Config>::scavenge):
|
||
(bmalloc::IsoHeapImpl<Config>::scavengeToHighWatermark):
|
||
(bmalloc::IsoHeapImplBase::freeableMemory):
|
||
(bmalloc::IsoHeapImpl<Config>::numLiveObjects):
|
||
(bmalloc::IsoHeapImpl<Config>::numCommittedPages):
|
||
(bmalloc::IsoHeapImpl<Config>::forEachDirectory):
|
||
(bmalloc::IsoHeapImpl<Config>::forEachCommittedPage):
|
||
(bmalloc::IsoHeapImpl<Config>::forEachLiveObject):
|
||
(bmalloc::IsoHeapImplBase::footprint):
|
||
(bmalloc::IsoHeapImplBase::didCommit):
|
||
(bmalloc::IsoHeapImplBase::didDecommit):
|
||
(bmalloc::IsoHeapImplBase::isNowFreeable):
|
||
(bmalloc::IsoHeapImplBase::isNoLongerFreeable):
|
||
(bmalloc::IsoHeapImpl<Config>::allocateFromShared):
|
||
(bmalloc::IsoHeapImpl<Config>::freeableMemory): Deleted.
|
||
(bmalloc::IsoHeapImpl<Config>::footprint): Deleted.
|
||
(bmalloc::IsoHeapImpl<Config>::didCommit): Deleted.
|
||
(bmalloc::IsoHeapImpl<Config>::didDecommit): Deleted.
|
||
(bmalloc::IsoHeapImpl<Config>::isNowFreeable): Deleted.
|
||
(bmalloc::IsoHeapImpl<Config>::isNoLongerFreeable): Deleted.
|
||
* bmalloc/IsoPage.h:
|
||
(bmalloc::IsoPageBase::IsoPageBase):
|
||
* bmalloc/IsoPageInlines.h:
|
||
(bmalloc::IsoPage<Config>::IsoPage):
|
||
(bmalloc::IsoPage<Config>::free):
|
||
(bmalloc::IsoPage<Config>::startAllocating):
|
||
(bmalloc::IsoPage<Config>::stopAllocating):
|
||
(bmalloc::IsoPage<Config>::forEachLiveObject):
|
||
* bmalloc/IsoSharedHeap.h:
|
||
(bmalloc::IsoSharedHeap::IsoSharedHeap):
|
||
* bmalloc/IsoSharedHeapInlines.h:
|
||
(bmalloc::IsoSharedHeap::allocateNew):
|
||
(bmalloc::IsoSharedHeap::allocateSlow):
|
||
* bmalloc/IsoSharedPage.h:
|
||
* bmalloc/IsoSharedPageInlines.h:
|
||
(bmalloc::IsoSharedPage::free):
|
||
(bmalloc::IsoSharedPage::startAllocating):
|
||
(bmalloc::IsoSharedPage::stopAllocating):
|
||
* bmalloc/IsoTLS.h:
|
||
* bmalloc/IsoTLSAllocatorEntry.h:
|
||
* bmalloc/IsoTLSAllocatorEntryInlines.h:
|
||
(bmalloc::IsoTLSAllocatorEntry<Config>::scavenge):
|
||
* bmalloc/IsoTLSDeallocatorEntry.h:
|
||
* bmalloc/IsoTLSDeallocatorEntryInlines.h:
|
||
(bmalloc::IsoTLSDeallocatorEntry<Config>::scavenge):
|
||
* bmalloc/IsoTLSEntry.cpp:
|
||
(bmalloc::IsoTLSEntry::IsoTLSEntry):
|
||
* bmalloc/IsoTLSEntry.h:
|
||
* bmalloc/IsoTLSEntryInlines.h:
|
||
(bmalloc::DefaultIsoTLSEntry<EntryType>::DefaultIsoTLSEntry):
|
||
(bmalloc::DefaultIsoTLSEntry<EntryType>::~DefaultIsoTLSEntry): Deleted.
|
||
(bmalloc::DefaultIsoTLSEntry<EntryType>::scavenge): Deleted.
|
||
* bmalloc/IsoTLSInlines.h:
|
||
(bmalloc::IsoTLS::scavenge):
|
||
(bmalloc::IsoTLS::allocateImpl):
|
||
(bmalloc::IsoTLS::allocateFast):
|
||
(bmalloc::IsoTLS::allocateSlow):
|
||
* bmalloc/IsoTLSLayout.cpp:
|
||
(bmalloc::IsoTLSLayout::add):
|
||
* bmalloc/Packed.h: Added.
|
||
(bmalloc::Packed::Packed):
|
||
(bmalloc::Packed::get const):
|
||
(bmalloc::Packed::set):
|
||
(bmalloc::Packed::operator=):
|
||
(bmalloc::Packed::exchange):
|
||
(bmalloc::Packed::swap):
|
||
(bmalloc::alignof):
|
||
(bmalloc::PackedPtrTraits::exchange):
|
||
(bmalloc::PackedPtrTraits::swap):
|
||
(bmalloc::PackedPtrTraits::unwrap):
|
||
* bmalloc/Scavenger.cpp:
|
||
(bmalloc::Scavenger::Scavenger):
|
||
* bmalloc/Scavenger.h:
|
||
* bmalloc/VMHeap.cpp:
|
||
(bmalloc::VMHeap::VMHeap):
|
||
* bmalloc/VMHeap.h:
|
||
* bmalloc/Zone.cpp:
|
||
(bmalloc::Zone::Zone):
|
||
* bmalloc/Zone.h:
|
||
|
||
2020-01-14 Basuke Suzuki <basuke.suzuki@sony.com>
|
||
|
||
[bmalloc] Calculate LineMetadata for specific VM page size in compile time
|
||
https://bugs.webkit.org/show_bug.cgi?id=206044
|
||
|
||
Reviewed by Yusuke Suzuki.
|
||
|
||
LineMetadata is dependent only on VM page size. This patch enables the pre-calculation for
|
||
specific VM page sizes by compiler flags. The benefit is both runtime initialization speed
|
||
up and avoiding extra VM allocation on runtime.
|
||
|
||
First targets are 4k (Mac) and 16k (some iOS, PlayStation) VM page sizes.
|
||
|
||
* bmalloc/Algorithm.h:
|
||
(bmalloc::divideRoundingUp):
|
||
* bmalloc/BPlatform.h:
|
||
* bmalloc/HeapConstants.cpp:
|
||
(bmalloc::fillLineMetadata):
|
||
(bmalloc::computeLineMetadata):
|
||
(bmalloc::HeapConstants::initializeLineMetadata):
|
||
* bmalloc/HeapConstants.h:
|
||
(bmalloc::HeapConstants::smallLineCount const):
|
||
(bmalloc::HeapConstants::startOffset const):
|
||
(bmalloc::HeapConstants::objectCount const):
|
||
(bmalloc::HeapConstants::lineMetadata const):
|
||
(bmalloc::HeapConstants::startOffset): Deleted.
|
||
(bmalloc::HeapConstants::objectCount): Deleted.
|
||
(bmalloc::HeapConstants::lineMetadata): Deleted.
|
||
* bmalloc/LineMetadata.h:
|
||
* bmalloc/Sizes.h:
|
||
(bmalloc::Sizes::maskObjectSize):
|
||
(bmalloc::Sizes::logSizeClass):
|
||
(bmalloc::Sizes::logObjectSize):
|
||
(bmalloc::Sizes::sizeClass):
|
||
(bmalloc::Sizes::objectSize):
|
||
(bmalloc::Sizes::pageSize):
|
||
(bmalloc::Sizes::smallLineCount):
|
||
|
||
2020-01-14 David Kilzer <ddkilzer@apple.com>
|
||
|
||
Enable -Wconditional-uninitialized in bmalloc, WTF, JavaScriptCore
|
||
<https://webkit.org/b/206190>
|
||
<rdar://problem/58540387>
|
||
|
||
Reviewed by Mark Lam.
|
||
|
||
* Configurations/Base.xcconfig:
|
||
(WARNING_CFLAGS): Add -Wconditional-uninitialized.
|
||
|
||
2020-01-09 Basuke Suzuki <basuke.suzuki@sony.com>
|
||
|
||
[bmalloc] Extract constants from Heap and share it among Heaps.
|
||
https://bugs.webkit.org/show_bug.cgi?id=205834
|
||
|
||
Reviewed by Geoffrey Garen.
|
||
|
||
A Heap has many constants (m_vmPageSizePhysical, m_smallLineMetadata and m_pageClasses) and they
|
||
are dependent only to vmPageSizePhysical and identical for all Heaps.
|
||
|
||
Extracting them into a class and make it sharable among heaps. Also this is the first step for
|
||
making Heap constants to actual `constexpr`.
|
||
|
||
* CMakeLists.txt: Added HeapConstants.cpp.
|
||
* bmalloc.xcodeproj/project.pbxproj: Ditto.
|
||
* bmalloc/Heap.cpp: Referencing HeapConstants object to get information.
|
||
(bmalloc::Heap::Heap):
|
||
(bmalloc::Heap::allocateSmallPage):
|
||
(bmalloc::Heap::deallocateSmallLine):
|
||
(bmalloc::Heap::allocateSmallBumpRangesByMetadata):
|
||
(bmalloc::Heap::allocateSmallBumpRangesByObject):
|
||
(bmalloc::Heap::initializeLineMetadata): Moved to HeapConstants.cpp.
|
||
(bmalloc::Heap::initializePageMetadata): Moved to HeapConstants.cpp.
|
||
* bmalloc/Heap.h: Extract metadata initialization and member variables.
|
||
* bmalloc/HeapConstants.cpp: Added.
|
||
(bmalloc::HeapConstants::HeapConstants):
|
||
(bmalloc::HeapConstants::initializeLineMetadata):
|
||
(bmalloc::HeapConstants::initializePageMetadata):
|
||
* bmalloc/HeapConstants.h:
|
||
(bmalloc::HeapConstants::pageClass const):
|
||
(bmalloc::HeapConstants::smallLineCount const):
|
||
(bmalloc::HeapConstants::startOffset):
|
||
(bmalloc::HeapConstants::objectCount):
|
||
(bmalloc::HeapConstants::lineMetadata):
|
||
|
||
2020-01-02 Yusuke Suzuki <ysuzuki@apple.com> and Simon Fraser <simon.fraser@apple.com>
|
||
|
||
Experiment: create lots of different malloc zones for easier accounting of memory use
|
||
https://bugs.webkit.org/show_bug.cgi?id=186422
|
||
|
||
Reviewed by Saam Barati.
|
||
|
||
* bmalloc/BPlatform.h:
|
||
* bmalloc/Environment.cpp:
|
||
(bmalloc::Environment::computeIsDebugHeapEnabled):
|
||
* bmalloc/IsoHeap.h:
|
||
(bmalloc::api::IsoHeap::IsoHeap):
|
||
* bmalloc/IsoHeapInlines.h:
|
||
(bmalloc::api::IsoHeap<Type>::IsoHeap):
|
||
* bmalloc/IsoTLSInlines.h:
|
||
(bmalloc::IsoTLS::allocateSlow):
|
||
(bmalloc::IsoTLS::deallocateSlow):
|
||
|
||
2019-12-07 Basuke Suzuki <basuke.suzuki@sony.com>
|
||
|
||
[bmalloc] Decommit unused region in chunk metadata.
|
||
https://bugs.webkit.org/show_bug.cgi?id=204810
|
||
|
||
Reviewed by Yusuke Suzuki.
|
||
|
||
There is an unused memory region from just after Chunk object to next page border.
|
||
We can decommit those memory to kernel at the initialization of Chunk.
|
||
|
||
* bmalloc/Heap.cpp:
|
||
(bmalloc::Heap::allocateSmallChunk):
|
||
|
||
2019-11-25 Fujii Hironori <Hironori.Fujii@sony.com>
|
||
|
||
Ran sort-Xcode-project-file.
|
||
|
||
Unreviewed.
|
||
|
||
* bmalloc.xcodeproj/project.pbxproj:
|
||
|
||
2019-11-18 Basuke Suzuki <basuke.suzuki@sony.com>
|
||
|
||
[bmalloc] Some chunks have unused region in the tail of its memory block.
|
||
https://bugs.webkit.org/show_bug.cgi?id=204286
|
||
|
||
Reviewed by Yusuke Suzuki.
|
||
|
||
When chunk is initialized, some amount of memory are not used and be kept untouched until its end.
|
||
This patch tries to decommit those region at the end of initialization.
|
||
|
||
For instance, think about the case that the pageClass is 5. Then pageSize is 24k. With this pageSize,
|
||
a chunk can hold 42 pages and its size is 24k * 42 = 1008k which is smaller than chunkSize = 1024k.
|
||
Here is the complete result:
|
||
|
||
page page page
|
||
class size count remainings
|
||
----------------------------------
|
||
0 4kB 256 0
|
||
1 8kB 128 0
|
||
2 12kB 85 4kB
|
||
3 16kB 64 0
|
||
4 20kB 51 4kB
|
||
5 24kB 42 16kB
|
||
6 28kB 36 16kB
|
||
7 32kB 32 0
|
||
8 36kB 28 16kB
|
||
9 40kB 25 24kB
|
||
10 44kB 23 12kB
|
||
11 48kB 21 16kB
|
||
12 52kB 19 36kB
|
||
13 56kB 18 16kB
|
||
14 60kB 17 4kB
|
||
15 64kB 16 0
|
||
|
||
Tested on Mac testmem and result is almost same or in error margin.
|
||
|
||
Before: After:
|
||
end score: 8.5425 MB end score: 8.5127 MB
|
||
peak score: 8.7997 MB peak score: 8.7884 MB
|
||
total memory score: 8.6702 MB total memory score: 8.6495 MB
|
||
time score: 668.19 ms time score: 666.27 ms
|
||
|
||
* bmalloc/Chunk.h:
|
||
(bmalloc::Chunk::metadataSize):
|
||
(bmalloc::forEachPage):
|
||
* bmalloc/Heap.cpp:
|
||
(bmalloc::Heap::allocateSmallChunk):
|
||
|
||
2019-11-15 Basuke Suzuki <basuke.suzuki@sony.com>
|
||
|
||
[bmalloc] The tracking of freeableMemory of Heap doesn't count Chunk's metadata size.
|
||
https://bugs.webkit.org/show_bug.cgi?id=204135
|
||
|
||
Reviewed by Yusuke Suzuki.
|
||
|
||
When chunk is allocated in allocateSmallChunk(), all chunk size is added to freeableMemory.
|
||
This is wrong. Only free pages should be added to it.
|
||
|
||
* bmalloc/Heap.cpp:
|
||
(bmalloc::Heap::allocateSmallChunk):
|
||
|
||
2019-11-15 Basuke Suzuki <basuke.suzuki@sony.com>
|
||
|
||
[Mac] Use better describing name for Mac's scavenger compiler flag.
|
||
https://bugs.webkit.org/show_bug.cgi?id=203922
|
||
|
||
Reviewed by Yusuke Suzuki.
|
||
|
||
Changing BPLATFORM(MAC) to BUSE(PARTIAL_SCAVENGE).
|
||
|
||
* bmalloc/BPlatform.h:
|
||
* bmalloc/Heap.cpp:
|
||
(bmalloc::Heap::scavenge):
|
||
(bmalloc::Heap::allocateSmallChunk):
|
||
(bmalloc::Heap::allocateSmallPage):
|
||
(bmalloc::Heap::allocateLarge):
|
||
* bmalloc/Heap.h:
|
||
* bmalloc/IsoDirectory.h:
|
||
* bmalloc/IsoDirectoryInlines.h:
|
||
(bmalloc::passedNumPages>::takeFirstEligible):
|
||
(bmalloc::passedNumPages>::scavenge):
|
||
* bmalloc/IsoHeapImpl.h:
|
||
* bmalloc/IsoHeapImplInlines.h:
|
||
* bmalloc/LargeMap.cpp:
|
||
(bmalloc::LargeMap::add):
|
||
* bmalloc/LargeRange.h:
|
||
(bmalloc::LargeRange::LargeRange):
|
||
(bmalloc::merge):
|
||
* bmalloc/Scavenger.cpp:
|
||
(bmalloc::Scavenger::Scavenger):
|
||
(bmalloc::Scavenger::scavenge):
|
||
(bmalloc::Scavenger::threadRunLoop):
|
||
* bmalloc/Scavenger.h:
|
||
* bmalloc/SmallPage.h:
|
||
|
||
2019-11-07 Basuke Suzuki <Basuke.Suzuki@sony.com> and Yusuke Suzuki <ysuzuki@apple.com>
|
||
|
||
[bmalloc] Bug fix for wait time when it's in mini mode.
|
||
https://bugs.webkit.org/show_bug.cgi?id=203121
|
||
|
||
Reviewed by Saam Barati.
|
||
|
||
Basuke pointed out that we never changed m_waitTime in scavenger when we are in iOS and mini-mode.
|
||
So previously, we are always executing scavenger for each 10ms in mini-mode. After fixing it, we
|
||
found that this unintentional behavior was better policy for RAMification.
|
||
In this patch, we explicitly use the old behavior, "scavenge for each 10ms" in iOS mini-mode.
|
||
We should revisit scavenger's behavior in the future to explore further better behavior.
|
||
|
||
* bmalloc/Scavenger.cpp:
|
||
(bmalloc::Scavenger::threadRunLoop):
|
||
|
||
2019-11-06 Yusuke Suzuki <ysuzuki@apple.com>
|
||
|
||
Unreviewed, roll-out r251268 due to RAMification regression
|
||
https://bugs.webkit.org/show_bug.cgi?id=203121
|
||
|
||
The code was the bug, but it would be possible that this bug accidentally makes our policy better.
|
||
We should clean up to make this bug's behavior default. And we should look for a bit larger interval here.
|
||
|
||
* bmalloc/Scavenger.cpp:
|
||
(bmalloc::Scavenger::threadRunLoop):
|
||
|
||
2019-10-17 Basuke Suzuki <Basuke.Suzuki@sony.com>
|
||
|
||
[bmalloc] Bug fix for wait time when it's in mini mode.
|
||
https://bugs.webkit.org/show_bug.cgi?id=203121
|
||
|
||
Reviewed by Yusuke Suzuki.
|
||
|
||
Since r243144, m_waitTime is never changed on mini mode.
|
||
|
||
* bmalloc/Scavenger.cpp:
|
||
(bmalloc::Scavenger::threadRunLoop):
|
||
|
||
2019-10-07 Wenson Hsieh <wenson_hsieh@apple.com>
|
||
|
||
Add a missing #include in DeferredTriggerInlines.h
|
||
https://bugs.webkit.org/show_bug.cgi?id=202677
|
||
|
||
Reviewed by Yusuke Suzuki.
|
||
|
||
DeferredTriggerInlines.h uses RELEASE_BASSERT, but does not include BAssert.h. This causes
|
||
subsequent build failures when shuffling around some sources in WebCore that then attempt to
|
||
include <wtf/IsoMallocInlines.h>.
|
||
|
||
* bmalloc/DeferredTriggerInlines.h:
|
||
|
||
2019-09-24 Keith Rollin <krollin@apple.com>
|
||
|
||
Address static analysis warning in Allocator.cpp: Null pointer argument in call to memory copy function
|
||
https://bugs.webkit.org/show_bug.cgi?id=202152
|
||
<rdar://problem/55671444>
|
||
|
||
Reviewed by Geoffrey Garen.
|
||
|
||
Xcode's static analysis facility flags the following:
|
||
|
||
.../OpenSource/Source/bmalloc/bmalloc/Allocator.cpp:98:5: warning: Null pointer argument in call to memory copy function
|
||
memcpy(result, object, copySize);
|
||
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||
|
||
https://en.cppreference.com/w/cpp/string/byte/memcpy explains that
|
||
this is undefined behavior:
|
||
|
||
"If either dest or src is a null pointer, the behavior is
|
||
undefined, even if count is zero."
|
||
|
||
I suppose that passing in a null source pointer could be bad if the
|
||
implementation fetched the first source byte to copy before actually
|
||
checking the number of bytes to copy. So far, it doesn’t seem to be an
|
||
issue, but we should clean this up.
|
||
|
||
Simply adding "if (result && object)" before the memcpy will add tests
|
||
and branches in the hot path of this function and so might not be the
|
||
best solution. Instead, straighten out the code a little bit by
|
||
putting an early check and return on "object". This also allows us to
|
||
remove some intermediate code.
|
||
|
||
* bmalloc/Allocator.cpp:
|
||
(bmalloc::Allocator::reallocateImpl):
|
||
|
||
2019-09-17 Mark Lam <mark.lam@apple.com>
|
||
|
||
Use constexpr instead of const in symbol definitions that are obviously constexpr.
|
||
https://bugs.webkit.org/show_bug.cgi?id=201879
|
||
|
||
Rubber-stamped by Joseph Pecoraro.
|
||
|
||
* bmalloc/AvailableMemory.cpp:
|
||
* bmalloc/IsoTLS.h:
|
||
* bmalloc/Map.h:
|
||
* bmalloc/Mutex.cpp:
|
||
(bmalloc::Mutex::lockSlowCase):
|
||
* bmalloc/PerThread.h:
|
||
* bmalloc/Vector.h:
|
||
* bmalloc/Zone.h:
|
||
|
||
2019-09-13 Mark Lam <mark.lam@apple.com>
|
||
|
||
We should never fail to allocate VA for the Gigacage.
|
||
https://bugs.webkit.org/show_bug.cgi?id=201774
|
||
<rdar://problem/55352042>
|
||
|
||
Reviewed by Yusuke Suzuki.
|
||
|
||
Unfortunately, this is still not true for OS(LINUX). So, I'll leave the
|
||
GIGACAGE_ALLOCATION_CAN_FAIL code in place but disable it for everything else.
|
||
|
||
* bmalloc/Gigacage.h:
|
||
|
||
2019-09-09 Michael Saboff <msaboff@apple.com>
|
||
|
||
Revert to pre-r243144 scavenging behavior for macOS
|
||
https://bugs.webkit.org/show_bug.cgi?id=201555
|
||
|
||
Reviewed by Saam Barati.
|
||
|
||
The change in r243144 regressed internal power metrics for some Mac models.
|
||
|
||
* bmalloc/Heap.cpp:
|
||
(bmalloc::Heap::scavenge):
|
||
(bmalloc::Heap::scavengeToHighWatermark):
|
||
(bmalloc::Heap::allocateSmallChunk):
|
||
(bmalloc::Heap::allocateSmallPage):
|
||
(bmalloc::Heap::allocateLarge):
|
||
* bmalloc/Heap.h:
|
||
* bmalloc/IsoDirectory.h:
|
||
* bmalloc/IsoDirectoryInlines.h:
|
||
(bmalloc::passedNumPages>::takeFirstEligible):
|
||
(bmalloc::passedNumPages>::scavenge):
|
||
(bmalloc::passedNumPages>::scavengeToHighWatermark):
|
||
* bmalloc/IsoHeapImpl.h:
|
||
* bmalloc/IsoHeapImplInlines.h:
|
||
(bmalloc::IsoHeapImpl<Config>::scavengeToHighWatermark):
|
||
* bmalloc/LargeMap.cpp:
|
||
(bmalloc::LargeMap::add):
|
||
* bmalloc/LargeRange.h:
|
||
(bmalloc::LargeRange::LargeRange):
|
||
(bmalloc::merge):
|
||
* bmalloc/Scavenger.cpp:
|
||
(bmalloc::Scavenger::Scavenger):
|
||
(bmalloc::Scavenger::timeSinceLastPartialScavenge):
|
||
(bmalloc::Scavenger::scavenge):
|
||
(bmalloc::Scavenger::partialScavenge):
|
||
(bmalloc::Scavenger::threadRunLoop):
|
||
* bmalloc/Scavenger.h:
|
||
* bmalloc/SmallPage.h:
|
||
|
||
2019-09-07 Mark Lam <mark.lam@apple.com>
|
||
|
||
[Follow up fix] Harden protection of the Gigacage Config parameters.
|
||
https://bugs.webkit.org/show_bug.cgi?id=201570
|
||
<rdar://problem/55134229>
|
||
|
||
Reviewed by Saam Barati.
|
||
|
||
The previously landed hardening measures is failing the assertion in
|
||
forbidDisablingPrimitiveGigacage() when a replacement heap (e.g. gmalloc) is
|
||
used.
|
||
|
||
1. forbidDisablingPrimitiveGigacage() should just ensureGigacage() so that we
|
||
know that the Gigacage is properly configured (whether enabled or disabled).
|
||
2. Strengthen the assertion in forbidDisablingPrimitiveGigacage() to ensure that
|
||
we have already determined if the Gigacage should be enabled.
|
||
|
||
* bmalloc/Gigacage.cpp:
|
||
(Gigacage::forbidDisablingPrimitiveGigacage):
|
||
|
||
2019-09-07 Mark Lam <mark.lam@apple.com>
|
||
|
||
performJITMemcpy() source buffer should not be in the Gigacage.
|
||
https://bugs.webkit.org/show_bug.cgi?id=201577
|
||
<rdar://problem/55142606>
|
||
|
||
Reviewed by Michael Saboff.
|
||
|
||
1. Add the Gigacage start address and totalSize to the Config.
|
||
2. Add a contains() function that uses the start address and totalSize to check
|
||
if a given pointer is in the Gigacage's address range.
|
||
|
||
* bmalloc/Gigacage.cpp:
|
||
(Gigacage::ensureGigacage):
|
||
(Gigacage::verifyGigacageIsEnabled):
|
||
* bmalloc/Gigacage.h:
|
||
(Gigacage::contains):
|
||
|
||
2019-09-06 Mark Lam <mark.lam@apple.com>
|
||
|
||
Harden protection of the Gigacage Config parameters.
|
||
https://bugs.webkit.org/show_bug.cgi?id=201570
|
||
<rdar://problem/55134229>
|
||
|
||
Reviewed by Saam Barati.
|
||
|
||
1. Rename protectGigacageBasePtrs() and unprotectGigacageBasePtrs() to
|
||
freezeGigacageConfig() and unfreezeGigacageConfig() respectively.
|
||
|
||
Also move the alignment check in protectGigacageBasePtrs() into ensureGigacage().
|
||
There's no need to check it more than once.
|
||
|
||
2. Introduce permanentlyFreezeGigacageConfig() which permanently makes the
|
||
Config ReadOnly. Once invoked, the Config cannot be made writable again.
|
||
|
||
This is made possible by using vm_protect with a true set_maximum argument.
|
||
We also add a g_gigacageConfig.isPermanentlyFrozen flag that we assert.
|
||
Note: this permanence is only true for OS(DARWIN) since vm_protect is Mach API.
|
||
|
||
3. Rename disableDisablingPrimitiveGigacageIfShouldBeEnabled() to
|
||
forbidDisablingPrimitiveGigacage() because "disablingDisabling" is a tongue
|
||
twister.
|
||
|
||
Also, we don't want to make it conditional on "IfShouldBeEnabled". We want
|
||
forbidDisablingPrimitiveGigacage() to be irreversible. It is up to the client
|
||
to ensure that the Gigacage is already initialized (if possible) before
|
||
invoking forbidDisablingPrimitiveGigacage(). Conceptually, the forbidding
|
||
isn't about guaranteeing that the Gigacage is enabled. It only takes away the
|
||
option to disable it.
|
||
|
||
That said, forbidDisablingPrimitiveGigacage() is only invoked by clients that
|
||
care about keeping the Gigacage enabled. So, it does a sanity check (with an
|
||
assertion) that if !GIGACAGE_ALLOCATION_CAN_FAIL, then the Gigacage should be
|
||
have been initialized and enabled before invoking it.
|
||
|
||
We also make sure that forbidDisablingPrimitiveGigacage() calls
|
||
permanentlyFreezeGigacageConfig() unconditionally. It is safe to call it more
|
||
than once. This guarantees that the Config is permanently frozen after this,
|
||
even if a bug should inadvertantly set the
|
||
g_gigacageConfig.disablingPrimitiveGigacageIsForbidden flag before
|
||
forbidDisablingPrimitiveGigacage() is invoked.
|
||
|
||
4. Assert that ensureGigacage() is only called once.
|
||
|
||
5. Assert that shouldBeEnabled() is only called once.
|
||
Also moved its cached result into the Config so that it can be frozen.
|
||
|
||
* bmalloc/Gigacage.cpp:
|
||
(Gigacage::bmalloc::freezeGigacageConfig):
|
||
(Gigacage::bmalloc::unfreezeGigacageConfig):
|
||
(Gigacage::bmalloc::permanentlyFreezeGigacageConfig):
|
||
(Gigacage::bmalloc::UnfreezeGigacageConfigScope::UnfreezeGigacageConfigScope):
|
||
(Gigacage::bmalloc::UnfreezeGigacageConfigScope::~UnfreezeGigacageConfigScope):
|
||
(Gigacage::ensureGigacage):
|
||
(Gigacage::disablePrimitiveGigacage):
|
||
(Gigacage::verifyGigacageIsEnabled):
|
||
(Gigacage::forbidDisablingPrimitiveGigacage):
|
||
(Gigacage::isDisablingPrimitiveGigacageForbidden):
|
||
(Gigacage::shouldBeEnabled):
|
||
(Gigacage::bmalloc::protectGigacageBasePtrs): Deleted.
|
||
(Gigacage::bmalloc::unprotectGigacageBasePtrs): Deleted.
|
||
(Gigacage::bmalloc::UnprotectGigacageBasePtrsScope::UnprotectGigacageBasePtrsScope): Deleted.
|
||
(Gigacage::bmalloc::UnprotectGigacageBasePtrsScope::~UnprotectGigacageBasePtrsScope): Deleted.
|
||
(Gigacage::primitiveGigacageDisabled): Deleted.
|
||
(Gigacage::disableDisablingPrimitiveGigacageIfShouldBeEnabled): Deleted.
|
||
(Gigacage::isDisablingPrimitiveGigacageDisabled): Deleted.
|
||
* bmalloc/Gigacage.h:
|
||
(Gigacage::isPrimitiveGigacagePermanentlyEnabled):
|
||
(Gigacage::canPrimitiveGigacageBeDisabled):
|
||
(Gigacage::forbidDisablingPrimitiveGigacage):
|
||
(Gigacage::disableDisablingPrimitiveGigacageIfShouldBeEnabled): Deleted.
|
||
|
||
2019-09-06 Mark Lam <mark.lam@apple.com>
|
||
|
||
Use FailureAction everywhere in bmalloc instead of a crashOnFailure boolean.
|
||
https://bugs.webkit.org/show_bug.cgi?id=201553
|
||
|
||
Reviewed by Yusuke Suzuki.
|
||
|
||
* bmalloc/Cache.cpp:
|
||
(bmalloc::Cache::tryAllocateSlowCaseNullCache):
|
||
(bmalloc::Cache::allocateSlowCaseNullCache):
|
||
(bmalloc::Cache::tryReallocateSlowCaseNullCache):
|
||
(bmalloc::Cache::reallocateSlowCaseNullCache):
|
||
* bmalloc/DebugHeap.cpp:
|
||
(bmalloc::DebugHeap::malloc):
|
||
(bmalloc::DebugHeap::memalign):
|
||
(bmalloc::DebugHeap::realloc):
|
||
* bmalloc/DebugHeap.h:
|
||
|
||
2019-09-06 Mark Lam <mark.lam@apple.com>
|
||
|
||
Fix bmalloc::Allocator:tryAllocate() to return null on failure to allocate.
|
||
https://bugs.webkit.org/show_bug.cgi?id=201529
|
||
<rdar://problem/53935772>
|
||
|
||
Reviewed by Yusuke Suzuki.
|
||
|
||
In this implementation, we pass FailureAction in as a runtime option. If this
|
||
proves to be a perf issue, we can easily fix this by passing it as a template
|
||
argument. That will also automatically elide unneeded code paths. We'll defer
|
||
that exercise until we have evidence that it is warranted.
|
||
|
||
* CMakeLists.txt:
|
||
* bmalloc.xcodeproj/project.pbxproj:
|
||
* bmalloc/Allocator.cpp:
|
||
(bmalloc::Allocator::allocateImpl):
|
||
(bmalloc::Allocator::reallocateImpl):
|
||
(bmalloc::Allocator::refillAllocatorSlowCase):
|
||
(bmalloc::Allocator::refillAllocator):
|
||
(bmalloc::Allocator::allocateLarge):
|
||
(bmalloc::Allocator::allocateLogSizeClass):
|
||
(bmalloc::Allocator::allocateSlowCase):
|
||
(bmalloc::Allocator::tryAllocate): Deleted.
|
||
(bmalloc::Allocator::allocate): Deleted.
|
||
(bmalloc::Allocator::reallocate): Deleted.
|
||
(bmalloc::Allocator::tryReallocate): Deleted.
|
||
* bmalloc/Allocator.h:
|
||
(bmalloc::Allocator::tryAllocate):
|
||
(bmalloc::Allocator::allocate):
|
||
(bmalloc::Allocator::tryReallocate):
|
||
(bmalloc::Allocator::reallocate):
|
||
(bmalloc::Allocator::allocateImpl):
|
||
* bmalloc/BumpAllocator.h:
|
||
* bmalloc/FailureAction.h: Added.
|
||
* bmalloc/Heap.cpp:
|
||
(bmalloc::Heap::allocateSmallChunk):
|
||
(bmalloc::Heap::allocateSmallPage):
|
||
(bmalloc::Heap::allocateSmallBumpRangesByMetadata):
|
||
(bmalloc::Heap::allocateSmallBumpRangesByObject):
|
||
(bmalloc::Heap::allocateLarge):
|
||
(bmalloc::Heap::tryAllocateLarge): Deleted.
|
||
* bmalloc/Heap.h:
|
||
(bmalloc::Heap::allocateSmallBumpRanges):
|
||
* bmalloc/bmalloc.cpp:
|
||
(bmalloc::api::tryLargeZeroedMemalignVirtual):
|
||
|
||
2019-09-05 Mark Lam <mark.lam@apple.com>
|
||
|
||
Refactor the Gigacage code to require less pointer casting.
|
||
https://bugs.webkit.org/show_bug.cgi?id=201521
|
||
|
||
Reviewed by Saam Barati.
|
||
|
||
1. Define a Gigacage::Config struct instead of hacking around a g_gigacageBasePtrs
|
||
array of bytes.
|
||
2. Change Gigacage::basePtr() to return a pointer instead of a reference to the
|
||
requested basePtr. Instead, make it explicit when the client is trying to
|
||
take the address of the basePtr, or setting it.
|
||
3. Renamed wasEnabled() to isEnabled() because it returns the present state of the
|
||
flag, not some past state.
|
||
|
||
* bmalloc/Gigacage.cpp:
|
||
(Gigacage::bmalloc::protectGigacageBasePtrs):
|
||
(Gigacage::bmalloc::unprotectGigacageBasePtrs):
|
||
(Gigacage::bmalloc::runwaySize):
|
||
(Gigacage::ensureGigacage):
|
||
(Gigacage::disablePrimitiveGigacage):
|
||
(Gigacage::addPrimitiveDisableCallback):
|
||
(Gigacage::primitiveGigacageDisabled):
|
||
* bmalloc/Gigacage.h:
|
||
(Gigacage::name):
|
||
(Gigacage::Config::basePtr const):
|
||
(Gigacage::Config::setBasePtr):
|
||
(Gigacage::isEnabled):
|
||
(Gigacage::basePtr):
|
||
(Gigacage::addressOfBasePtr):
|
||
(Gigacage::size):
|
||
(Gigacage::caged):
|
||
(Gigacage::wasEnabled): Deleted.
|
||
(Gigacage::setWasEnabled): Deleted.
|
||
(Gigacage::basePtrs): Deleted.
|
||
* bmalloc/HeapKind.h:
|
||
(bmalloc::heapKind):
|
||
(bmalloc::isActiveHeapKindAfterEnsuringGigacage):
|
||
(bmalloc::mapToActiveHeapKindAfterEnsuringGigacage):
|
||
|
||
2019-09-03 Yusuke Suzuki <ysuzuki@apple.com>
|
||
|
||
[bmalloc] IsoTLSLayout and AllIsoHeaps registration is racy with derived class initialization with virtual functions
|
||
https://bugs.webkit.org/show_bug.cgi?id=201448
|
||
|
||
Reviewed by Mark Lam.
|
||
|
||
In the base class of IsoTLSEntry and IsoHeapImplBase, we register each instance with the per-process linked-list singleton to
|
||
offer a way to iterate all these instances. But since derived classes of IsoTLSEntry and IsoHeapImplBase have virtual functions,
|
||
the instance is not fully instantiated yet when executing the base constructor! In particular, the register instance needs vtable pointer
|
||
initialization in the derived constructor.
|
||
|
||
So, there is a race condition,
|
||
|
||
1. IsoTLSEntry adds itself to the global linked-list.
|
||
2. IsoTLSEntry's derived class is initializing the instance including vtable pointer, this happens because base and derived classes have virtual functions.
|
||
3. While doing (2), other thread iterates instances through (1)'s linked-list and call virtual functions
|
||
|
||
Then, crash happens because the instance vtable pointer hasn't been set to the derived class' vtable yet. IsoHeapImpl has the same problem.
|
||
This issue causes some crashes in bmalloc::Scavenger::scavenge / bmalloc::IsoTLS::ensureEntries.
|
||
|
||
In this patch,
|
||
|
||
1. We introduce IsoTLSEntryHolder, which initialize the TLS entry. And after fully initializing it, the holder registers the entry with the IsoTLSLayout singleton.
|
||
2. We call IsoHeapImplBase::addToAllIsoHeaps after IsoHeapImpl is fully initialized.
|
||
3. We put memory barrier in IsoTLSLayout since IsoTLSLayout::head does not take a lock.
|
||
4. We add unit-test that reliably reproduces IsoHeapImpl crash if we run this test ~10 times!
|
||
|
||
* bmalloc/AllIsoHeaps.h:
|
||
* bmalloc/IsoHeapImpl.h:
|
||
* bmalloc/IsoHeapImplInlines.h:
|
||
(bmalloc::IsoHeapImpl<Config>::IsoHeapImpl):
|
||
(bmalloc::IsoHeapImpl<Config>::allocatorOffset):
|
||
(bmalloc::IsoHeapImpl<Config>::deallocatorOffset):
|
||
* bmalloc/IsoHeapInlines.h:
|
||
(bmalloc::api::IsoHeap<Type>::initialize):
|
||
* bmalloc/IsoTLSAllocatorEntry.h:
|
||
* bmalloc/IsoTLSDeallocatorEntry.h:
|
||
* bmalloc/IsoTLSEntry.cpp:
|
||
(bmalloc::IsoTLSEntry::IsoTLSEntry):
|
||
* bmalloc/IsoTLSEntry.h:
|
||
(bmalloc::IsoTLSEntryHolder::IsoTLSEntryHolder):
|
||
(bmalloc::IsoTLSEntryHolder::operator* const):
|
||
(bmalloc::IsoTLSEntryHolder::operator*):
|
||
(bmalloc::IsoTLSEntryHolder::operator-> const):
|
||
(bmalloc::IsoTLSEntryHolder::operator->):
|
||
* bmalloc/IsoTLSLayout.cpp:
|
||
(bmalloc::IsoTLSLayout::add):
|
||
* bmalloc/IsoTLSLayout.h:
|
||
|
||
2019-08-29 Keith Rollin <krollin@apple.com>
|
||
|
||
Update .xcconfig symbols to reflect the current set of past and future product versions.
|
||
https://bugs.webkit.org/show_bug.cgi?id=200720
|
||
<rdar://problem/54305032>
|
||
|
||
Reviewed by Alex Christensen.
|
||
|
||
Remove version symbols related to old OS's we no longer support,
|
||
ensure that version symbols are defined for OS's we do support.
|
||
|
||
* Configurations/Base.xcconfig:
|
||
* Configurations/DebugRelease.xcconfig:
|
||
|
||
2019-08-26 Yusuke Suzuki <ysuzuki@apple.com>
|
||
|
||
[bmalloc] Disable IsoHeap completely if DebugHeap is enabled
|
||
https://bugs.webkit.org/show_bug.cgi?id=201154
|
||
|
||
Reviewed by Simon Fraser.
|
||
|
||
Previously we had the guarantee that IsoHeap is disabled when DebugHeap is enabled.
|
||
But this is guaranteed in a bit tricky way: when DebugHeap is enabled, Gigacage is disabled.
|
||
And IsoHeap is disabled when Gigacage is disabled. However r249065 enabled IsoHeap even if
|
||
Gigacage is disabled. This accidentally enabled IsoHeap even if DebugHeap is enabled.
|
||
|
||
Currently, this is incorrect. When DebugHeap is enabled, we do not start bmalloc::Scavenger.
|
||
So IsoHeap does not work. In addition, when DebugHeap is enabled, we want to investigate the Malloc data.
|
||
However IsoHeap wipes these information for IsoHeaped objects. Moreover enabling IsoHeap is not free
|
||
in terms of memory usage: bmalloc::Scavenger starts working.
|
||
|
||
So we should not enable IsoHeap in such an accidental way for DebugHeap environment. If we consider enabling
|
||
IsoHeap even if `Malloc=1` is specified, we should first examine how memory is used by this change because
|
||
the users of `Malloc=1` requires explicitly tight memory usage.
|
||
|
||
In this patch, we remove the accidental enabling of IsoHeap for DebugHeap by checking DebugHeap status in IsoTLS.
|
||
|
||
* bmalloc/IsoTLS.cpp:
|
||
(bmalloc::IsoTLS::determineMallocFallbackState):
|
||
|
||
2019-08-22 Mark Lam <mark.lam@apple.com>
|
||
|
||
Undo disabling of IsoHeaps when Gigacage is off.
|
||
https://bugs.webkit.org/show_bug.cgi?id=201061
|
||
<rdar://problem/54622500>
|
||
|
||
Reviewed by Saam Barati and Michael Saboff.
|
||
|
||
* CMakeLists.txt:
|
||
* bmalloc.xcodeproj/project.pbxproj:
|
||
* bmalloc/IsoTLS.cpp:
|
||
(bmalloc::IsoTLS::determineMallocFallbackState):
|
||
* bmalloc/PerThread.cpp: Removed.
|
||
* bmalloc/PerThread.h:
|
||
|
||
2019-08-19 Yusuke Suzuki <ysuzuki@apple.com>
|
||
|
||
[WTF] Add makeUnique<T>, which ensures T is fast-allocated, actual static_assert part
|
||
https://bugs.webkit.org/show_bug.cgi?id=200620
|
||
|
||
Reviewed by Geoff Garen.
|
||
|
||
Expose T::webkitFastMalloced type to perform static_assert in makeUnique.
|
||
|
||
* bmalloc/IsoHeap.h:
|
||
* bmalloc/IsoHeapInlines.h:
|
||
|
||
2019-07-24 Yusuke Suzuki <ysuzuki@apple.com>
|
||
|
||
[bmalloc] Add IsoHeap test to ensure that IsoHeap pages are not allocating too large VA
|
||
https://bugs.webkit.org/show_bug.cgi?id=200103
|
||
|
||
Reviewed by Mark Lam.
|
||
|
||
* bmalloc/IsoPage.cpp:
|
||
|
||
2019-07-22 Yusuke Suzuki <ysuzuki@apple.com>
|
||
|
||
[bmalloc] Each IsoPage gets 1MB VA because VMHeap::tryAllocateLargeChunk rounds up
|
||
https://bugs.webkit.org/show_bug.cgi?id=200024
|
||
|
||
Reviewed by Saam Barati.
|
||
|
||
When we allocate IsoHeap's page, we reused VMHeap::tryAllocateLargeChunk. However, this function is originally designed
|
||
to be used for Large allocation in bmalloc (e.g. allocating Chunk in bmalloc). As a result, this function rounds up the
|
||
requested size with 1MB (bmalloc::chunkSize). As a result, all IsoHeap's 16KB page gets 1MB VA while it just uses 16KB of
|
||
the allocated region. This leads to VA exhaustion since IsoHeap now uses 64x VA than we expected!
|
||
|
||
This patch fixes the above VA exhaustion issue by allocating a page by using tryVMAllocate. When allocating a page, we start
|
||
using a VM tag for IsoHeap. We discussed at e-mail and we decided reusing a VM tag previously assigned to CLoop Stack since
|
||
this is less profitable. Since this tag is not Malloc-related tag, Leaks tool can scan memory region conservatively without
|
||
registering allocated region into Zone, which was previously done in VMHeap and that's why we reused VMHeap for IsoHeap.
|
||
|
||
* bmalloc/BVMTags.h:
|
||
* bmalloc/IsoPage.cpp:
|
||
(bmalloc::IsoPageBase::allocatePageMemory):
|
||
* bmalloc/IsoTLS.cpp:
|
||
(bmalloc::IsoTLS::ensureEntries):
|
||
* bmalloc/VMAllocate.h:
|
||
(bmalloc::vmAllocate):
|
||
|
||
2019-07-22 Yusuke Suzuki <ysuzuki@apple.com>
|
||
|
||
Unreviewed, follow-up fix for tls->size() access
|
||
https://bugs.webkit.org/show_bug.cgi?id=200019
|
||
|
||
* bmalloc/IsoTLS.cpp:
|
||
(bmalloc::IsoTLS::ensureEntries):
|
||
(bmalloc::IsoTLS::destructor):
|
||
|
||
2019-07-22 Yusuke Suzuki <ysuzuki@apple.com>
|
||
|
||
[bmalloc] IsoTLS is not deallocated in TLS destructor
|
||
https://bugs.webkit.org/show_bug.cgi?id=200019
|
||
|
||
Reviewed by Mark Lam.
|
||
|
||
TLS destructor for IsoTLS needs to deallocate memory used for IsoTLS itself.
|
||
While we are correctly deallocating old IsoTLS when extending it to the new one,
|
||
we lack deallocation when the thread is destroyed. IsoTLS is per-thread, so we
|
||
should destroy IsoTLS when a thread finishes.
|
||
|
||
* bmalloc/IsoTLS.cpp:
|
||
(bmalloc::IsoTLS::ensureEntries):
|
||
(bmalloc::IsoTLS::destructor):
|
||
|
||
2019-07-12 Keith Miller <keith_miller@apple.com>
|
||
|
||
Increase JSValue cage size on iOS and reduce the max slide
|
||
https://bugs.webkit.org/show_bug.cgi?id=199765
|
||
|
||
Reviewed by Saam Barati.
|
||
|
||
Since the WebContent jetsam limit has changed we sometimes run out
|
||
of JSValue cage VA space causing us to run out of memory for
|
||
arrays. This change makes the JSValue cage a more reasonable upper
|
||
limit for what should be possible without jetsamming.
|
||
|
||
The worst case memory mapping with this configuration is has not
|
||
changed from before. Under both configurations we could map 36GB
|
||
with a temporary mapping of 38GB (to align the VA to 2GB).
|
||
|
||
* bmalloc/Gigacage.h:
|
||
|
||
2019-07-03 Keith Miller <keith_miller@apple.com>
|
||
|
||
PACCage should first cage leaving PAC bits intact then authenticate
|
||
https://bugs.webkit.org/show_bug.cgi?id=199372
|
||
|
||
Reviewed by Saam Barati.
|
||
|
||
* bmalloc/ProcessCheck.mm:
|
||
(bmalloc::shouldProcessUnconditionallyUseBmalloc):
|
||
|
||
2019-07-02 Commit Queue <commit-queue@webkit.org>
|
||
|
||
Unreviewed, rolling out r247041.
|
||
https://bugs.webkit.org/show_bug.cgi?id=199425
|
||
|
||
broke some iOS arm64e tests (Requested by keith_miller on
|
||
#webkit).
|
||
|
||
Reverted changeset:
|
||
|
||
"PACCage should first cage leaving PAC bits intact then
|
||
authenticate"
|
||
https://bugs.webkit.org/show_bug.cgi?id=199372
|
||
https://trac.webkit.org/changeset/247041
|
||
|
||
2019-07-02 Keith Miller <keith_miller@apple.com>
|
||
|
||
PACCage should first cage leaving PAC bits intact then authenticate
|
||
https://bugs.webkit.org/show_bug.cgi?id=199372
|
||
|
||
Reviewed by Saam Barati.
|
||
|
||
* bmalloc/ProcessCheck.mm:
|
||
(bmalloc::shouldProcessUnconditionallyUseBmalloc):
|
||
|
||
2019-06-27 Beth Dakin <bdakin@apple.com>
|
||
|
||
Upstream use of MACCATALYST
|
||
https://bugs.webkit.org/show_bug.cgi?id=199245
|
||
rdar://problem/51687723
|
||
|
||
Reviewed by Tim Horton.
|
||
|
||
* Configurations/SDKVariant.xcconfig:
|
||
|
||
2019-06-24 Yusuke Suzuki <ysuzuki@apple.com>
|
||
|
||
[bmalloc] IsoHeap shared tier threshold is small
|
||
https://bugs.webkit.org/show_bug.cgi?id=199145
|
||
|
||
Reviewed by Saam Barati.
|
||
|
||
We accidentally picked 3 for the shared tier threshold. While this is OK because the important part of IsoHeap shared tier is putting
|
||
small # of shared objects before tiering up to the page-based allocation, increasing this threshold can potentially improve memory footprint.
|
||
This patch increases this threshold from 3 to 8. A/B test says that this offers stable 0.25% improvement in PLUM2.
|
||
|
||
* bmalloc/IsoHeapImpl.h:
|
||
|
||
2019-06-21 Yusuke Suzuki <ysuzuki@apple.com>
|
||
|
||
[bmalloc] IsoTLS Layout extension initializes one IsoTLSEntry twice
|
||
https://bugs.webkit.org/show_bug.cgi?id=199077
|
||
|
||
Reviewed by Saam Barati.
|
||
|
||
Found that IsoTLS::ensureEntries can construct the same IsoTLSEntry many times, it can leak memory because the construction clears previous fields including freelist.
|
||
|
||
1. We have oldLastEntry.
|
||
2. In that case, startEntry is oldLastEntry.
|
||
3. We find some targetEntry.
|
||
4. Finally, if startEntry exists, we newly construct [startEntry, targetEntry]
|
||
5. In the above sequence, oldLastEntry (== startEntry) is constructed again, while oldLastEntry is already constructed previously.
|
||
|
||
We fix this issue by changing the startEntry. We already have `RELEASE_BASSERT(!oldLastEntry || oldLastEntry->offset() < offset);`
|
||
assertion. This means that `oldLastEntry->m_next` must exist, otherwise the following loop would not find a `targetEntry`. And `layout.head()`
|
||
must return non nullptr at `IsoTLS::ensureEntries` because `IsoTLS::ensureEntries` requires that `IsoHeap<>` is initialized, and `IsoHeap<>`
|
||
must add at least one TLS entry to the IsoTLSLayout.
|
||
|
||
* bmalloc/IsoTLS.cpp:
|
||
(bmalloc::IsoTLS::ensureEntries):
|
||
|
||
2019-06-19 Yusuke Suzuki <ysuzuki@apple.com>
|
||
|
||
[bmalloc] IsoHeap's initialization is racy with IsoHeap::isInitialized
|
||
https://bugs.webkit.org/show_bug.cgi?id=199053
|
||
|
||
Reviewed by Saam Barati.
|
||
|
||
IsoHeap's initialization code is racy. Let's see the `isInitialized` and the initialization code.
|
||
|
||
isInitialized:
|
||
|
||
template<typename Type>
|
||
bool IsoHeap<Type>::isInitialized()
|
||
{
|
||
std::atomic<unsigned>* atomic =
|
||
reinterpret_cast<std::atomic<unsigned>*>(&m_allocatorOffsetPlusOne);
|
||
return !!atomic->load(std::memory_order_acquire);
|
||
}
|
||
|
||
initialization:
|
||
|
||
if (!handle.isInitialized()) {
|
||
std::lock_guard<Mutex> locker(handle.m_initializationLock);
|
||
if (!handle.isInitialized()) {
|
||
auto* heap = new IsoHeapImpl<typename api::IsoHeap<Type>::Config>();
|
||
std::atomic_thread_fence(std::memory_order_seq_cst);
|
||
handle.setAllocatorOffset(heap->allocatorOffset()); // <================= (1)
|
||
handle.setDeallocatorOffset(heap->deallocatorOffset()); // (2)
|
||
handle.m_impl = heap;
|
||
}
|
||
}
|
||
|
||
IsoHeap::isInitialized is loading m_allocatorOffsetPlusOne with acquire fence. On the other hand, the initialization
|
||
code configures m_allocatorOffsetPlusOne (1) before configuring m_deallocatorOffsetPlusOne (2). Let's consider the following
|
||
case.
|
||
|
||
1. Thread A is at (1)
|
||
2. Thread B calls handle.isInitialized(). Then B think that handle is already initialized while it lacks m_deallocatorOffsetPlusOne and m_impl pointer.
|
||
3. Thread B uses this handle, and does `std::max(handle.allocatorOffset(), handle.deallocatorOffset())`. But m_deallocatorOffsetPlusOne is not configured
|
||
yet. As a result, deallocatorOffset() returns 0xffffffff (b/c it calculates m_deallocatorOffsetPlusOne - 1, and m_deallocatorOffsetPlusOne is first
|
||
zero-initialized before IsoHeap initialization happens).
|
||
4. std::max returns 0xffffffff as an offset. Of course, this is wrong, and leading to the release assertion.
|
||
|
||
This patch fixes the above issue by,
|
||
|
||
1. Add IsoHeap::initialize() function instead of initializing it in IsoTLS
|
||
2. Change `isInitialized()` function to load m_impl pointer instead of m_allocatorOffsetPlusOne with acquire fence.
|
||
3. In initialize() function, we store m_heap with release fence at last.
|
||
|
||
* bmalloc/IsoHeap.h:
|
||
* bmalloc/IsoHeapInlines.h:
|
||
(bmalloc::api::IsoHeap<Type>::isInitialized):
|
||
(bmalloc::api::IsoHeap<Type>::initialize):
|
||
* bmalloc/IsoTLSInlines.h:
|
||
(bmalloc::IsoTLS::ensureHeap):
|
||
|
||
2019-06-14 Keith Miller <keith_miller@apple.com>
|
||
|
||
Restore PAC based cage.
|
||
https://bugs.webkit.org/show_bug.cgi?id=198872
|
||
|
||
Rubber-stamped by Saam Barati.
|
||
|
||
* bmalloc/Gigacage.h:
|
||
|
||
2019-06-12 Commit Queue <commit-queue@webkit.org>
|
||
|
||
Unreviewed, rolling out r246322.
|
||
https://bugs.webkit.org/show_bug.cgi?id=198796
|
||
|
||
"It's a huge page load regression on iOS" (Requested by
|
||
saamyjoon on #webkit).
|
||
|
||
Reverted changeset:
|
||
|
||
"Roll out PAC cage"
|
||
https://bugs.webkit.org/show_bug.cgi?id=198726
|
||
https://trac.webkit.org/changeset/246322
|
||
|
||
2019-06-11 Saam Barati <sbarati@apple.com>
|
||
|
||
Roll out PAC cage
|
||
https://bugs.webkit.org/show_bug.cgi?id=198726
|
||
|
||
Reviewed by Keith Miller.
|
||
|
||
* bmalloc/Gigacage.h:
|
||
(Gigacage::isEnabled):
|
||
(Gigacage::caged):
|
||
(Gigacage::cagedMayBeNull): Deleted.
|
||
|
||
2019-06-09 Commit Queue <commit-queue@webkit.org>
|
||
|
||
Unreviewed, rolling out r246150, r246160, and r246166.
|
||
https://bugs.webkit.org/show_bug.cgi?id=198698
|
||
|
||
Regresses page loading time on iOS 13 (Requested by keith_m__
|
||
on #webkit).
|
||
|
||
Reverted changesets:
|
||
|
||
"Reenable Gigacage on ARM64."
|
||
https://bugs.webkit.org/show_bug.cgi?id=198453
|
||
https://trac.webkit.org/changeset/246150
|
||
|
||
"Unrevied build fix for FTL without Gigacage."
|
||
https://trac.webkit.org/changeset/246160
|
||
|
||
"Fix typo in cageWithoutUntagging"
|
||
https://bugs.webkit.org/show_bug.cgi?id=198617
|
||
https://trac.webkit.org/changeset/246166
|
||
|
||
2019-06-06 Keith Miller <keith_miller@apple.com>
|
||
|
||
Reenable Gigacage on ARM64.
|
||
https://bugs.webkit.org/show_bug.cgi?id=198453
|
||
|
||
Reviewed by Michael Saboff.
|
||
|
||
* bmalloc/Gigacage.h:
|
||
|
||
2019-06-03 Commit Queue <commit-queue@webkit.org>
|
||
|
||
Unreviewed, rolling out r246022.
|
||
https://bugs.webkit.org/show_bug.cgi?id=198486
|
||
|
||
Causing Internal build failures and JSC test failures
|
||
(Requested by ShawnRoberts on #webkit).
|
||
|
||
Reverted changeset:
|
||
|
||
"Reenable Gigacage on ARM64."
|
||
https://bugs.webkit.org/show_bug.cgi?id=198453
|
||
https://trac.webkit.org/changeset/246022
|
||
|
||
2019-06-02 Keith Miller <keith_miller@apple.com>
|
||
|
||
Reenable Gigacage on ARM64.
|
||
https://bugs.webkit.org/show_bug.cgi?id=198453
|
||
|
||
Reviewed by Filip Pizlo.
|
||
|
||
* bmalloc/Gigacage.h:
|
||
|
||
2019-05-30 Don Olmstead <don.olmstead@sony.com>
|
||
|
||
[CMake] Add WEBKIT_FRAMEWORK_TARGET macro
|
||
https://bugs.webkit.org/show_bug.cgi?id=198396
|
||
|
||
Reviewed by Konstantin Tokarev.
|
||
|
||
Use WEBKIT_FRAMEWORK_TARGET.
|
||
|
||
* CMakeLists.txt:
|
||
|
||
2019-05-30 Keith Miller <keith_miller@apple.com>
|
||
|
||
IsoHeaps don't notice uncommitted VA becoming the first eligible.
|
||
https://bugs.webkit.org/show_bug.cgi?id=198301
|
||
|
||
Reviewed by Yusuke Suzuki.
|
||
|
||
IsoDirectory has a firstEligible member that is used as an
|
||
optimization to help find the first fit. However if the scavenger
|
||
decommitted a page before firstEligible then we wouldn't move
|
||
firstEligible. Thus, if no space is ever freed below firstEligible
|
||
we will never reused the decommitted memory (e.g. if the VA page
|
||
is decommitted). The fix is to make IsoDirectory::didDecommit move
|
||
the firstEligible page back if the decommitted page is smaller
|
||
than the current firstEligible. As such, this patch renames
|
||
firstEligible to firstEligibleOrDecommitted.
|
||
|
||
Also, this patch changes gigacageEnabledForProcess to check if the
|
||
process starts with Test rather than just test as TestWTF does.
|
||
|
||
Lastly, unbeknownst to me IsoHeaps are dependent on gigacage, so
|
||
by removing gigacage from arm64 I accidentally disabled
|
||
IsoHeaps...
|
||
|
||
* bmalloc.xcodeproj/project.pbxproj:
|
||
* bmalloc/IsoDirectory.h:
|
||
* bmalloc/IsoDirectoryInlines.h:
|
||
(bmalloc::passedNumPages>::takeFirstEligible):
|
||
(bmalloc::passedNumPages>::didBecome):
|
||
(bmalloc::passedNumPages>::didDecommit):
|
||
* bmalloc/IsoHeapImpl.h:
|
||
* bmalloc/IsoHeapImplInlines.h:
|
||
(bmalloc::IsoHeapImpl<Config>::takeFirstEligible):
|
||
(bmalloc::IsoHeapImpl<Config>::didBecomeEligibleOrDecommited):
|
||
(bmalloc::IsoHeapImpl<Config>::didCommit):
|
||
(bmalloc::IsoHeapImpl<Config>::didBecomeEligible): Deleted.
|
||
* bmalloc/IsoTLS.cpp:
|
||
(bmalloc::IsoTLS::determineMallocFallbackState):
|
||
* bmalloc/ProcessCheck.mm:
|
||
(bmalloc::gigacageEnabledForProcess):
|
||
|
||
2019-05-23 Don Olmstead <don.olmstead@sony.com>
|
||
|
||
[CMake] Use target oriented design for bmalloc
|
||
https://bugs.webkit.org/show_bug.cgi?id=198046
|
||
|
||
Reviewed by Konstantin Tokarev.
|
||
|
||
Switch to a target oriented dsign for bmalloc. Use target_include_directories directly
|
||
instead of include_directories.
|
||
|
||
List the headers for bmalloc and copy them using WEBKIT_COPY_FILES.
|
||
|
||
Add an intermediate target bmalloc_PostBuild which depends on bmalloc and the headers
|
||
being copied. Then alias that to WebKit::bmalloc.
|
||
|
||
* CMakeLists.txt:
|
||
* PlatformMac.cmake:
|
||
|
||
2019-05-16 Keith Miller <keith_miller@apple.com>
|
||
|
||
Wasm should cage the memory base pointers in structs
|
||
https://bugs.webkit.org/show_bug.cgi?id=197620
|
||
|
||
Reviewed by Saam Barati.
|
||
|
||
Fix signature to take Gigacage::Kind, which matches GIGACAGE_ENABLED build.
|
||
|
||
* bmalloc/Gigacage.h:
|
||
(Gigacage::isEnabled):
|
||
|
||
2019-05-08 Keith Miller <keith_miller@apple.com>
|
||
|
||
Remove Gigacage from arm64 and use PAC for arm64e instead
|
||
https://bugs.webkit.org/show_bug.cgi?id=197110
|
||
|
||
Reviewed by Saam Barati.
|
||
|
||
Stop using gigacage on arm64 and add a new cage function cagedMayBeNull that is the same as
|
||
cage but returns a nullptr if the incoming pointer is already null.
|
||
|
||
* bmalloc/Gigacage.h:
|
||
(Gigacage::cagedMayBeNull):
|
||
|
||
2019-04-29 Alex Christensen <achristensen@webkit.org>
|
||
|
||
<rdar://problem/50299396> Fix internal High Sierra build
|
||
https://bugs.webkit.org/show_bug.cgi?id=197388
|
||
|
||
* Configurations/Base.xcconfig:
|
||
|
||
2019-04-25 Yusuke Suzuki <ysuzuki@apple.com>
|
||
|
||
[bmalloc] Follow-up and fixing bug after r244481
|
||
https://bugs.webkit.org/show_bug.cgi?id=197294
|
||
|
||
Reviewed by Saam Barati.
|
||
|
||
This patch includes follow-up after r244481 and bug fixes which is introduced in the refactoring.
|
||
|
||
* bmalloc/IsoAllocator.h: Remove unused function.
|
||
* bmalloc/IsoAllocatorInlines.h:
|
||
(bmalloc::IsoAllocator<Config>::allocateSlow):
|
||
* bmalloc/IsoDeallocatorInlines.h:
|
||
(bmalloc::IsoDeallocator<Config>::deallocate):
|
||
* bmalloc/IsoHeapImpl.h: Rename m_usableBits to m_availableShared and add static_assert.
|
||
* bmalloc/IsoHeapImplInlines.h: Do not clear m_numberOfAllocationsFromSharedInOneCycle etc. in scavenge since IsoHeapImpl::scavenge
|
||
is not related to thread-local IsoAllocator's status.
|
||
(bmalloc::IsoHeapImpl<Config>::scavenge):
|
||
(bmalloc::IsoHeapImpl<Config>::forEachLiveObject):
|
||
(bmalloc::IsoHeapImpl<Config>::updateAllocationMode): Update m_allocationMode correctly.
|
||
(bmalloc::IsoHeapImpl<Config>::allocateFromShared):
|
||
* bmalloc/IsoSharedHeapInlines.h:
|
||
(bmalloc::computeObjectSizeForSharedCell):
|
||
(bmalloc::IsoSharedHeap::allocateNew):
|
||
(bmalloc::IsoSharedHeap::allocateSlow): Add computeObjectSizeForSharedCell.
|
||
* bmalloc/IsoSharedPage.h:
|
||
* bmalloc/IsoSharedPageInlines.h:
|
||
(bmalloc::IsoSharedPage::free): Pass `const std::lock_guard<Mutex>&` in its parameter.
|
||
|
||
2019-04-25 Alex Christensen <achristensen@webkit.org>
|
||
|
||
Start using C++17
|
||
https://bugs.webkit.org/show_bug.cgi?id=197131
|
||
|
||
Reviewed by Darin Adler.
|
||
|
||
* Configurations/Base.xcconfig:
|
||
|
||
2019-04-24 Yusuke Suzuki <ysuzuki@apple.com>
|
||
|
||
Unreviewed, fix typo in r244481
|
||
https://bugs.webkit.org/show_bug.cgi?id=196837
|
||
|
||
* bmalloc/IsoHeapImplInlines.h:
|
||
(bmalloc::IsoHeapImpl<Config>::allocateFromShared):
|
||
|
||
2019-04-21 Yusuke Suzuki <ysuzuki@apple.com>
|
||
|
||
[bmalloc] Use StaticPerProcess' mutex as bmalloc::Heap does with PerProcess
|
||
https://bugs.webkit.org/show_bug.cgi?id=197135
|
||
|
||
Reviewed by Darin Adler.
|
||
|
||
This patch leverages StaticPerProcess::mutex() for per process instance's lock in various classes,
|
||
as Heap does with PerProcess::mutex().
|
||
|
||
* bmalloc/AllIsoHeaps.cpp:
|
||
(bmalloc::AllIsoHeaps::add):
|
||
(bmalloc::AllIsoHeaps::head):
|
||
* bmalloc/AllIsoHeaps.h:
|
||
* bmalloc/CryptoRandom.cpp:
|
||
(bmalloc::ARC4RandomNumberGenerator::randomValues):
|
||
* bmalloc/DebugHeap.cpp:
|
||
(bmalloc::DebugHeap::memalignLarge):
|
||
(bmalloc::DebugHeap::freeLarge):
|
||
* bmalloc/DebugHeap.h:
|
||
* bmalloc/Scavenger.cpp:
|
||
(bmalloc::Scavenger::run):
|
||
(bmalloc::Scavenger::runSoon):
|
||
(bmalloc::Scavenger::scheduleIfUnderMemoryPressure):
|
||
(bmalloc::Scavenger::schedule):
|
||
(bmalloc::Scavenger::timeSinceLastFullScavenge):
|
||
(bmalloc::Scavenger::scavenge):
|
||
(bmalloc::Scavenger::threadRunLoop):
|
||
* bmalloc/Scavenger.h:
|
||
|
||
2019-04-19 Yusuke Suzuki <ysuzuki@apple.com>
|
||
|
||
[bmalloc] IsoHeap should have lower tier using shared IsoPage
|
||
https://bugs.webkit.org/show_bug.cgi?id=196837
|
||
|
||
Reviewed by Filip Pizlo.
|
||
|
||
IsoHeap had a scalability problem. Once one instance is allocated from IsoHeap, it immediately allocates 16KB page for this type.
|
||
But some types allocate only a few instances. It leads to memory wastage, and it also limits the scalability of IsoHeap since
|
||
we need to carefully select classes which will be confined in IsoHeap due to this characteristics. If we can remove this wastage,
|
||
we can apply IsoHeap more aggressively without causing memory regression, this is the goal of this patch.
|
||
|
||
In this patch, we introduce a slow tier to IsoHeap allocation. Initially, the allocator for a certain type allocates instances from
|
||
a shared page with the other allocators, and eventually, the allocator tiers up and gets dedicated pages if instances of the type
|
||
are allocated a lot. This "shared" tier is slow, but it is totally OK because we will tier up to the normal fast tier if allocation
|
||
frequently happens. Even the instance is allocated from pages shared with the other allocators, we still make the allocated memory
|
||
region dedicated to the specific type: once a memory region is allocated for a certain type from a shared page, this region continues
|
||
being used only for this type even after this memory is freed. To summarize the changes:
|
||
|
||
1. We introduce "shared" tier to IsoHeap allocation. Up to N (N = 8 for now, but we can pick any power-of-two numbers up to 32) allocations,
|
||
we continue using this tier. We allocate memory from shared pages so that we do not waste 16KB pages for types which only allocates a few instances.
|
||
|
||
2. We eventually tier up to the "fast" tier, and eventually tier down to the "shared" tier too. We measure the period between slow paths,
|
||
and switch the appropriate tier for the type. Currently, we use 1 seconds as heuristics. We also count # of allocations per cycle to
|
||
avoid pathological slow downs.
|
||
|
||
3. Shared page mechanism must keep the characteristics of IsoHeap. Once a memory region is allocated for a certain type, this memory region
|
||
must be dedicated to this type. We keep track the allocated memory regions from shared pages in IsoHeapImpl, and ensure that we never
|
||
reuse a memory region for a different type.
|
||
|
||
This patch improves PLUM2 by 1.4% (128.4MB v.s. 126.62MB), and early Speedometer2 results are performance-neutral.
|
||
|
||
* CMakeLists.txt:
|
||
* bmalloc.xcodeproj/project.pbxproj:
|
||
* bmalloc/Algorithm.h:
|
||
(bmalloc::roundUpToMultipleOfImpl):
|
||
(bmalloc::roundUpToMultipleOf):
|
||
* bmalloc/BCompiler.h:
|
||
* bmalloc/BExport.h:
|
||
* bmalloc/FreeList.h:
|
||
* bmalloc/IsoAllocator.h:
|
||
* bmalloc/IsoAllocatorInlines.h:
|
||
(bmalloc::IsoAllocator<Config>::allocateSlow):
|
||
* bmalloc/IsoDeallocator.h:
|
||
* bmalloc/IsoDeallocatorInlines.h:
|
||
(bmalloc::IsoDeallocator<Config>::deallocate):
|
||
* bmalloc/IsoHeapImpl.h:
|
||
* bmalloc/IsoHeapImplInlines.h:
|
||
(bmalloc::IsoHeapImpl<Config>::scavenge):
|
||
(bmalloc::IsoHeapImpl<Config>::forEachLiveObject):
|
||
(bmalloc::IsoHeapImpl<Config>::updateAllocationMode):
|
||
(bmalloc::IsoHeapImpl<Config>::allocateFromShared):
|
||
* bmalloc/IsoPage.h:
|
||
(bmalloc::IsoPageBase::IsoPageBase):
|
||
(bmalloc::IsoPageBase::isShared const):
|
||
* bmalloc/IsoPageInlines.h:
|
||
(bmalloc::IsoPage<Config>::IsoPage):
|
||
(bmalloc::IsoPageBase::pageFor):
|
||
(bmalloc::IsoPage<Config>::pageFor):
|
||
(bmalloc::IsoPage<Config>::free):
|
||
* bmalloc/IsoSharedConfig.h: Copied from Source/bmalloc/bmalloc/BExport.h.
|
||
* bmalloc/IsoSharedHeap.cpp: Copied from Source/bmalloc/bmalloc/BExport.h.
|
||
* bmalloc/IsoSharedHeap.h: Copied from Source/bmalloc/bmalloc/IsoAllocator.h.
|
||
(bmalloc::VariadicBumpAllocator::VariadicBumpAllocator):
|
||
(bmalloc::IsoSharedHeap::IsoSharedHeap):
|
||
* bmalloc/IsoSharedHeapInlines.h: Added.
|
||
(bmalloc::VariadicBumpAllocator::allocate):
|
||
(bmalloc::IsoSharedHeap::allocateNew):
|
||
(bmalloc::IsoSharedHeap::allocateSlow):
|
||
* bmalloc/IsoSharedPage.cpp: Copied from Source/bmalloc/bmalloc/BExport.h.
|
||
(bmalloc::IsoSharedPage::tryCreate):
|
||
* bmalloc/IsoSharedPage.h: Copied from Source/bmalloc/bmalloc/IsoDeallocator.h.
|
||
(bmalloc::IsoSharedPage::IsoSharedPage):
|
||
(bmalloc::indexSlotFor):
|
||
* bmalloc/IsoSharedPageInlines.h: Added.
|
||
(bmalloc::IsoSharedPage::free):
|
||
(bmalloc::IsoSharedPage::startAllocating):
|
||
(bmalloc::IsoSharedPage::stopAllocating):
|
||
* bmalloc/IsoTLS.h:
|
||
* bmalloc/IsoTLSInlines.h:
|
||
(bmalloc::IsoTLS::deallocateImpl):
|
||
(bmalloc::IsoTLS::deallocateFast):
|
||
(bmalloc::IsoTLS::deallocateSlow):
|
||
* bmalloc/StdLibExtras.h:
|
||
(bmalloc::bitwise_cast):
|
||
* test/testbmalloc.cpp:
|
||
(testIsoMallocAndFreeFast):
|
||
(run):
|
||
|
||
2019-04-18 Yusuke Suzuki <ysuzuki@apple.com>
|
||
|
||
Unreviewed, fix build failure
|
||
https://bugs.webkit.org/show_bug.cgi?id=195938
|
||
|
||
Including <array>.
|
||
|
||
* bmalloc/AvailableMemory.cpp:
|
||
|
||
2019-04-15 Yoshiaki Jitsukawa <yoshiaki.jitsukawa@sony.com>
|
||
|
||
Unreviewed. Build fix after r244244.
|
||
|
||
* bmalloc/AvailableMemory.cpp:
|
||
|
||
2019-04-13 Zan Dobersek <zdobersek@igalia.com>
|
||
|
||
[bmalloc][Linux] Add support for memory status calculation
|
||
https://bugs.webkit.org/show_bug.cgi?id=195938
|
||
|
||
Reviewed by Carlos Garcia Campos.
|
||
|
||
Memory status and under-memory-pressure capabilities in bmalloc can be
|
||
implemented on Linux by reading and parsing the statm file under the
|
||
proc filesystem.
|
||
|
||
We retrieve the resident set size from the statm file and multiply it
|
||
with the page size. This gives an upper-bound estimate of the memory
|
||
that's being consumed by the process.
|
||
|
||
The statm-based estimate seems preferable to other alternatives. One
|
||
such alternative would be reading and parsing more-detailed smaps file,
|
||
also exposed under the proc filesystem. This is at the moment being done
|
||
in WTF's MemoryFootprint implementation for Linux systems, but on Linux
|
||
ports this operation is being throttled to only execute once per second
|
||
because of the big computing expense required to read and parse out the
|
||
data. A future MemoryFootprint implementation could simply retrieve the
|
||
memory footprint value from bmalloc.
|
||
|
||
Another alternative is the Linux taskstats interface. This one would
|
||
require utilizing a netlink socket to retrieve the necessary statistics,
|
||
but it requires the process to have elevated privileges, which is a
|
||
blocker.
|
||
|
||
* bmalloc/AvailableMemory.cpp:
|
||
(bmalloc::LinuxMemory::singleton):
|
||
(bmalloc::LinuxMemory::footprint const):
|
||
(bmalloc::computeAvailableMemory):
|
||
(bmalloc::memoryStatus):
|
||
* bmalloc/AvailableMemory.h:
|
||
(bmalloc::isUnderMemoryPressure):
|
||
* bmalloc/bmalloc.h:
|
||
|
||
2019-04-04 Yusuke Suzuki <ysuzuki@apple.com>
|
||
|
||
[WebCore] Put most of derived classes of ScriptWrappable into IsoHeap
|
||
https://bugs.webkit.org/show_bug.cgi?id=196475
|
||
|
||
Reviewed by Saam Barati.
|
||
|
||
Add MAKE_BISO_MALLOCED_IMPL_TEMPLATE, which can be used for explicit specialization for template classes.
|
||
|
||
* bmalloc/IsoHeap.h:
|
||
* bmalloc/IsoHeapInlines.h:
|
||
|
||
2019-03-22 Keith Rollin <krollin@apple.com>
|
||
|
||
Enable ThinLTO support in Production builds
|
||
https://bugs.webkit.org/show_bug.cgi?id=190758
|
||
<rdar://problem/45413233>
|
||
|
||
Reviewed by Daniel Bates.
|
||
|
||
Enable building with Thin LTO in Production when using Xcode 10.2 or
|
||
later. This change results in a 1.45% progression in PLT5. Full
|
||
Production build times increase about 2-3%. Incremental build times
|
||
are more severely affected, and so LTO is not enabled for local
|
||
engineering builds.
|
||
|
||
LTO is enabled only on macOS for now, until rdar://problem/49013399,
|
||
which affects ARM builds, is fixed.
|
||
|
||
To change the LTO setting when building locally:
|
||
|
||
- If building with `make`, specify WK_LTO_MODE={none,thin,full} on the
|
||
command line.
|
||
- If building with `build-webkit`, specify --lto-mode={none,thin,full}
|
||
on the command line.
|
||
- If building with `build-root`, specify --lto={none,thin,full} on the
|
||
command line.
|
||
- If building with Xcode, create a LocalOverrides.xcconfig file at the
|
||
top level of your repository directory (if needed) and define
|
||
WK_LTO_MODE to full, thin, or none.
|
||
|
||
* Configurations/Base.xcconfig:
|
||
|
||
2019-03-21 Michael Saboff <msaboff@apple.com>
|
||
|
||
[BMalloc] No need to delay deallocating chunks based on recent use
|
||
https://bugs.webkit.org/show_bug.cgi?id=196121
|
||
|
||
Reviewed by Mark Lam.
|
||
|
||
The "used since last scavenge" logic is not needed for small chunks since their memory isn't decommitted directly.
|
||
We can deallocate small chunks immediately as that adds them to the LargeRange free list. That free list employs the
|
||
"used since last scavenge" logic before the scavenger decommits the backing memory.
|
||
|
||
* bmalloc/Chunk.h:
|
||
(bmalloc::Chunk::usedSinceLastScavenge): Deleted.
|
||
(bmalloc::Chunk::clearUsedSinceLastScavenge): Deleted.
|
||
(bmalloc::Chunk::setUsedSinceLastScavenge): Deleted.
|
||
* bmalloc/Heap.cpp:
|
||
(bmalloc::Heap::scavenge):
|
||
(bmalloc::Heap::allocateSmallPage):
|
||
|
||
2019-03-21 Brady Eidson <beidson@apple.com>
|
||
|
||
Certain WebProcesses should opt-out of the freezer.
|
||
<rdar://problem/42846139> and https://bugs.webkit.org/show_bug.cgi?id=196062
|
||
|
||
Reviewed by Andy Estes.
|
||
|
||
* bmalloc.xcodeproj/project.pbxproj:
|
||
* bmalloc/darwin/MemoryStatusSPI.h:
|
||
|
||
2019-03-19 Michael Catanzaro <mcatanzaro@igalia.com>
|
||
|
||
Unreviewed, fix -Wformat warning
|
||
https://bugs.webkit.org/show_bug.cgi?id=195895
|
||
<rdar://problem/48517629>
|
||
|
||
* bmalloc/Scavenger.cpp:
|
||
(bmalloc::Scavenger::threadRunLoop):
|
||
|
||
2019-03-18 Michael Saboff <msaboff@apple.com>
|
||
|
||
[BMalloc] Scavenger should react to recent memory activity
|
||
https://bugs.webkit.org/show_bug.cgi?id=195895
|
||
|
||
Reviewed by Geoffrey Garen.
|
||
|
||
This change adds a recently used bit to objects that are scavenged. When an object is allocated, that bit is set.
|
||
When we scavenge, if the bit is set, we clear it. If the bit was already clear, we decommit the object. The timing
|
||
to scavenging has been changed as well. We perform our first scavne almost immediately after bmalloc is initialized
|
||
(10ms later). Subsequent scavenging is done as a multiple of the time it took to scavenge. We bound this computed
|
||
time between a minimum and maximum. Through empirical testing, the multiplier, minimum and maximum are
|
||
150x, 100ms and 10,000ms respectively. For mini-mode, when the JIT is disabled, we use much more aggressive values of
|
||
50x, 25ms and 500ms.
|
||
|
||
Eliminated partial scavenging since this change allows for any scavenge to be partial or full based on recent use of
|
||
the objects on the various free lists.
|
||
|
||
* bmalloc/Chunk.h:
|
||
(bmalloc::Chunk::usedSinceLastScavenge):
|
||
(bmalloc::Chunk::clearUsedSinceLastScavenge):
|
||
(bmalloc::Chunk::setUsedSinceLastScavenge):
|
||
* bmalloc/Heap.cpp:
|
||
(bmalloc::Heap::scavenge):
|
||
(bmalloc::Heap::allocateSmallChunk):
|
||
(bmalloc::Heap::allocateSmallPage):
|
||
(bmalloc::Heap::splitAndAllocate):
|
||
(bmalloc::Heap::tryAllocateLarge):
|
||
(bmalloc::Heap::scavengeToHighWatermark): Deleted.
|
||
* bmalloc/Heap.h:
|
||
* bmalloc/IsoDirectory.h:
|
||
* bmalloc/IsoDirectoryInlines.h:
|
||
(bmalloc::passedNumPages>::takeFirstEligible):
|
||
(bmalloc::passedNumPages>::scavenge):
|
||
(bmalloc::passedNumPages>::scavengeToHighWatermark): Deleted.
|
||
* bmalloc/IsoHeapImpl.h:
|
||
* bmalloc/IsoHeapImplInlines.h:
|
||
(bmalloc::IsoHeapImpl<Config>::scavengeToHighWatermark): Deleted.
|
||
* bmalloc/LargeRange.h:
|
||
(bmalloc::LargeRange::LargeRange):
|
||
(bmalloc::LargeRange::usedSinceLastScavenge):
|
||
(bmalloc::LargeRange::clearUsedSinceLastScavenge):
|
||
(bmalloc::LargeRange::setUsedSinceLastScavenge):
|
||
(): Deleted.
|
||
* bmalloc/Scavenger.cpp:
|
||
(bmalloc::Scavenger::Scavenger):
|
||
(bmalloc::Scavenger::threadRunLoop):
|
||
(bmalloc::Scavenger::timeSinceLastPartialScavenge): Deleted.
|
||
(bmalloc::Scavenger::partialScavenge): Deleted.
|
||
* bmalloc/Scavenger.h:
|
||
* bmalloc/SmallPage.h:
|
||
(bmalloc::SmallPage::usedSinceLastScavenge):
|
||
(bmalloc::SmallPage::clearUsedSinceLastScavenge):
|
||
(bmalloc::SmallPage::setUsedSinceLastScavenge):
|
||
|
||
2019-03-14 Yusuke Suzuki <ysuzuki@apple.com>
|
||
|
||
[bmalloc] Add StaticPerProcess for known types to save pages
|
||
https://bugs.webkit.org/show_bug.cgi?id=195691
|
||
|
||
Reviewed by Mark Lam.
|
||
|
||
As initial memory footprint of VM + JSGlobalObject becomes 488KB dirty size in fast malloc memory (w/ JSC_useJIT=0 and Malloc=1), pages for PerProcess is costly.
|
||
For example, under Malloc=1 mode, we still need to allocate PerProcess<DebugHeap> and PerProcess<Environment>. And sizeof(Environment) is only 1 (bool flag), and
|
||
sizeof(DebugHeap) is 120. But we are allocating 1 pages for them. Since page size in iOS is 16KB, this 121B consumes 16KB dirty memory, and it is not negligible
|
||
size if we keep in mind that the current fast malloc heap size is 488KB. Putting them into the __DATA section, close to the other mutable data, we can avoid allocating
|
||
this page.
|
||
|
||
This patch revives the SafePerProcess concept in r228107. We add "StaticPerProcess<T>", which allocates underlying storage statically in the __DATA section instead of
|
||
allocating it at runtime. And we use this StaticPerProcess<T> for types where (1) T is known a priori, and (2) sizeof(T) is not huge.
|
||
|
||
* bmalloc.xcodeproj/project.pbxproj:
|
||
* bmalloc/AllIsoHeaps.cpp:
|
||
* bmalloc/AllIsoHeaps.h:
|
||
* bmalloc/Allocator.cpp:
|
||
(bmalloc::Allocator::Allocator):
|
||
* bmalloc/Cache.cpp:
|
||
(bmalloc::Cache::Cache):
|
||
* bmalloc/CryptoRandom.cpp:
|
||
(bmalloc::cryptoRandom):
|
||
* bmalloc/Deallocator.cpp:
|
||
(bmalloc::Deallocator::Deallocator):
|
||
* bmalloc/DebugHeap.cpp:
|
||
* bmalloc/DebugHeap.h:
|
||
(bmalloc::DebugHeap::tryGet):
|
||
* bmalloc/Environment.cpp:
|
||
* bmalloc/Environment.h:
|
||
* bmalloc/Gigacage.cpp:
|
||
(Gigacage::Callback::Callback):
|
||
(Gigacage::Callback::function):
|
||
(bmalloc::PrimitiveDisableCallbacks::PrimitiveDisableCallbacks):
|
||
(Gigacage::disablePrimitiveGigacage):
|
||
(Gigacage::addPrimitiveDisableCallback):
|
||
(Gigacage::removePrimitiveDisableCallback):
|
||
(Gigacage::shouldBeEnabled):
|
||
(Gigacage::bmalloc::Callback::Callback): Deleted.
|
||
(Gigacage::bmalloc::Callback::function): Deleted.
|
||
(Gigacage::bmalloc::PrimitiveDisableCallbacks::PrimitiveDisableCallbacks): Deleted.
|
||
* bmalloc/Heap.cpp:
|
||
(bmalloc::Heap::Heap):
|
||
(bmalloc::Heap::tryAllocateLarge):
|
||
* bmalloc/IsoDirectoryInlines.h:
|
||
(bmalloc::passedNumPages>::takeFirstEligible):
|
||
(bmalloc::passedNumPages>::didBecome):
|
||
* bmalloc/IsoHeapImpl.cpp:
|
||
(bmalloc::IsoHeapImplBase::addToAllIsoHeaps):
|
||
* bmalloc/IsoPage.cpp:
|
||
(bmalloc::IsoPageBase::allocatePageMemory):
|
||
* bmalloc/IsoTLS.cpp:
|
||
(bmalloc::IsoTLS::IsoTLS):
|
||
(bmalloc::IsoTLS::ensureEntries):
|
||
(bmalloc::IsoTLS::forEachEntry):
|
||
* bmalloc/IsoTLSEntry.cpp:
|
||
(bmalloc::IsoTLSEntry::IsoTLSEntry):
|
||
* bmalloc/IsoTLSInlines.h:
|
||
(bmalloc::IsoTLS::allocateSlow):
|
||
(bmalloc::IsoTLS::deallocateSlow):
|
||
* bmalloc/IsoTLSLayout.cpp:
|
||
* bmalloc/IsoTLSLayout.h:
|
||
* bmalloc/Scavenger.cpp:
|
||
(bmalloc::Scavenger::Scavenger):
|
||
(bmalloc::dumpStats):
|
||
(bmalloc::Scavenger::scavenge):
|
||
(bmalloc::Scavenger::partialScavenge):
|
||
(bmalloc::Scavenger::freeableMemory):
|
||
(bmalloc::Scavenger::footprint):
|
||
* bmalloc/Scavenger.h:
|
||
* bmalloc/StaticPerProcess.h: Added.
|
||
* bmalloc/VMHeap.cpp:
|
||
* bmalloc/VMHeap.h:
|
||
* bmalloc/Zone.h:
|
||
* bmalloc/bmalloc.cpp:
|
||
(bmalloc::api::scavenge):
|
||
(bmalloc::api::isEnabled):
|
||
(bmalloc::api::setScavengerThreadQOSClass):
|
||
(bmalloc::api::enableMiniMode):
|
||
* test/testbmalloc.cpp:
|
||
(assertEmptyPointerSet):
|
||
(assertHasObjects):
|
||
(assertHasOnlyObjects):
|
||
(assertClean):
|
||
|
||
2019-03-13 Yoshiaki Jitsukawa <yoshiaki.jitsukawa@sony.com>
|
||
|
||
[bmalloc] Use MADV_FREE on FreeBSD
|
||
https://bugs.webkit.org/show_bug.cgi?id=195665
|
||
|
||
Reviewed by Geoffrey Garen.
|
||
|
||
* bmalloc/BPlatform.h:
|
||
|
||
Introduce BOS_FREEBSD, which is equivalent to WTF_OS_FREEBSD
|
||
|
||
* bmalloc/VMAllocate.h:
|
||
(bmalloc::vmDeallocatePhysicalPages):
|
||
|
||
Use MADV_FREE instead of MADV_DONTNEED if BOS(FREEBSD), since on FreeBSD,
|
||
unlike on Linux, MADV_DONTNEED doesn't let the OS discard the contents of
|
||
the pages.
|
||
|
||
2019-03-13 Sam Weinig <sam@webkit.org>
|
||
|
||
Fix testbmalloc build
|
||
https://bugs.webkit.org/show_bug.cgi?id=195660
|
||
|
||
Reviewed by Geoffrey Garen.
|
||
|
||
* bmalloc.xcodeproj/project.pbxproj:
|
||
Link Foundation in when building testbmalloc. Since bmalloc requires Foundation, and is a static
|
||
library, all clients of bmalloc are required to link it themselves.
|
||
|
||
* bmalloc/IsoPageInlines.h:
|
||
* bmalloc/StdLibExtras.h: Added.
|
||
(bmalloc::bitwise_cast):
|
||
Add bitwise_cast implementation, and use it in IsoPageInlines.h. It is a layering violation
|
||
to expect the one from WTF to be available, as seems to have been the case.
|
||
|
||
2019-03-12 Robin Morisset <rmorisset@apple.com>
|
||
|
||
A lot more classes have padding that can be reduced by reordering their fields
|
||
https://bugs.webkit.org/show_bug.cgi?id=195579
|
||
|
||
Reviewed by Mark Lam.
|
||
|
||
* bmalloc/Heap.h:
|
||
* bmalloc/Scavenger.h:
|
||
|
||
2019-03-05 Yusuke Suzuki <ysuzuki@apple.com>
|
||
|
||
[bmalloc] Bmalloc DebugHeap should have dump and scavenge features
|
||
https://bugs.webkit.org/show_bug.cgi?id=195305
|
||
|
||
Reviewed by Saam Barati.
|
||
|
||
As the same to bmalloc, bmalloc::DebugHeap should have scavenge feature to make it scavengable if we want.
|
||
We also add DebugHeap::dump feature which dumps system malloc data in the WebKit Malloc zone.
|
||
|
||
* bmalloc/DebugHeap.cpp:
|
||
(bmalloc::DebugHeap::scavenge):
|
||
(bmalloc::DebugHeap::dump):
|
||
* bmalloc/DebugHeap.h:
|
||
* bmalloc/bmalloc.cpp:
|
||
(bmalloc::api::scavenge):
|
||
|
||
2019-02-23 Keith Miller <keith_miller@apple.com>
|
||
|
||
Add new mac target numbers
|
||
https://bugs.webkit.org/show_bug.cgi?id=194955
|
||
|
||
Reviewed by Tim Horton.
|
||
|
||
* Configurations/Base.xcconfig:
|
||
* Configurations/DebugRelease.xcconfig:
|
||
|
||
2019-02-19 Yusuke Suzuki <ysuzuki@apple.com>
|
||
|
||
[bmalloc] bmalloc::Heap is allocated even though we use system malloc mode
|
||
https://bugs.webkit.org/show_bug.cgi?id=194836
|
||
|
||
Reviewed by Mark Lam.
|
||
|
||
Previously, bmalloc::Heap holds DebugHeap, and delegates allocation and deallocation to debug heap.
|
||
However, bmalloc::Heap is large. We would like to avoid initialization of bmalloc::Heap under the
|
||
system malloc mode.
|
||
|
||
This patch extracts out DebugHeap from bmalloc::Heap, and logically puts this in a boundary of
|
||
bmalloc::api. bmalloc::api delegates allocation and deallocation to DebugHeap if DebugHeap is enabled.
|
||
Otherwise, using bmalloc's usual mechanism. The challenge is that we would like to keep bmalloc fast
|
||
path fast.
|
||
|
||
1. For IsoHeaps, we use the similar techniques done in Cache. If the debug mode is enabled, we always go
|
||
to the slow path of the IsoHeap allocation, and keep IsoTLS::get() returning nullptr. In the slow path,
|
||
we just fallback to the usual bmalloc::api::tryMalloc implementation. This is efficient because bmalloc
|
||
continues using the fast path.
|
||
|
||
2. For the other APIs, like freeLargeVirtual, we just put DebugHeap check because this API itself takes fair
|
||
amount of time. Then debug heap check does not matter.
|
||
|
||
* bmalloc/Allocator.cpp:
|
||
(bmalloc::Allocator::reallocateImpl):
|
||
* bmalloc/Cache.cpp:
|
||
(bmalloc::Cache::tryAllocateSlowCaseNullCache):
|
||
(bmalloc::Cache::allocateSlowCaseNullCache):
|
||
(bmalloc::Cache::deallocateSlowCaseNullCache):
|
||
(bmalloc::Cache::tryReallocateSlowCaseNullCache):
|
||
(bmalloc::Cache::reallocateSlowCaseNullCache):
|
||
(): Deleted.
|
||
(bmalloc::debugHeap): Deleted.
|
||
* bmalloc/DebugHeap.cpp:
|
||
* bmalloc/DebugHeap.h:
|
||
(bmalloc::DebugHeap::tryGet):
|
||
* bmalloc/Heap.cpp:
|
||
(bmalloc::Heap::Heap):
|
||
(bmalloc::Heap::footprint):
|
||
(bmalloc::Heap::tryAllocateLarge):
|
||
(bmalloc::Heap::deallocateLarge):
|
||
* bmalloc/Heap.h:
|
||
(bmalloc::Heap::debugHeap): Deleted.
|
||
* bmalloc/IsoTLS.cpp:
|
||
(bmalloc::IsoTLS::IsoTLS):
|
||
(bmalloc::IsoTLS::isUsingDebugHeap): Deleted.
|
||
(bmalloc::IsoTLS::debugMalloc): Deleted.
|
||
(bmalloc::IsoTLS::debugFree): Deleted.
|
||
* bmalloc/IsoTLS.h:
|
||
* bmalloc/IsoTLSInlines.h:
|
||
(bmalloc::IsoTLS::allocateSlow):
|
||
(bmalloc::IsoTLS::deallocateSlow):
|
||
* bmalloc/ObjectType.cpp:
|
||
(bmalloc::objectType):
|
||
* bmalloc/ObjectType.h:
|
||
* bmalloc/Scavenger.cpp:
|
||
(bmalloc::Scavenger::Scavenger):
|
||
* bmalloc/bmalloc.cpp:
|
||
(bmalloc::api::tryLargeZeroedMemalignVirtual):
|
||
(bmalloc::api::freeLargeVirtual):
|
||
(bmalloc::api::scavenge):
|
||
(bmalloc::api::isEnabled):
|
||
(bmalloc::api::setScavengerThreadQOSClass):
|
||
(bmalloc::api::commitAlignedPhysical):
|
||
(bmalloc::api::decommitAlignedPhysical):
|
||
(bmalloc::api::enableMiniMode):
|
||
|
||
2019-02-20 Andy Estes <aestes@apple.com>
|
||
|
||
[Xcode] Add SDKVariant.xcconfig to various Xcode projects
|
||
https://bugs.webkit.org/show_bug.cgi?id=194869
|
||
|
||
Rubber-stamped by Jer Noble.
|
||
|
||
* bmalloc.xcodeproj/project.pbxproj:
|
||
|
||
2019-02-20 Yusuke Suzuki <ysuzuki@apple.com>
|
||
|
||
[bmalloc] DebugHeap::malloc does not have "try" version.
|
||
https://bugs.webkit.org/show_bug.cgi?id=194837
|
||
|
||
Reviewed by Mark Lam.
|
||
|
||
Since DebugHeap::malloc does not have "try" version, our tryAllocate implementation does not work well with DebugHeap.
|
||
This patch adds crashOnFailure flag to DebugHeap::malloc.
|
||
|
||
* bmalloc/Cache.cpp:
|
||
(bmalloc::Cache::tryAllocateSlowCaseNullCache):
|
||
(bmalloc::Cache::allocateSlowCaseNullCache):
|
||
* bmalloc/DebugHeap.cpp:
|
||
(bmalloc::DebugHeap::malloc):
|
||
* bmalloc/DebugHeap.h:
|
||
* bmalloc/IsoTLS.cpp:
|
||
(bmalloc::IsoTLS::debugMalloc):
|
||
|
||
2019-02-20 Yusuke Suzuki <ysuzuki@apple.com>
|
||
|
||
[bmalloc] bmalloc::Cache should not be instantiated if we are using system malloc
|
||
https://bugs.webkit.org/show_bug.cgi?id=194811
|
||
|
||
Reviewed by Mark Lam.
|
||
|
||
bmalloc::Cache is very large. It is 13KB. Since it exists per HeapKind, it takes 40KB.
|
||
But this is meaningless if we are under the system malloc mode by using "Malloc=1". We
|
||
found that it continues using so much dirty memory region even under the system malloc mode.
|
||
This patch avoids instantiation of bmalloc::Cache under the system malloc mode.
|
||
|
||
* bmalloc/Allocator.cpp:
|
||
(bmalloc::Allocator::Allocator):
|
||
(bmalloc::Allocator::tryAllocate):
|
||
(bmalloc::Allocator::allocateImpl):
|
||
(bmalloc::Allocator::reallocateImpl):
|
||
(bmalloc::Allocator::allocateSlowCase):
|
||
Allocator is a per Cache object. So we no longer need to keep m_debugHeap. If debug heap is enabled,
|
||
Allocator is never created.
|
||
|
||
* bmalloc/Allocator.h:
|
||
* bmalloc/Cache.cpp:
|
||
(bmalloc::debugHeap):
|
||
(bmalloc::Cache::Cache):
|
||
(bmalloc::Cache::tryAllocateSlowCaseNullCache):
|
||
(bmalloc::Cache::allocateSlowCaseNullCache):
|
||
(bmalloc::Cache::deallocateSlowCaseNullCache):
|
||
(bmalloc::Cache::tryReallocateSlowCaseNullCache):
|
||
(bmalloc::Cache::reallocateSlowCaseNullCache):
|
||
* bmalloc/Cache.h:
|
||
(bmalloc::Cache::tryAllocate):
|
||
(bmalloc::Cache::tryReallocate):
|
||
If the debug heap mode is enabled, we keep Cache::getFast() returning nullptr. And in the slow path case, we use debugHeap.
|
||
This makes bmalloc fast path fast, while we avoid Cache instantiation.
|
||
|
||
* bmalloc/Deallocator.cpp:
|
||
(bmalloc::Deallocator::Deallocator):
|
||
(bmalloc::Deallocator::scavenge):
|
||
(bmalloc::Deallocator::deallocateSlowCase):
|
||
* bmalloc/Deallocator.h:
|
||
Ditto for Deallocator.
|
||
|
||
* bmalloc/bmalloc.cpp:
|
||
(bmalloc::api::isEnabled):
|
||
We used `getFastCase()` for Heap. But it is basically wrong since we do not have any guarantee that someone already initializes
|
||
Heap when this is called. Previously, luckily, Cache is initialized, and Cache initialized Heap. But Cache initialization is removed
|
||
for system malloc mode and now PerProcess<PerHeapKind<Heap>>::getFastCase() returns nullptr at an early phase. This patch just uses
|
||
Environment::isDebugHeapEnabled() instead.
|
||
|
||
2019-02-20 Commit Queue <commit-queue@webkit.org>
|
||
|
||
Unreviewed, rolling out r241789.
|
||
https://bugs.webkit.org/show_bug.cgi?id=194856
|
||
|
||
GuardMalloc crashes (Requested by yusukesuzuki on #webkit).
|
||
|
||
Reverted changeset:
|
||
|
||
"[bmalloc] bmalloc::Cache should not be instantiated if we are
|
||
using system malloc"
|
||
https://bugs.webkit.org/show_bug.cgi?id=194811
|
||
https://trac.webkit.org/changeset/241789
|
||
|
||
2019-02-19 Yusuke Suzuki <ysuzuki@apple.com>
|
||
|
||
[bmalloc] bmalloc::Cache should not be instantiated if we are using system malloc
|
||
https://bugs.webkit.org/show_bug.cgi?id=194811
|
||
|
||
Reviewed by Mark Lam.
|
||
|
||
bmalloc::Cache is very large. It is 13KB. Since it exists per HeapKind, it takes 40KB.
|
||
But this is meaningless if we are under the system malloc mode by using "Malloc=1". We
|
||
found that it continues using so much dirty memory region even under the system malloc mode.
|
||
This patch avoids instantiation of bmalloc::Cache under the system malloc mode.
|
||
|
||
* bmalloc/Allocator.cpp:
|
||
(bmalloc::Allocator::Allocator):
|
||
(bmalloc::Allocator::tryAllocate):
|
||
(bmalloc::Allocator::allocateImpl):
|
||
(bmalloc::Allocator::reallocateImpl):
|
||
(bmalloc::Allocator::allocateSlowCase):
|
||
Allocator is a per Cache object. So we no longer need to keep m_debugHeap. If debug heap is enabled,
|
||
Allocator is never created.
|
||
|
||
* bmalloc/Allocator.h:
|
||
* bmalloc/Cache.cpp:
|
||
(bmalloc::debugHeap):
|
||
(bmalloc::Cache::Cache):
|
||
(bmalloc::Cache::tryAllocateSlowCaseNullCache):
|
||
(bmalloc::Cache::allocateSlowCaseNullCache):
|
||
(bmalloc::Cache::deallocateSlowCaseNullCache):
|
||
(bmalloc::Cache::tryReallocateSlowCaseNullCache):
|
||
(bmalloc::Cache::reallocateSlowCaseNullCache):
|
||
* bmalloc/Cache.h:
|
||
(bmalloc::Cache::tryAllocate):
|
||
(bmalloc::Cache::tryReallocate):
|
||
If the debug heap mode is enabled, we keep Cache::getFast() returning nullptr. And in the slow path case, we use debugHeap.
|
||
This makes bmalloc fast path fast, while we avoid Cache instantiation.
|
||
|
||
* bmalloc/Deallocator.cpp:
|
||
(bmalloc::Deallocator::Deallocator):
|
||
(bmalloc::Deallocator::scavenge):
|
||
(bmalloc::Deallocator::deallocateSlowCase):
|
||
* bmalloc/Deallocator.h:
|
||
Ditto for Deallocator.
|
||
|
||
2019-02-15 Yusuke Suzuki <ysuzuki@apple.com>
|
||
|
||
[bmalloc] NSBundle-based application name check should be executed after debug-heap environment variable check
|
||
https://bugs.webkit.org/show_bug.cgi?id=194694
|
||
|
||
Reviewed by Mark Lam.
|
||
|
||
Interestingly, NSBundle allocates fair amount of memory and keeps it for a process-long time. For example, it
|
||
allocates global NSConcreteHashTable, which takes 2.5KB. This patch changes the order of gigacage-check, we
|
||
first check "Malloc=1" status, and then check the process name through NSBundle. This allows us to remove NSBundle
|
||
related allocation in JSC initialization in the system malloc mode.
|
||
|
||
* bmalloc/Gigacage.cpp:
|
||
(Gigacage::shouldBeEnabled):
|
||
|
||
2019-02-15 Yusuke Suzuki <ysuzuki@apple.com>
|
||
|
||
[bmalloc] Do not start scavenger thread if we use system malloc
|
||
https://bugs.webkit.org/show_bug.cgi?id=194674
|
||
|
||
Reviewed by Mark Lam.
|
||
|
||
We always start the scavenger thread even if system malloc is used by the environment variable like "Malloc=1".
|
||
Because bmalloc allocation goes to the system malloc if "Malloc=1" is set, we do not need to scavenge. This patch
|
||
changes it not to start the scavenger thread.
|
||
|
||
* bmalloc/Scavenger.cpp:
|
||
(bmalloc::Scavenger::Scavenger):
|
||
|
||
2019-02-12 Commit Queue <commit-queue@webkit.org>
|
||
|
||
Unreviewed, rolling out r241182.
|
||
https://bugs.webkit.org/show_bug.cgi?id=194547
|
||
|
||
causes a 2-3% Speedometer2 regression. (Requested by
|
||
keith_miller on #webkit).
|
||
|
||
Reverted changeset:
|
||
|
||
"bmalloc uses more memory on iOS compared to macOS due to
|
||
physical page size differences"
|
||
https://bugs.webkit.org/show_bug.cgi?id=192389
|
||
https://trac.webkit.org/changeset/241182
|
||
|
||
2019-02-07 Michael Saboff <msaboff@apple.com>
|
||
|
||
bmalloc uses more memory on iOS compared to macOS due to physical page size differences
|
||
https://bugs.webkit.org/show_bug.cgi?id=192389
|
||
|
||
Reviewed by Geoffrey Garen.
|
||
|
||
Changed small line allocations to be in smallPageSize "virtual page" multiples instead of physical
|
||
page size increments for sizes less that the physical page size. This required changing the small
|
||
page commit / decommit code to work in full physical page increments. For page classes that are
|
||
physical page size and larger, there isn't any functional change.
|
||
|
||
When scavenging page classes smaller than the physical page size, we need to consider whether or
|
||
not the adjacent small pages on the same physical page are also free before decommiting that
|
||
containing page. When we need to commit more memory, we commit the whole page, and add any
|
||
adjacent virtual pages that were fully committed as well.
|
||
|
||
* bmalloc/Chunk.h:
|
||
(bmalloc::forEachPage):
|
||
* bmalloc/Heap.cpp:
|
||
(bmalloc::Heap::initializeLineMetadata):
|
||
(bmalloc::Heap::initializePageMetadata):
|
||
(bmalloc::Heap::scavenge):
|
||
(bmalloc::__attribute__):
|
||
(bmalloc::Heap::commitSmallPagesInPhysicalPage):
|
||
(bmalloc::Heap::allocateSmallPage):
|
||
(bmalloc::Heap::allocateSmallBumpRangesByMetadata):
|
||
* bmalloc/Heap.h:
|
||
* bmalloc/SmallPage.h:
|
||
(bmalloc::SmallPage::refCount):
|
||
|
||
2019-01-18 Keith Miller <keith_miller@apple.com>
|
||
|
||
gigacage slide should randomize both start and end
|
||
https://bugs.webkit.org/show_bug.cgi?id=193601
|
||
|
||
Reviewed by Yusuke Suzuki.
|
||
|
||
This patch makes it so that the gigacade slide has an arbitrary
|
||
distance from the end as well as the start. This is done by
|
||
picking a random size then based on that size picking an random
|
||
starting offset.
|
||
|
||
* bmalloc/Gigacage.h:
|
||
* bmalloc/Heap.cpp:
|
||
(bmalloc::Heap::Heap):
|
||
|
||
2019-01-18 Jer Noble <jer.noble@apple.com>
|
||
|
||
SDK_VARIANT build destinations should be separate from non-SDK_VARIANT builds
|
||
https://bugs.webkit.org/show_bug.cgi?id=189553
|
||
|
||
Reviewed by Tim Horton.
|
||
|
||
* Configurations/Base.xcconfig:
|
||
* Configurations/SDKVariant.xcconfig: Added.
|
||
|
||
2019-01-18 Keith Miller <keith_miller@apple.com>
|
||
|
||
Gigacages should start allocations from a slide
|
||
https://bugs.webkit.org/show_bug.cgi?id=193523
|
||
|
||
Reviewed by Mark Lam.
|
||
|
||
This patch makes it so that Gigacage Heaps slide the start of the
|
||
cage by some random amount. We still ensure that there is always
|
||
at least 4/2GB, on MacOS/iOS respectively, of VA space available
|
||
for allocation.
|
||
|
||
Also, this patch changes some macros into constants since macros
|
||
are the devil.
|
||
|
||
* bmalloc/Gigacage.cpp:
|
||
(Gigacage::bmalloc::protectGigacageBasePtrs):
|
||
(Gigacage::bmalloc::unprotectGigacageBasePtrs):
|
||
(Gigacage::bmalloc::runwaySize):
|
||
(Gigacage::ensureGigacage):
|
||
(Gigacage::shouldBeEnabled):
|
||
* bmalloc/Gigacage.h:
|
||
(Gigacage::name):
|
||
(Gigacage::gigacageSizeToMask):
|
||
(Gigacage::size):
|
||
(Gigacage::mask):
|
||
(Gigacage::basePtr):
|
||
(Gigacage::ensureGigacage):
|
||
(Gigacage::wasEnabled):
|
||
(Gigacage::isCaged):
|
||
(Gigacage::isEnabled):
|
||
(Gigacage::caged):
|
||
(Gigacage::disableDisablingPrimitiveGigacageIfShouldBeEnabled):
|
||
(Gigacage::canPrimitiveGigacageBeDisabled):
|
||
(Gigacage::disablePrimitiveGigacage):
|
||
(Gigacage::addPrimitiveDisableCallback):
|
||
(Gigacage::removePrimitiveDisableCallback):
|
||
* bmalloc/Heap.cpp:
|
||
(bmalloc::Heap::Heap):
|
||
* bmalloc/Sizes.h:
|
||
(bmalloc::Sizes::maskSizeClass):
|
||
(bmalloc::Sizes::maskObjectSize):
|
||
(bmalloc::Sizes::logSizeClass):
|
||
(bmalloc::Sizes::logObjectSize):
|
||
(bmalloc::Sizes::sizeClass):
|
||
(bmalloc::Sizes::objectSize):
|
||
(bmalloc::Sizes::pageSize):
|
||
|
||
2019-01-18 Matt Lewis <jlewis3@apple.com>
|
||
|
||
Unreviewed, rolling out r240160.
|
||
|
||
This broke multiple internal builds.
|
||
|
||
Reverted changeset:
|
||
|
||
"Gigacages should start allocations from a slide"
|
||
https://bugs.webkit.org/show_bug.cgi?id=193523
|
||
https://trac.webkit.org/changeset/240160
|
||
|
||
2019-01-18 Keith Miller <keith_miller@apple.com>
|
||
|
||
Gigacages should start allocations from a slide
|
||
https://bugs.webkit.org/show_bug.cgi?id=193523
|
||
|
||
Reviewed by Mark Lam.
|
||
|
||
This patch makes it so that Gigacage Heaps slide the start of the
|
||
cage by some random amount. We still ensure that there is always
|
||
at least 4/2GB, on MacOS/iOS respectively, of VA space available
|
||
for allocation.
|
||
|
||
Also, this patch changes some macros into constants since macros
|
||
are the devil.
|
||
|
||
* bmalloc/Gigacage.cpp:
|
||
(Gigacage::bmalloc::protectGigacageBasePtrs):
|
||
(Gigacage::bmalloc::unprotectGigacageBasePtrs):
|
||
(Gigacage::bmalloc::runwaySize):
|
||
(Gigacage::ensureGigacage):
|
||
(Gigacage::shouldBeEnabled):
|
||
* bmalloc/Gigacage.h:
|
||
(Gigacage::name):
|
||
(Gigacage::gigacageSizeToMask):
|
||
(Gigacage::size):
|
||
(Gigacage::mask):
|
||
(Gigacage::basePtr):
|
||
(Gigacage::ensureGigacage):
|
||
(Gigacage::wasEnabled):
|
||
(Gigacage::isCaged):
|
||
(Gigacage::caged):
|
||
(Gigacage::disableDisablingPrimitiveGigacageIfShouldBeEnabled):
|
||
(Gigacage::disablePrimitiveGigacage):
|
||
(Gigacage::addPrimitiveDisableCallback):
|
||
(Gigacage::removePrimitiveDisableCallback):
|
||
* bmalloc/Heap.cpp:
|
||
(bmalloc::Heap::Heap):
|
||
* bmalloc/Sizes.h:
|
||
(bmalloc::Sizes::maskSizeClass):
|
||
(bmalloc::Sizes::maskObjectSize):
|
||
(bmalloc::Sizes::logSizeClass):
|
||
(bmalloc::Sizes::logObjectSize):
|
||
(bmalloc::Sizes::sizeClass):
|
||
(bmalloc::Sizes::objectSize):
|
||
(bmalloc::Sizes::pageSize):
|
||
|
||
2019-01-17 Truitt Savell <tsavell@apple.com>
|
||
|
||
Unreviewed, rolling out r240124.
|
||
|
||
This commit broke an internal build.
|
||
|
||
Reverted changeset:
|
||
|
||
"SDK_VARIANT build destinations should be separate from non-
|
||
SDK_VARIANT builds"
|
||
https://bugs.webkit.org/show_bug.cgi?id=189553
|
||
https://trac.webkit.org/changeset/240124
|
||
|
||
2019-01-17 Jer Noble <jer.noble@apple.com>
|
||
|
||
SDK_VARIANT build destinations should be separate from non-SDK_VARIANT builds
|
||
https://bugs.webkit.org/show_bug.cgi?id=189553
|
||
|
||
Reviewed by Tim Horton.
|
||
|
||
* Configurations/Base.xcconfig:
|
||
* Configurations/SDKVariant.xcconfig: Added.
|
||
|
||
2019-01-16 Keith Miller <keith_miller@apple.com>
|
||
|
||
bmalloc should use JSC VM tag for gigacage
|
||
https://bugs.webkit.org/show_bug.cgi?id=193496
|
||
|
||
Reviewed by Mark Lam.
|
||
|
||
This patch moves the VMTag info from WTF to bmalloc so that we can
|
||
tag gigacage memory with the unused JSC memory tag. The JSC memory
|
||
tag was previously used for wasm but since wasm is now allocated
|
||
out of the primitive cage it was unused.
|
||
|
||
* bmalloc.xcodeproj/project.pbxproj:
|
||
* bmalloc/BVMTags.h: Copied from Source/WTF/wtf/VMTags.h.
|
||
* bmalloc/Gigacage.cpp:
|
||
(Gigacage::ensureGigacage):
|
||
* bmalloc/VMAllocate.h:
|
||
(bmalloc::tryVMAllocate):
|
||
(bmalloc::vmZeroAndPurge):
|
||
|
||
2019-01-09 Mark Lam <mark.lam@apple.com>
|
||
|
||
Gigacage disabling checks should handle the GIGACAGE_ALLOCATION_CAN_FAIL case properly.
|
||
https://bugs.webkit.org/show_bug.cgi?id=193292
|
||
<rdar://problem/46485450>
|
||
|
||
Reviewed by Yusuke Suzuki.
|
||
|
||
Previously, when GIGACAGE_ALLOCATION_CAN_FAIL is true, we allow the Gigacage to
|
||
be disabled if we fail to allocate memory for it. However, Gigacage::primitiveGigacageDisabled()
|
||
still always assumes that the Gigacage is always enabled after ensureGigacage() is
|
||
called.
|
||
|
||
This patch updates Gigacage::primitiveGigacageDisabled() to allow the Gigacage to
|
||
already be disabled if GIGACAGE_ALLOCATION_CAN_FAIL is true and wasEnabled() is
|
||
false.
|
||
|
||
In this patch, we also put the wasEnabled flag in the 0th slot of the
|
||
g_gigacageBasePtrs buffer to ensure that it is also protected against writes just
|
||
like the Gigacage base pointers.
|
||
|
||
To achieve this, we do the following:
|
||
1. Added a reservedForFlags field in struct BasePtrs.
|
||
2. Added a ReservedForFlagsAndNotABasePtr Gigacage::Kind.
|
||
3. Added assertions to ensure that the BasePtrs::primitive is at the offset
|
||
matching the offset computed from Gigacage::Primitive. Ditto for
|
||
BasePtrs::jsValue and Gigacage::JSValue.
|
||
4. Added assertions to ensure that Gigacage::ReservedForFlagsAndNotABasePtr is not
|
||
used for fetching a Gigacage base pointer.
|
||
5. Added RELEASE_BASSERT_NOT_REACHED() to implement such assertions in bmalloc.
|
||
|
||
No test added because this issue requires Gigacage allocation to fail in order to
|
||
manifest. I've tested it manually by modifying the code locally to force an
|
||
allocation failure.
|
||
|
||
* bmalloc/BAssert.h:
|
||
* bmalloc/Gigacage.cpp:
|
||
(Gigacage::ensureGigacage):
|
||
(Gigacage::primitiveGigacageDisabled):
|
||
* bmalloc/Gigacage.h:
|
||
(Gigacage::wasEnabled):
|
||
(Gigacage::setWasEnabled):
|
||
(Gigacage::name):
|
||
(Gigacage::basePtr):
|
||
(Gigacage::size):
|
||
* bmalloc/HeapKind.h:
|
||
(bmalloc::heapKind):
|
||
|
||
2018-12-15 Yusuke Suzuki <yusukesuzuki@slowstart.org>
|
||
|
||
Unreviewed, suppress warnings in Linux
|
||
|
||
* bmalloc/Gigacage.cpp:
|
||
|
||
2018-12-14 Keith Miller <keith_miller@apple.com>
|
||
|
||
Gigacage runway should immediately follow the primitive cage
|
||
https://bugs.webkit.org/show_bug.cgi?id=192733
|
||
|
||
Reviewed by Saam Barati.
|
||
|
||
This patch makes sure that the Gigacage runway is always
|
||
immediately after the primitive cage. Since writing outside the
|
||
primitive gigacage is likely to be more dangerous than the JSValue
|
||
cage. The ordering of the cages is still random however.
|
||
|
||
* bmalloc/Gigacage.cpp:
|
||
(Gigacage::ensureGigacage):
|
||
|
||
2018-12-13 Mark Lam <mark.lam@apple.com>
|
||
|
||
Verify that tryLargeZeroedMemalignVirtual()'s aligned size and alignment values are valid.
|
||
https://bugs.webkit.org/show_bug.cgi?id=192682
|
||
<rdar://problem/37751522>
|
||
|
||
Reviewed by Saam Barati.
|
||
|
||
* bmalloc/bmalloc.cpp:
|
||
(bmalloc::api::tryLargeZeroedMemalignVirtual):
|
||
|
||
2018-11-21 Dominik Infuehr <dinfuehr@igalia.com>
|
||
|
||
Enable JIT on ARM/Linux
|
||
https://bugs.webkit.org/show_bug.cgi?id=191548
|
||
|
||
Reviewed by Yusuke Suzuki.
|
||
|
||
* bmalloc/IsoPageInlines.h:
|
||
(bmalloc::IsoPage<Config>::startAllocating):
|
||
|
||
2018-11-01 Jiewen Tan <jiewen_tan@apple.com>
|
||
|
||
Replace CommonRandom SPI with API
|
||
https://bugs.webkit.org/show_bug.cgi?id=191178
|
||
<rdar://problem/45722391>
|
||
|
||
Reviewed by Brent Fulgham.
|
||
|
||
* bmalloc/CryptoRandom.cpp:
|
||
(bmalloc::ARC4RandomNumberGenerator::stir):
|
||
|
||
2018-10-29 Mark Lam <mark.lam@apple.com>
|
||
|
||
Correctly detect string overflow when using the 'Function' constructor.
|
||
https://bugs.webkit.org/show_bug.cgi?id=184883
|
||
<rdar://problem/36320331>
|
||
|
||
Reviewed by Saam Barati.
|
||
|
||
* bmalloc/Allocator.cpp:
|
||
(bmalloc::Allocator::reallocate):
|
||
(bmalloc::Allocator::tryReallocate):
|
||
(bmalloc::Allocator::reallocateImpl):
|
||
* bmalloc/Allocator.h:
|
||
* bmalloc/Cache.h:
|
||
(bmalloc::Cache::tryReallocate):
|
||
* bmalloc/DebugHeap.cpp:
|
||
(bmalloc::DebugHeap::realloc):
|
||
* bmalloc/DebugHeap.h:
|
||
* bmalloc/bmalloc.h:
|
||
(bmalloc::api::tryRealloc):
|
||
|
||
2018-10-25 Ross Kirsling <ross.kirsling@sony.com>
|
||
|
||
Cleanup: inline constexpr is redundant as constexpr implies inline
|
||
https://bugs.webkit.org/show_bug.cgi?id=190819
|
||
|
||
Reviewed by Mark Lam.
|
||
|
||
* bmalloc/Algorithm.h:
|
||
(bmalloc::max):
|
||
(bmalloc::min):
|
||
(bmalloc::mask):
|
||
(bmalloc::test):
|
||
(bmalloc::isPowerOfTwo):
|
||
(bmalloc::roundDownToMultipleOf):
|
||
(bmalloc::sizeOf):
|
||
(bmalloc::bitCount):
|
||
(bmalloc::log2):
|
||
* bmalloc/Bits.h:
|
||
(bmalloc::bitsArrayLength):
|
||
* bmalloc/Sizes.h:
|
||
(bmalloc::Sizes::maskSizeClass):
|
||
|
||
2018-10-24 Alexey Proskuryakov <ap@apple.com>
|
||
|
||
Add BPLATFORM(IOS_FAMILY)
|
||
https://bugs.webkit.org/show_bug.cgi?id=190878
|
||
|
||
Reviewed by Saam Barati.
|
||
|
||
* bmalloc/AvailableMemory.cpp:
|
||
(bmalloc::memorySizeAccordingToKernel):
|
||
(bmalloc::computeAvailableMemory):
|
||
* bmalloc/AvailableMemory.h:
|
||
(bmalloc::isUnderMemoryPressure):
|
||
* bmalloc/BPlatform.h:
|
||
* bmalloc/Gigacage.h:
|
||
* bmalloc/Logging.cpp:
|
||
(bmalloc::logVMFailure):
|
||
* bmalloc/VMAllocate.h:
|
||
(bmalloc::vmPageSizePhysical):
|
||
* bmalloc/bmalloc.h:
|
||
* bmalloc/darwin/MemoryStatusSPI.h:
|
||
|
||
2018-10-12 Ryan Haddad <ryanhaddad@apple.com>
|
||
|
||
Unreviewed, rolling out r237063.
|
||
|
||
Caused layout test fast/dom/Window/window-postmessage-clone-
|
||
deep-array.html to fail on macOS and iOS Debug bots.
|
||
|
||
Reverted changeset:
|
||
|
||
"[JSC] Remove gcc warnings on mips and armv7"
|
||
https://bugs.webkit.org/show_bug.cgi?id=188598
|
||
https://trac.webkit.org/changeset/237063
|
||
|
||
2018-10-11 Guillaume Emont <guijemont@igalia.com>
|
||
|
||
[JSC] Remove gcc warnings on mips and armv7
|
||
https://bugs.webkit.org/show_bug.cgi?id=188598
|
||
|
||
Reviewed by Mark Lam.
|
||
|
||
Add bitwise_cast (from WTF) and use it instead of reinterpret_cast in
|
||
a couple places where reinterpret_cast triggers a warning about
|
||
alignment even though we know that alignment is correct.
|
||
|
||
* bmalloc/Algorithm.h:
|
||
(bmalloc::bitwise_cast): Copied from WTF/wtf/StdLibextras.h
|
||
* bmalloc/IsoDirectoryPageInlines.h:
|
||
(bmalloc::IsoDirectoryPage<Config>::pageFor):
|
||
* bmalloc/IsoPageInlines.h:
|
||
(bmalloc::IsoPage<Config>::startAllocating):
|
||
|
||
2018-10-03 Dan Bernstein <mitz@apple.com>
|
||
|
||
bmalloc part of [Xcode] Update some build settings as recommended by Xcode 10
|
||
https://bugs.webkit.org/show_bug.cgi?id=190250
|
||
|
||
Reviewed by Alex Christensen.
|
||
|
||
* Configurations/Base.xcconfig: Enabled CLANG_WARN_COMMA, CLANG_WARN_DEPRECATED_OBJC_IMPLEMENTATIONS,
|
||
and CLANG_WARN_OBJC_IMPLICIT_RETAIN_SELF.
|
||
|
||
* bmalloc.xcodeproj/project.pbxproj: Let Xcode update LastUpgradeCheck.
|
||
|
||
2018-09-25 Alex Christensen <achristensen@webkit.org>
|
||
|
||
Allow for suffixes to com.apple.WebKit.WebContent
|
||
https://bugs.webkit.org/show_bug.cgi?id=189972
|
||
|
||
Reviewed by Chris Dumez.
|
||
|
||
* bmalloc/ProcessCheck.mm:
|
||
(bmalloc::gigacageEnabledForProcess):
|
||
|
||
2018-09-24 Fujii Hironori <Hironori.Fujii@sony.com>
|
||
|
||
Rename WTF_COMPILER_GCC_OR_CLANG to WTF_COMPILER_GCC_COMPATIBLE
|
||
https://bugs.webkit.org/show_bug.cgi?id=189733
|
||
|
||
Reviewed by Michael Catanzaro.
|
||
|
||
* bmalloc/BCompiler.h:
|
||
|
||
2018-08-27 Keith Rollin <krollin@apple.com>
|
||
|
||
Unreviewed build fix -- disable LTO for production builds
|
||
|
||
* Configurations/Base.xcconfig:
|
||
|
||
2018-08-27 Keith Rollin <krollin@apple.com>
|
||
|
||
Build system support for LTO
|
||
https://bugs.webkit.org/show_bug.cgi?id=187785
|
||
<rdar://problem/42353132>
|
||
|
||
Reviewed by Dan Bernstein.
|
||
|
||
Update Base.xcconfig and DebugRelease.xcconfig to optionally enable
|
||
LTO.
|
||
|
||
* Configurations/Base.xcconfig:
|
||
* Configurations/DebugRelease.xcconfig:
|
||
|
||
2018-08-16 Tomas Popela <tpopela@redhat.com>
|
||
|
||
bmalloc: Coverity scan issues
|
||
https://bugs.webkit.org/show_bug.cgi?id=186763
|
||
|
||
Reviewed by Saam Barati.
|
||
|
||
* bmalloc/DebugHeap.h: Initialize the m_pageSize variable.
|
||
* bmalloc/IsoTLS.cpp:
|
||
(bmalloc::IsoTLS::ensureEntries): Check the return value of
|
||
pthread_key_create return().
|
||
* bmalloc/VMAllocate.h:
|
||
(bmalloc::vmPageSize): Correctly check the return value of sysconf().
|
||
|
||
2018-07-27 Mark Lam <mark.lam@apple.com>
|
||
|
||
Initialize bmalloc::DebugHeap::m_pageSize for non-Darwin builds.
|
||
https://bugs.webkit.org/show_bug.cgi?id=188132
|
||
<rdar://problem/40401599>
|
||
|
||
Reviewed by Saam Barati.
|
||
|
||
* bmalloc/DebugHeap.cpp:
|
||
(bmalloc::DebugHeap::DebugHeap):
|
||
|
||
2018-07-27 Saam Barati <sbarati@apple.com>
|
||
|
||
Explicitly handle memlimit_active < 0
|
||
https://bugs.webkit.org/show_bug.cgi?id=188125
|
||
|
||
Reviewed by Mark Lam.
|
||
|
||
This may come up during development when someone wants the limit
|
||
to be "infinite".
|
||
|
||
* bmalloc/AvailableMemory.cpp:
|
||
(bmalloc::jetsamLimit):
|
||
|
||
2018-07-27 Saam Barati <sbarati@apple.com>
|
||
|
||
Use SPI to compute the jetsam limit on iOS instead of hardcoding 840MB
|
||
https://bugs.webkit.org/show_bug.cgi?id=188091
|
||
<rdar://problem/42647697>
|
||
|
||
Reviewed by Simon Fraser.
|
||
|
||
We want bmalloc to dynamically adapt to the jetsam limit of the process
|
||
it's running in. WTF::ramSize() is based off bmalloc's availableMemory,
|
||
so it will now reflect the result of the real jetsam limit when we can
|
||
read it.
|
||
|
||
Reading the jetsam limit requires an entitlement, so this patch opts in
|
||
the WebContent/Storage/Network processes. We fall back to 840MB (the
|
||
old hard coded value) when the SPI call fails (e.g, when we're in a
|
||
process without the proper entitlement).
|
||
|
||
* bmalloc.xcodeproj/project.pbxproj:
|
||
* bmalloc/AvailableMemory.cpp:
|
||
(bmalloc::jetsamLimit):
|
||
(bmalloc::computeAvailableMemory):
|
||
* bmalloc/darwin/MemoryStatusSPI.h: Added.
|
||
|
||
2018-07-24 Saam Barati <sbarati@apple.com>
|
||
|
||
Revert back to using phys_footprint to calculate isUnderMemoryPressure()
|
||
https://bugs.webkit.org/show_bug.cgi?id=187919
|
||
<rdar://problem/42552888>
|
||
|
||
Reviewed by Simon Fraser.
|
||
|
||
Currently on iOS, bmalloc will run the scavenger more frequently when it detects
|
||
that the process is under memory pressure. However, it only uses bmalloc's
|
||
own footprint as a percentage of the HW available memory to determine if
|
||
the process is under memory pressure. This is a change I recently made
|
||
in an effort to run the scavenger less when bmalloc wasn't contributing
|
||
to the dirty footprint in the process. However, this fails to run the
|
||
scavenger eagerly when the process in question has a heap split
|
||
between a lot of dirty bmalloc memory as well as a lot of dirty memory
|
||
from elsewhere. We also have evidence that we may have increased jetsams
|
||
in the Web Content process. Since my original change was not a measurable
|
||
speedup, this patch reverts isUnderMemoryPressure() to its previous
|
||
behavior of using phys_footprint to determine if 75% of the available
|
||
HW memory is being used.
|
||
|
||
* bmalloc/AvailableMemory.cpp:
|
||
(bmalloc::memoryStatus):
|
||
|
||
2019-07-12 Michael Saboff <msaboff@apple.com>
|
||
|
||
Disable IsoHeaps when Gigacage is off
|
||
https://bugs.webkit.org/show_bug.cgi?id=187160
|
||
|
||
Reviewed by Saam Barati.
|
||
|
||
Relanding change sets 233547 and 233550 with the added fix that Gigacage is also
|
||
enabled for DumpRenderTree.
|
||
|
||
Updated determineMallocFallbackState to base enabling of Iso Heaps on Gigacage
|
||
being enabled. We do this because if Gigacage is disabled, it may be due to lack
|
||
of address space.
|
||
|
||
To work around a compiler issue uncovered by the change above, I added explicit
|
||
instantiation of PerThread's static variables. Defined the same explicit
|
||
instantiated static variables with export scope in the new file PerThread.cpp
|
||
to eliminate separate variables allocations in each linked framework / library.
|
||
|
||
* CMakeLists.txt:
|
||
* bmalloc.xcodeproj/project.pbxproj:
|
||
* bmalloc/IsoTLS.cpp:
|
||
(bmalloc::IsoTLS::determineMallocFallbackState):
|
||
* bmalloc/PerThread.cpp: Added.
|
||
* bmalloc/PerThread.h:
|
||
* bmalloc/ProcessCheck.mm:
|
||
(bmalloc::gigacageEnabledForProcess):
|
||
|
||
2018-07-09 Commit Queue <commit-queue@webkit.org>
|
||
|
||
Unreviewed, rolling out r233547 and r233550.
|
||
https://bugs.webkit.org/show_bug.cgi?id=187497
|
||
|
||
Introduced flakiness for media/fullscreen-* tests on mac-wk1
|
||
(Requested by ryanhaddad on #webkit).
|
||
|
||
Reverted changesets:
|
||
|
||
"Disable IsoHeaps when Gigacage is off"
|
||
https://bugs.webkit.org/show_bug.cgi?id=187160
|
||
https://trac.webkit.org/changeset/233547
|
||
|
||
"Build fix (r233547): Disable IsoHeaps when Gigacage is off"
|
||
https://bugs.webkit.org/show_bug.cgi?id=187160
|
||
https://trac.webkit.org/changeset/233550
|
||
|
||
2018-07-05 David Kilzer <ddkilzer@apple.com>
|
||
|
||
Build fix (r233547): Disable IsoHeaps when Gigacage is off
|
||
<https://webkit.org/b/187160>
|
||
|
||
* bmalloc/PerThread.cpp: Add #if !HAVE_PTHREAD_MACHDEP_H/#endif
|
||
around variables only used when that macro is 0. Include what
|
||
you use: Cache.h and Heap.h.
|
||
* bmalloc/PerThread.h: Include <memory> for std::once_flag.
|
||
|
||
2018-07-05 Michael Saboff <msaboff@apple.com>
|
||
|
||
Disable IsoHeaps when Gigacage is off
|
||
https://bugs.webkit.org/show_bug.cgi?id=187160
|
||
|
||
Reviewed by Saam Barati.
|
||
|
||
Updated determineMallocFallbackState to base enabling of Iso Heaps on Gigacage
|
||
being enabled. We do this because if Gigacage is disabled, it may be due to lack
|
||
of address space.
|
||
|
||
To work around a compiler issue uncovered by the change above, I added explicit
|
||
instantiation of PerThread's static variables. Defined the same explicit
|
||
instantiated static variables with export scope in the new file PerThread.cpp
|
||
to eliminate separate variables allocations in each linked framework / library.
|
||
|
||
* CMakeLists.txt:
|
||
* bmalloc.xcodeproj/project.pbxproj:
|
||
* bmalloc/IsoTLS.cpp:
|
||
(bmalloc::IsoTLS::determineMallocFallbackState):
|
||
* bmalloc/PerThread.cpp: Added.
|
||
* bmalloc/PerThread.h:
|
||
|
||
2018-07-04 Tim Horton <timothy_horton@apple.com>
|
||
|
||
Introduce PLATFORM(IOSMAC)
|
||
https://bugs.webkit.org/show_bug.cgi?id=187315
|
||
|
||
Reviewed by Dan Bernstein.
|
||
|
||
* Configurations/Base.xcconfig:
|
||
|
||
2018-06-29 Ryan Haddad <ryanhaddad@apple.com>
|
||
|
||
Unreviewed, rolling out r233347.
|
||
|
||
Causes crashes during WK1 tests.
|
||
|
||
Reverted changeset:
|
||
|
||
"Disable IsoHeaps when Gigacage is off"
|
||
https://bugs.webkit.org/show_bug.cgi?id=187160
|
||
https://trac.webkit.org/changeset/233347
|
||
|
||
2018-06-28 Michael Saboff <msaboff@apple.com>
|
||
|
||
Disable IsoHeaps when Gigacage is off
|
||
https://bugs.webkit.org/show_bug.cgi?id=187160
|
||
|
||
Reviewed by Saam Barati.
|
||
|
||
If Gigacage is disabled, it may be due to lack of address space.
|
||
Therefore we should also turn off IsoHeaps since it uses more virtual
|
||
address space as well.
|
||
|
||
* bmalloc/IsoTLS.cpp:
|
||
(bmalloc::IsoTLS::determineMallocFallbackState):
|
||
|
||
2018-06-27 Simon Fraser <simon.fraser@apple.com>
|
||
|
||
https://hackernoon.com/ uses lots of layer backing store
|
||
https://bugs.webkit.org/show_bug.cgi?id=186909
|
||
rdar://problem/40257540
|
||
|
||
Reviewed by Tim Horton.
|
||
|
||
Drive-by typo fix.
|
||
|
||
* bmalloc/Scavenger.cpp:
|
||
(bmalloc::dumpStats):
|
||
|
||
2018-06-26 Saam Barati <sbarati@apple.com>
|
||
|
||
Unreviewed followup. Fix the watchos build after r233192.
|
||
|
||
This patch also correct the changelog entry below to have the correct
|
||
bug and title info.
|
||
|
||
* bmalloc/ProcessCheck.mm:
|
||
|
||
2018-06-26 Saam Barati <sbarati@apple.com>
|
||
|
||
Switch to system malloc on iOS when nano malloc is disabled
|
||
https://bugs.webkit.org/show_bug.cgi?id=186322
|
||
<rdar://problem/41140257>
|
||
|
||
Reviewed by Keith Miller.
|
||
|
||
We have evidence showing that processes with small heaps using the
|
||
JS API are more space efficient when using system malloc. Our main
|
||
hypothesis as to why this is, is that when dealing with small heaps,
|
||
one malloc can be more efficient at optimizing memory usage than
|
||
two mallocs.
|
||
|
||
* bmalloc/BPlatform.h:
|
||
* bmalloc/Environment.cpp:
|
||
(bmalloc::isNanoMallocEnabled):
|
||
(bmalloc::Environment::computeIsDebugHeapEnabled):
|
||
* bmalloc/ProcessCheck.h:
|
||
(bmalloc::shouldProcessUnconditionallyUseBmalloc):
|
||
* bmalloc/ProcessCheck.mm:
|
||
(bmalloc::shouldProcessUnconditionallyUseBmalloc):
|
||
|
||
2018-06-24 Yusuke Suzuki <utatane.tea@gmail.com>
|
||
|
||
[bmalloc][Linux] Remove static initializers for PerProcess<>::s_object
|
||
https://bugs.webkit.org/show_bug.cgi?id=186966
|
||
|
||
Reviewed by Anders Carlsson.
|
||
|
||
chrome/tools/linux/dump-static-initializers.py can dump static initializers
|
||
in the binary and we found that PerProcess<>::s_object initialization is done
|
||
by static initializers in GCC + Linux environments. The example is the following.
|
||
|
||
Scavenger.cpp (initializer offset 0x38c210 size 0x3e)
|
||
_GLOBAL__sub_I_Scavenger.cpp+0x1e
|
||
_GLOBAL__sub_I_Scavenger.cpp+0x2d
|
||
_GLOBAL__sub_I_Scavenger.cpp+0x3c
|
||
_GLOBAL__sub_I_Scavenger.cpp+0xf
|
||
guard variable for bmalloc::PerProcess<bmalloc::AllIsoHeaps>::s_object@@Base-0x3f0d8
|
||
guard variable for bmalloc::PerProcess<bmalloc::Environment>::s_object@@Base-0x3f0e8
|
||
guard variable for bmalloc::PerProcess<bmalloc::PerHeapKind<bmalloc::Heap> >::s_object@@Base-0x3c600
|
||
guard variable for bmalloc::PerProcess<bmalloc::Scavenger>::s_object@@Base-0x38ce8
|
||
|
||
We can remove this by initializing `nullptr`, which leads to constexpr initialization.
|
||
After this change, Linux JSCOnly libJavaScriptCore.so has no static initializers.
|
||
|
||
* bmalloc/PerProcess.h:
|
||
|
||
2018-06-09 Dan Bernstein <mitz@apple.com>
|
||
|
||
[Xcode] Clean up and modernize some build setting definitions
|
||
https://bugs.webkit.org/show_bug.cgi?id=186463
|
||
|
||
Reviewed by Sam Weinig.
|
||
|
||
* Configurations/Base.xcconfig: Removed definition for macOS 10.11.
|
||
* Configurations/DebugRelease.xcconfig: Ditto.
|
||
|
||
2018-06-07 Darin Adler <darin@apple.com>
|
||
|
||
[Cocoa] Turn on ARC for the single Objective-C++ source file in bmalloc
|
||
https://bugs.webkit.org/show_bug.cgi?id=186398
|
||
|
||
Reviewed by Saam Barati.
|
||
|
||
* Configurations/Base.xcconfig: Turn on ARC.
|
||
* bmalloc/ProcessCheck.mm:
|
||
(bmalloc::gigacageEnabledForProcess): Removed the globals from this function,
|
||
since it's only called once. If it was called more than once, we could optimize
|
||
that with a single boolean global rather than two strings and two booleans.
|
||
|
||
2018-06-07 David Kilzer <ddkilzer@apple.com>
|
||
|
||
bmalloc: Fix 'noreturn' warnings when compiling with -std=gnu++17
|
||
<https://webkit.org/b/186400>
|
||
|
||
Reviewed by Saam Barati.
|
||
|
||
Fixes the following warnings when compiling with gnu++17:
|
||
|
||
Source/bmalloc/bmalloc/Scavenger.cpp:363:1: error: function 'threadRunLoop' could be declared with attribute 'noreturn' [-Werror,-Wmissing-noreturn]
|
||
{
|
||
^
|
||
Source/bmalloc/bmalloc/Scavenger.cpp:358:1: error: function 'threadEntryPoint' could be declared with attribute 'noreturn' [-Werror,-Wmissing-noreturn]
|
||
{
|
||
^
|
||
|
||
* bmalloc/BCompiler.h:
|
||
(BCOMPILER): Add support for the BCOMPILER() macro, then add
|
||
BCOMPILER(GCC_OR_CLANG). Taken from Source/WTF/wtf/Compiler.h.
|
||
(BNO_RETURN): Implement 'norerturn' support using the new
|
||
BCOMPILER() macros. Taken from Source/WTF/wtf/Compiler.h.
|
||
* bmalloc/Scavenger.cpp:
|
||
(bmalloc::Scavenger::threadRunLoop): Remove the workaround that
|
||
tricked older compilers into thinking the while() loop wasn't
|
||
infinite.
|
||
* bmalloc/Scavenger.h:
|
||
(bmalloc::Scavenger::threadEntryPoint): Add BNO_RETURN attribute.
|
||
(bmalloc::Scavenger::threadRunLoop): Ditto.
|
||
|
||
2018-05-29 Saam Barati <sbarati@apple.com>
|
||
|
||
JSC should put bmalloc's scavenger into mini mode
|
||
https://bugs.webkit.org/show_bug.cgi?id=185988
|
||
|
||
Reviewed by Michael Saboff.
|
||
|
||
We expose an API for putting bmalloc into mini mode. All that means now
|
||
is that we'll run the scavenger more aggressively.
|
||
|
||
* bmalloc/Scavenger.cpp:
|
||
(bmalloc::Scavenger::enableMiniMode):
|
||
(bmalloc::Scavenger::threadRunLoop):
|
||
* bmalloc/Scavenger.h:
|
||
* bmalloc/Sizes.h:
|
||
* bmalloc/bmalloc.cpp:
|
||
(bmalloc::api::enableMiniMode):
|
||
* bmalloc/bmalloc.h:
|
||
|
||
2018-05-29 Geoffrey Garen <ggaren@apple.com>
|
||
|
||
Fixed the bmalloc build
|
||
https://bugs.webkit.org/show_bug.cgi?id=186025
|
||
|
||
Reviewed by Sam Weinig.
|
||
|
||
* bmalloc.xcodeproj/project.pbxproj: Link Foundation because the
|
||
gigacage check needs it.
|
||
|
||
2018-05-23 Antti Koivisto <antti@apple.com>
|
||
|
||
Increase the simulated memory size on PLATFORM(IOS_SIMULATOR) from 512MB to 1024MB
|
||
https://bugs.webkit.org/show_bug.cgi?id=185908
|
||
|
||
Reviewed by Geoffrey Garen.
|
||
|
||
We don't support 512MB devices anymore. This will make the simulator behave more
|
||
like a real device.
|
||
|
||
* bmalloc/AvailableMemory.cpp:
|
||
(bmalloc::memorySizeAccordingToKernel):
|
||
|
||
Factor to a function.
|
||
Don't use availableMemoryGuess for the simulator value as it is not a guess.
|
||
|
||
(bmalloc::computeAvailableMemory):
|
||
|
||
Apply the same adjustments to the simulated value too.
|
||
|
||
2018-05-22 Ryan Haddad <ryanhaddad@apple.com>
|
||
|
||
Unreviewed, rolling out r232052.
|
||
|
||
Breaks internal builds.
|
||
|
||
Reverted changeset:
|
||
|
||
"Use more C++17"
|
||
https://bugs.webkit.org/show_bug.cgi?id=185176
|
||
https://trac.webkit.org/changeset/232052
|
||
|
||
2018-05-22 Yusuke Suzuki <utatane.tea@gmail.com>
|
||
|
||
Define GIGACAGE_ALLOCATION_CAN_FAIL on Linux
|
||
https://bugs.webkit.org/show_bug.cgi?id=183329
|
||
|
||
Reviewed by Michael Catanzaro.
|
||
|
||
We specify `GIGACAGE_ALLOCATION_CAN_FAIL 1` in Linux since
|
||
Linux can fail to `mmap` if `vm.overcommit_memory = 2`.
|
||
Users can enable Gigacage if users enable overcommit_memory.
|
||
|
||
* bmalloc/Gigacage.h:
|
||
|
||
2018-05-21 Yusuke Suzuki <utatane.tea@gmail.com>
|
||
|
||
Use more C++17
|
||
https://bugs.webkit.org/show_bug.cgi?id=185176
|
||
|
||
Reviewed by JF Bastien.
|
||
|
||
Add BNO_RETURN.
|
||
|
||
* Configurations/Base.xcconfig:
|
||
* bmalloc/BCompiler.h:
|
||
* bmalloc/Scavenger.h:
|
||
|
||
2018-05-06 Yusuke Suzuki <utatane.tea@gmail.com>
|
||
|
||
[JSC] Remove "using namespace std;" from JSC, bmalloc, WTF
|
||
https://bugs.webkit.org/show_bug.cgi?id=185362
|
||
|
||
Reviewed by Sam Weinig.
|
||
|
||
* bmalloc/Allocator.cpp:
|
||
* bmalloc/Deallocator.cpp:
|
||
|
||
2018-05-03 Filip Pizlo <fpizlo@apple.com>
|
||
|
||
Strings should not be allocated in a gigacage
|
||
https://bugs.webkit.org/show_bug.cgi?id=185218
|
||
|
||
Reviewed by Saam Barati.
|
||
|
||
This removes the string gigacage.
|
||
|
||
Putting strings in a gigacage prevents read gadgets. The other things that get to be in gigacages
|
||
are there to prevent read-write gadgets.
|
||
|
||
Also, putting strings in a gigacage seems to have been a bigger regression than putting other
|
||
things in gigacages.
|
||
|
||
Therefore, to maximize the benefit/cost ratio of gigacages, we should evict strings from them. If
|
||
we want to throw away perf for security, there are more beneficial things to sacrifice.
|
||
|
||
* bmalloc/Gigacage.h:
|
||
(Gigacage::name):
|
||
(Gigacage::basePtr):
|
||
(Gigacage::size):
|
||
(Gigacage::forEachKind):
|
||
* bmalloc/HeapKind.h:
|
||
(bmalloc::isGigacage):
|
||
(bmalloc::gigacageKind):
|
||
(bmalloc::heapKind):
|
||
(bmalloc::isActiveHeapKindAfterEnsuringGigacage):
|
||
(bmalloc::mapToActiveHeapKindAfterEnsuringGigacage):
|
||
|
||
2018-04-30 Yusuke Suzuki <utatane.tea@gmail.com>
|
||
|
||
Use WordLock instead of std::mutex for Threading
|
||
https://bugs.webkit.org/show_bug.cgi?id=185121
|
||
|
||
Reviewed by Geoffrey Garen.
|
||
|
||
Add constexpr to explicitly describe that bmalloc::Mutex constructor is constexpr.
|
||
|
||
* bmalloc/Mutex.h:
|
||
|
||
2018-04-23 Ting-Wei Lan <lantw44@gmail.com>
|
||
|
||
Include stdio.h before using stderr
|
||
https://bugs.webkit.org/show_bug.cgi?id=184872
|
||
|
||
Reviewed by Yusuke Suzuki.
|
||
|
||
* bmalloc/PerProcess.cpp:
|
||
* bmalloc/Scavenger.cpp:
|
||
|
||
2018-04-21 Yusuke Suzuki <utatane.tea@gmail.com>
|
||
|
||
Unreviewed, follow-up patch after r230474
|
||
https://bugs.webkit.org/show_bug.cgi?id=166684
|
||
|
||
Add "JavaScriptCore" to Darwin name. And use short name "BMScavenger"
|
||
for Linux since adding "JavaScriptCore" makes the name too long for Linux.
|
||
|
||
* bmalloc/Scavenger.cpp:
|
||
(bmalloc::Scavenger::threadRunLoop):
|
||
|
||
2018-04-18 Jer Noble <jer.noble@apple.com>
|
||
|
||
Don't put build products into WK_ALTERNATE_WEBKIT_SDK_PATH for engineering builds
|
||
https://bugs.webkit.org/show_bug.cgi?id=184762
|
||
|
||
Reviewed by Dan Bernstein.
|
||
|
||
* Configurations/Base.xcconfig:
|
||
|
||
2018-04-20 Daniel Bates <dabates@apple.com>
|
||
|
||
Remove code for compilers that did not support NSDMI for aggregates
|
||
https://bugs.webkit.org/show_bug.cgi?id=184599
|
||
|
||
Reviewed by Per Arne Vollan.
|
||
|
||
Remove workaround for earlier Visual Studio versions that did not support non-static data
|
||
member initializers (NSDMI) for aggregates. We have since updated all the build.webkit.org
|
||
and EWS bots to a newer version that supports this feature.
|
||
|
||
* bmalloc/BPlatform.h:
|
||
* bmalloc/List.h:
|
||
(bmalloc::ListNode::ListNode): Deleted.
|
||
(bmalloc::List::iterator::iterator): Deleted.
|
||
|
||
2018-04-19 David Kilzer <ddkilzer@apple.com>
|
||
|
||
Enable Objective-C weak references
|
||
<https://webkit.org/b/184789>
|
||
<rdar://problem/39571716>
|
||
|
||
Reviewed by Dan Bernstein.
|
||
|
||
* Configurations/Base.xcconfig:
|
||
(CLANG_ENABLE_OBJC_WEAK): Enable.
|
||
|
||
2018-04-12 Saam Barati <sbarati@apple.com>
|
||
|
||
Lessen partial scavenge interval on x86-64
|
||
https://bugs.webkit.org/show_bug.cgi?id=184577
|
||
|
||
Rubber-stamped by Filip Pizlo.
|
||
|
||
I initially made the scavenge interval longer because I had thought the
|
||
shorter interval caused a JetStream regression. I was mistaken though.
|
||
I was looking at the wrong commit range when analyzing perf data.
|
||
|
||
This patch shortens the interval, but still keeps x86-64 50% longer than
|
||
other architectures. We know that scavenging frequently on Mac is less
|
||
important to overall system performance than it is on iOS.
|
||
|
||
* bmalloc/Scavenger.cpp:
|
||
(bmalloc::Scavenger::threadRunLoop):
|
||
|
||
2018-04-12 Saam Barati <sbarati@apple.com>
|
||
|
||
Raise the partial scavenge interval even more on x86-64
|
||
https://bugs.webkit.org/show_bug.cgi?id=184551
|
||
|
||
Rubber-stamped by Filip Pizlo.
|
||
|
||
The JetStream regression didn't recover from my previous patch.
|
||
This is another attempt to get it to recover perf.
|
||
|
||
* bmalloc/Scavenger.cpp:
|
||
(bmalloc::Scavenger::threadRunLoop):
|
||
|
||
2018-04-11 Saam Barati <sbarati@apple.com>
|
||
|
||
raise partial scavenge interval on x86-64
|
||
https://bugs.webkit.org/show_bug.cgi?id=184521
|
||
|
||
Rubber-stamped by Filip Pizlo.
|
||
|
||
This patch is an attempt to recover the 1-3% JetStream regression
|
||
my initial partial scavenging patch introduced on some Macs.
|
||
|
||
* bmalloc/Scavenger.cpp:
|
||
(bmalloc::Scavenger::threadRunLoop):
|
||
|
||
2018-04-10 Saam Barati <sbarati@apple.com>
|
||
|
||
IsoHeapImpl::scavenge* needs to grab the lock
|
||
https://bugs.webkit.org/show_bug.cgi?id=184461
|
||
|
||
Reviewed by Filip Pizlo.
|
||
|
||
Another thread could be modifying the linked list that the scavenge* methods traverse.
|
||
|
||
* bmalloc/IsoHeapImplInlines.h:
|
||
(bmalloc::IsoHeapImpl<Config>::scavenge):
|
||
(bmalloc::IsoHeapImpl<Config>::scavengeToHighWatermark):
|
||
|
||
2018-04-10 Saam Barati <sbarati@apple.com>
|
||
|
||
bmalloc should do partial scavenges more frequently
|
||
https://bugs.webkit.org/show_bug.cgi?id=184176
|
||
|
||
Reviewed by Filip Pizlo.
|
||
|
||
This patch adds the ability for bmalloc to do a partial scavenge.
|
||
bmalloc will now do a partial scavenge with some frequency even
|
||
when the heap is growing.
|
||
|
||
For Heap, this means tracking the high water mark of where the Heap
|
||
has allocated since the last scavenge. Partial scavenging is just
|
||
decommitting entries in the LargeMap that are past this high water
|
||
mark. Because we allocate in first fit order out of LargeMap, tracking
|
||
the high water mark is a good heuristic of how much memory a partial
|
||
scavenge should decommit.
|
||
|
||
For IsoHeaps, each IsoDirectory also keeps track of its high water mark
|
||
for the furthest page it allocates into. Similar to Heap, we scavenge pages
|
||
past that high water mark. IsoHeapImpl then tracks the high water mark
|
||
for the IsoDirectory it allocates into. We then scavenge all directories
|
||
including and past the directory high water mark. This includes scavenging
|
||
the inline directory when its the only thing we allocate out of since
|
||
the last scavenge.
|
||
|
||
This patch also adds some other capabilities to bmalloc:
|
||
|
||
Heaps and IsoHeaps now track how much memory is freeable. Querying
|
||
this number is now cheap.
|
||
|
||
Heaps no longer hold the global lock when decommitting large ranges.
|
||
Instead, that range is just marked as non eligible to be allocated.
|
||
Then, without the lock held, the scavenger will decommit those ranges.
|
||
Once this is done, the scavenger will then reacquire the lock and mark
|
||
these ranges as eligible. This lessens lock contention between the
|
||
scavenger and the allocation slow path since threads that are taking an
|
||
allocation slow path can now allocate concurrently to the scavenger's
|
||
decommits. The main consideration in adding this functionality is that
|
||
a large allocation may fail while the scavenger is in the process of
|
||
decommitting memory. When the Heap fails to allocate a large range when
|
||
the scavenger is in the middle of a decommit, Heap will wait for the
|
||
Scavenger to finish and then it will try to allocate a large range again.
|
||
|
||
Decommitting from Heap now aggregates the ranges to decommit and tries to
|
||
merge them together to lower the number of calls to vmDeallocatePhysicalPages.
|
||
This is analogous to what IsoHeaps already do.
|
||
|
||
* bmalloc.xcodeproj/project.pbxproj:
|
||
* bmalloc/Allocator.cpp:
|
||
(bmalloc::Allocator::tryAllocate):
|
||
(bmalloc::Allocator::allocateImpl):
|
||
(bmalloc::Allocator::reallocate):
|
||
(bmalloc::Allocator::refillAllocatorSlowCase):
|
||
(bmalloc::Allocator::allocateLarge):
|
||
* bmalloc/BulkDecommit.h: Added.
|
||
(bmalloc::BulkDecommit::addEager):
|
||
(bmalloc::BulkDecommit::addLazy):
|
||
(bmalloc::BulkDecommit::processEager):
|
||
(bmalloc::BulkDecommit::processLazy):
|
||
(bmalloc::BulkDecommit::add):
|
||
(bmalloc::BulkDecommit::process):
|
||
* bmalloc/Deallocator.cpp:
|
||
(bmalloc::Deallocator::scavenge):
|
||
(bmalloc::Deallocator::processObjectLog):
|
||
(bmalloc::Deallocator::deallocateSlowCase):
|
||
* bmalloc/Deallocator.h:
|
||
(bmalloc::Deallocator::lineCache):
|
||
* bmalloc/Heap.cpp:
|
||
(bmalloc::Heap::freeableMemory):
|
||
(bmalloc::Heap::markAllLargeAsEligibile):
|
||
(bmalloc::Heap::decommitLargeRange):
|
||
(bmalloc::Heap::scavenge):
|
||
(bmalloc::Heap::scavengeToHighWatermark):
|
||
(bmalloc::Heap::deallocateLineCache):
|
||
(bmalloc::Heap::allocateSmallChunk):
|
||
(bmalloc::Heap::deallocateSmallChunk):
|
||
(bmalloc::Heap::allocateSmallPage):
|
||
(bmalloc::Heap::deallocateSmallLine):
|
||
(bmalloc::Heap::allocateSmallBumpRangesByMetadata):
|
||
(bmalloc::Heap::allocateSmallBumpRangesByObject):
|
||
(bmalloc::Heap::splitAndAllocate):
|
||
(bmalloc::Heap::tryAllocateLarge):
|
||
(bmalloc::Heap::allocateLarge):
|
||
(bmalloc::Heap::isLarge):
|
||
(bmalloc::Heap::largeSize):
|
||
(bmalloc::Heap::shrinkLarge):
|
||
(bmalloc::Heap::deallocateLarge):
|
||
(bmalloc::Heap::externalCommit):
|
||
(bmalloc::Heap::externalDecommit):
|
||
* bmalloc/Heap.h:
|
||
(bmalloc::Heap::allocateSmallBumpRanges):
|
||
(bmalloc::Heap::derefSmallLine):
|
||
* bmalloc/IsoDirectory.h:
|
||
* bmalloc/IsoDirectoryInlines.h:
|
||
(bmalloc::passedNumPages>::takeFirstEligible):
|
||
(bmalloc::passedNumPages>::didBecome):
|
||
(bmalloc::passedNumPages>::didDecommit):
|
||
(bmalloc::passedNumPages>::scavengePage):
|
||
(bmalloc::passedNumPages>::scavenge):
|
||
(bmalloc::passedNumPages>::scavengeToHighWatermark):
|
||
(bmalloc::passedNumPages>::freeableMemory): Deleted.
|
||
* bmalloc/IsoHeapImpl.h:
|
||
* bmalloc/IsoHeapImplInlines.h:
|
||
(bmalloc::IsoHeapImpl<Config>::takeFirstEligible):
|
||
(bmalloc::IsoHeapImpl<Config>::scavenge):
|
||
(bmalloc::IsoHeapImpl<Config>::scavengeToHighWatermark):
|
||
(bmalloc::IsoHeapImpl<Config>::freeableMemory):
|
||
(bmalloc::IsoHeapImpl<Config>::isNowFreeable):
|
||
(bmalloc::IsoHeapImpl<Config>::isNoLongerFreeable):
|
||
* bmalloc/LargeMap.cpp:
|
||
(bmalloc::LargeMap::remove):
|
||
(bmalloc::LargeMap::markAllAsEligibile):
|
||
* bmalloc/LargeMap.h:
|
||
(bmalloc::LargeMap::size):
|
||
(bmalloc::LargeMap::at):
|
||
* bmalloc/LargeRange.h:
|
||
(bmalloc::LargeRange::setEligible):
|
||
(bmalloc::LargeRange::isEligibile const):
|
||
(bmalloc::canMerge):
|
||
* bmalloc/ObjectType.cpp:
|
||
(bmalloc::objectType):
|
||
* bmalloc/Scavenger.cpp:
|
||
(bmalloc::PrintTime::PrintTime):
|
||
(bmalloc::PrintTime::~PrintTime):
|
||
(bmalloc::PrintTime::print):
|
||
(bmalloc::Scavenger::timeSinceLastFullScavenge):
|
||
(bmalloc::Scavenger::timeSinceLastPartialScavenge):
|
||
(bmalloc::Scavenger::scavenge):
|
||
(bmalloc::Scavenger::partialScavenge):
|
||
(bmalloc::Scavenger::freeableMemory):
|
||
(bmalloc::Scavenger::threadRunLoop):
|
||
* bmalloc/Scavenger.h:
|
||
* bmalloc/SmallLine.h:
|
||
(bmalloc::SmallLine::refCount):
|
||
(bmalloc::SmallLine::ref):
|
||
(bmalloc::SmallLine::deref):
|
||
* bmalloc/SmallPage.h:
|
||
(bmalloc::SmallPage::refCount):
|
||
(bmalloc::SmallPage::hasFreeLines const):
|
||
(bmalloc::SmallPage::setHasFreeLines):
|
||
(bmalloc::SmallPage::ref):
|
||
(bmalloc::SmallPage::deref):
|
||
* bmalloc/bmalloc.cpp:
|
||
(bmalloc::api::tryLargeZeroedMemalignVirtual):
|
||
(bmalloc::api::freeLargeVirtual):
|
||
|
||
2018-04-09 Yusuke Suzuki <utatane.tea@gmail.com>
|
||
|
||
[bmalloc] Name Scavenger thread "bmalloc scavenger"
|
||
https://bugs.webkit.org/show_bug.cgi?id=166684
|
||
|
||
Reviewed by Saam Barati.
|
||
|
||
We name the thread for bmalloc Scavenger "bmalloc scavenger".
|
||
It is useful for debugging. In Linux environment, it will be
|
||
shown in GDB.
|
||
|
||
* bmalloc/Scavenger.cpp:
|
||
(bmalloc::Scavenger::threadRunLoop):
|
||
(bmalloc::Scavenger::setName):
|
||
* bmalloc/Scavenger.h:
|
||
|
||
2018-04-09 Michael Catanzaro <mcatanzaro@igalia.com>
|
||
|
||
Rename UNUSED to BUNUSED
|
||
https://bugs.webkit.org/show_bug.cgi?id=184093
|
||
|
||
Reviewed by Yusuke Suzuki.
|
||
|
||
* bmalloc/BAssert.h:
|
||
* bmalloc/VMAllocate.h:
|
||
(bmalloc::vmValidate):
|
||
(bmalloc::vmValidatePhysical):
|
||
|
||
2018-04-08 Yusuke Suzuki <utatane.tea@gmail.com>
|
||
|
||
Use alignas instead of compiler-specific attributes
|
||
https://bugs.webkit.org/show_bug.cgi?id=183508
|
||
|
||
Reviewed by Mark Lam.
|
||
|
||
Use alignas for g_gigacageBasePtr. We also add reinterpret_cast to fix
|
||
compile errors in ARMv7 and MIPS JSCOnly ports.
|
||
|
||
* bmalloc/Gigacage.cpp:
|
||
* bmalloc/Gigacage.h:
|
||
(Gigacage::basePtrs):
|
||
|
||
2018-04-06 Saam Barati <sbarati@apple.com>
|
||
|
||
bmalloc virtual allocation API should not treat memory it vends as dirty with respect to how it drives the scavenger
|
||
https://bugs.webkit.org/show_bug.cgi?id=184342
|
||
|
||
Reviewed by Mark Lam.
|
||
|
||
Currently, the only user of this API is Wasm. Ideally, Wasm would tell
|
||
us exactly which page is dirtied. We should really do that at some point:
|
||
https://bugs.webkit.org/show_bug.cgi?id=184207
|
||
|
||
However, until we do that, it's better to treat none of the virtual memory
|
||
we vend as dirty, versus what we do now, which is treat it all as dirty.
|
||
This dirty memory tracking helps drive the scavenger, so on iOS, having the
|
||
scavenger think its under memory pressure because of memory it can't free isn't
|
||
useful.
|
||
|
||
* bmalloc/bmalloc.cpp:
|
||
(bmalloc::api::tryLargeZeroedMemalignVirtual):
|
||
(bmalloc::api::freeLargeVirtual):
|
||
* bmalloc/bmalloc.h:
|
||
|
||
2018-04-05 Saam Barati <sbarati@apple.com>
|
||
|
||
IsoHeapImpl not IsoHeapImplBase should add itself to AllIsoHeaps
|
||
https://bugs.webkit.org/show_bug.cgi?id=184174
|
||
|
||
Reviewed by Filip Pizlo.
|
||
|
||
Otherwise, another thread may see a non-fully formed IsoHeapImpl.
|
||
|
||
* bmalloc/IsoHeapImpl.cpp:
|
||
(bmalloc::IsoHeapImplBase::IsoHeapImplBase):
|
||
(bmalloc::IsoHeapImplBase::addToAllIsoHeaps):
|
||
* bmalloc/IsoHeapImpl.h:
|
||
* bmalloc/IsoHeapImplInlines.h:
|
||
(bmalloc::IsoHeapImpl<Config>::IsoHeapImpl):
|
||
|
||
2018-04-05 Yusuke Suzuki <utatane.tea@gmail.com>
|
||
|
||
bmalloc StaticMutex's constructor should be constexpr
|
||
https://bugs.webkit.org/show_bug.cgi?id=180600
|
||
|
||
Reviewed by Mark Lam.
|
||
|
||
StaticMutex and Mutex can be unified. This patch changes std::atomic_flag in StaticMutex
|
||
to std::atomic<bool> to add constexpr constructor to StaticMutex. Then, StaticMutex can
|
||
be initialized in static storage without calling any static initializers.
|
||
And we also rename StaticMutex to Mutex simply.
|
||
|
||
* CMakeLists.txt:
|
||
* bmalloc.xcodeproj/project.pbxproj:
|
||
* bmalloc/AllIsoHeaps.cpp:
|
||
(bmalloc::AllIsoHeaps::AllIsoHeaps):
|
||
* bmalloc/AllIsoHeaps.h:
|
||
* bmalloc/Allocator.cpp:
|
||
(bmalloc::Allocator::tryAllocate):
|
||
(bmalloc::Allocator::allocateImpl):
|
||
(bmalloc::Allocator::reallocate):
|
||
(bmalloc::Allocator::refillAllocatorSlowCase):
|
||
(bmalloc::Allocator::allocateLarge):
|
||
* bmalloc/CryptoRandom.cpp:
|
||
(bmalloc::ARC4RandomNumberGenerator::ARC4RandomNumberGenerator):
|
||
* bmalloc/Deallocator.cpp:
|
||
(bmalloc::Deallocator::scavenge):
|
||
(bmalloc::Deallocator::processObjectLog):
|
||
(bmalloc::Deallocator::deallocateSlowCase):
|
||
* bmalloc/Deallocator.h:
|
||
(bmalloc::Deallocator::lineCache):
|
||
* bmalloc/DebugHeap.cpp:
|
||
(bmalloc::DebugHeap::DebugHeap):
|
||
* bmalloc/DebugHeap.h:
|
||
* bmalloc/Environment.cpp:
|
||
(bmalloc::Environment::Environment):
|
||
* bmalloc/Environment.h:
|
||
* bmalloc/Gigacage.cpp:
|
||
(Gigacage::disablePrimitiveGigacage):
|
||
(Gigacage::addPrimitiveDisableCallback):
|
||
(Gigacage::removePrimitiveDisableCallback):
|
||
* bmalloc/Heap.cpp:
|
||
(bmalloc::Heap::Heap):
|
||
(bmalloc::Heap::freeableMemory):
|
||
(bmalloc::Heap::scavenge):
|
||
(bmalloc::Heap::deallocateLineCache):
|
||
(bmalloc::Heap::allocateSmallChunk):
|
||
(bmalloc::Heap::allocateSmallPage):
|
||
(bmalloc::Heap::deallocateSmallLine):
|
||
(bmalloc::Heap::allocateSmallBumpRangesByMetadata):
|
||
(bmalloc::Heap::allocateSmallBumpRangesByObject):
|
||
(bmalloc::Heap::splitAndAllocate):
|
||
(bmalloc::Heap::tryAllocateLarge):
|
||
(bmalloc::Heap::allocateLarge):
|
||
(bmalloc::Heap::isLarge):
|
||
(bmalloc::Heap::largeSize):
|
||
(bmalloc::Heap::shrinkLarge):
|
||
(bmalloc::Heap::deallocateLarge):
|
||
(bmalloc::Heap::externalCommit):
|
||
(bmalloc::Heap::externalDecommit):
|
||
* bmalloc/Heap.h:
|
||
(bmalloc::Heap::mutex):
|
||
(bmalloc::Heap::allocateSmallBumpRanges):
|
||
(bmalloc::Heap::derefSmallLine):
|
||
* bmalloc/IsoDeallocator.h:
|
||
* bmalloc/IsoHeap.h:
|
||
* bmalloc/IsoTLSDeallocatorEntry.h:
|
||
* bmalloc/IsoTLSDeallocatorEntryInlines.h:
|
||
(bmalloc::IsoTLSDeallocatorEntry<Config>::IsoTLSDeallocatorEntry):
|
||
* bmalloc/IsoTLSInlines.h:
|
||
(bmalloc::IsoTLS::ensureHeap):
|
||
* bmalloc/IsoTLSLayout.cpp:
|
||
(bmalloc::IsoTLSLayout::IsoTLSLayout):
|
||
(bmalloc::IsoTLSLayout::add):
|
||
* bmalloc/IsoTLSLayout.h:
|
||
* bmalloc/Mutex.cpp: Renamed from Source/bmalloc/bmalloc/StaticMutex.cpp.
|
||
(bmalloc::Mutex::lockSlowCase):
|
||
* bmalloc/Mutex.h:
|
||
(bmalloc::sleep):
|
||
(bmalloc::waitUntilFalse):
|
||
(bmalloc::Mutex::try_lock):
|
||
(bmalloc::Mutex::lock):
|
||
(bmalloc::Mutex::unlock):
|
||
(bmalloc::Mutex::Mutex): Deleted.
|
||
* bmalloc/ObjectType.cpp:
|
||
(bmalloc::objectType):
|
||
* bmalloc/PerProcess.cpp:
|
||
(bmalloc::getPerProcessData):
|
||
* bmalloc/PerProcess.h:
|
||
(bmalloc::PerProcess::mutex):
|
||
(bmalloc::PerProcess::getSlowCase):
|
||
* bmalloc/Scavenger.cpp:
|
||
(bmalloc::Scavenger::Scavenger):
|
||
(bmalloc::Scavenger::scavenge):
|
||
(bmalloc::Scavenger::freeableMemory):
|
||
* bmalloc/Scavenger.h:
|
||
* bmalloc/SmallLine.h:
|
||
(bmalloc::SmallLine::refCount):
|
||
(bmalloc::SmallLine::ref):
|
||
(bmalloc::SmallLine::deref):
|
||
* bmalloc/SmallPage.h:
|
||
(bmalloc::SmallPage::refCount):
|
||
(bmalloc::SmallPage::hasFreeLines const):
|
||
(bmalloc::SmallPage::setHasFreeLines):
|
||
(bmalloc::SmallPage::ref):
|
||
(bmalloc::SmallPage::deref):
|
||
* bmalloc/StaticMutex.h: Removed.
|
||
* bmalloc/VMHeap.cpp:
|
||
(bmalloc::VMHeap::VMHeap):
|
||
* bmalloc/VMHeap.h:
|
||
* bmalloc/Zone.cpp:
|
||
(bmalloc::Zone::Zone):
|
||
* bmalloc/Zone.h:
|
||
* bmalloc/bmalloc.cpp:
|
||
(bmalloc::api::tryLargeZeroedMemalignVirtual):
|
||
(bmalloc::api::freeLargeVirtual):
|
||
(bmalloc::api::isEnabled):
|
||
(bmalloc::api::setScavengerThreadQOSClass):
|
||
* bmalloc/bmalloc.h:
|
||
|
||
2018-04-04 Konstantin Tokarev <annulen@yandex.ru>
|
||
|
||
Enable Gigacage unconditionally when building JSCOnly on macOS (build fix)
|
||
https://bugs.webkit.org/show_bug.cgi?id=184301
|
||
|
||
Reviewed by Yusuke Suzuki.
|
||
|
||
bmalloc/ProcessCheck.mm implements specific behavior for Mac and iOS ports,
|
||
which is guarded with BPLATFORM(COCOA). if we don't enable BPLATFORM(MAC)
|
||
or BPLATFORM(IOS) in JSCOnly, then BPLATFORM(COCOA) won't be defined
|
||
as well, and code path from ProcessCheck.mm will not be taken.
|
||
|
||
* CMakeLists.txt: Exclude ProcessCheck.mm from port-independent file
|
||
list.
|
||
* PlatformMac.cmake: Build ProcessCheck.mm for Mac port.
|
||
* bmalloc/BPlatform.h: Don't enable BPLATFORM(MAC) or BPLATFORM(IOS)
|
||
when building JSCOnly port.
|
||
|
||
2018-04-03 Saam Barati <sbarati@apple.com>
|
||
|
||
totalPhysicalSize calculation when splitting a range must account for double rounding effects
|
||
https://bugs.webkit.org/show_bug.cgi?id=184275
|
||
|
||
Reviewed by Mark Lam.
|
||
|
||
The rounding error could happen when we split a range where the
|
||
range's total physical size equals the range's total size. The
|
||
rounding may cause the left size to lose a byte, and the right
|
||
size to gain a byte. This caused the right side to be a byte
|
||
large than its size.
|
||
|
||
* bmalloc/LargeRange.h:
|
||
(bmalloc::LargeRange::LargeRange):
|
||
(bmalloc::LargeRange::split const):
|
||
|
||
2018-04-02 Saam Barati <sbarati@apple.com>
|
||
|
||
bmalloc should compute its own estimate of its footprint
|
||
https://bugs.webkit.org/show_bug.cgi?id=184121
|
||
|
||
Reviewed by Filip Pizlo.
|
||
|
||
This patch makes it so that bmalloc keeps track of its own physical
|
||
footprint.
|
||
|
||
Doing this for IsoHeaps is trivial. It allocates/deallocates fixed
|
||
page sizes at a time. IsoHeapImpl just updates a count every time
|
||
a page is committed/decommitted.
|
||
|
||
Making Heap keep its footprint was a bit trickier because of how
|
||
LargeRange is constructed. Before this patch, LargeRange kept track
|
||
of the amount of physical memory at the start of its range. This
|
||
patch extends large range to also keep track of the total physical memory
|
||
in the range just for footprint bookkeeping. This was needed to make
|
||
Heap's footprint come close to resembling reality, because as we merge and split
|
||
large ranges, the start physical size often becomes wildly inaccurate.
|
||
The total physical size number stored in LargeRange is still just an
|
||
estimate. It's possible that as ranges are split, that the total physical
|
||
size split amongst the two ranges doesn't resemble reality. This can
|
||
happen when the total physical size is really all in one end of the split,
|
||
but we mark it as being proportionally split amongst the resulting two
|
||
ranges. In practice, I did not notice this being a problem. The footprint
|
||
estimate tracks reality very closely (in my testing, within less than 1MB for
|
||
heaps with sizes upwards of 1GB). The other nice thing about total physical
|
||
size is that even if it diverges from reality in terms of how memory is
|
||
using up physical RAM, it stays internally consistent inside bmalloc's
|
||
own data structures.
|
||
|
||
The main oversight of this patch is how it deals with Wasm memory. All Wasm
|
||
memory will be viewed by bmalloc as taking up physical space even when it
|
||
may not be. Wasm memory starts off as taking up purely virtual pages. When a
|
||
page is first accessed, only then will the OS page it in and cause it to use
|
||
physical RAM. I opened a bug to come up with a solution to this problem:
|
||
https://bugs.webkit.org/show_bug.cgi?id=184207
|
||
|
||
* bmalloc.xcodeproj/project.pbxproj:
|
||
* bmalloc/AvailableMemory.cpp:
|
||
(bmalloc::memoryStatus):
|
||
* bmalloc/BPlatform.h:
|
||
* bmalloc/Heap.cpp:
|
||
(bmalloc::Heap::Heap):
|
||
(bmalloc::Heap::freeableMemory):
|
||
(bmalloc::Heap::footprint):
|
||
(bmalloc::Heap::scavenge):
|
||
(bmalloc::Heap::deallocateSmallChunk):
|
||
(bmalloc::Heap::allocateSmallPage):
|
||
(bmalloc::Heap::splitAndAllocate):
|
||
(bmalloc::Heap::tryAllocateLarge):
|
||
(bmalloc::Heap::shrinkLarge):
|
||
(bmalloc::Heap::deallocateLarge):
|
||
(bmalloc::Heap::externalCommit):
|
||
(bmalloc::Heap::externalDecommit):
|
||
* bmalloc/Heap.h:
|
||
* bmalloc/IsoDirectory.h:
|
||
* bmalloc/IsoDirectoryInlines.h:
|
||
(bmalloc::passedNumPages>::takeFirstEligible):
|
||
(bmalloc::passedNumPages>::didDecommit):
|
||
(bmalloc::passedNumPages>::freeableMemory):
|
||
* bmalloc/IsoHeapImpl.h:
|
||
* bmalloc/IsoHeapImplInlines.h:
|
||
(bmalloc::IsoHeapImpl<Config>::freeableMemory):
|
||
(bmalloc::IsoHeapImpl<Config>::footprint):
|
||
(bmalloc::IsoHeapImpl<Config>::didCommit):
|
||
(bmalloc::IsoHeapImpl<Config>::didDecommit):
|
||
* bmalloc/LargeRange.h:
|
||
(bmalloc::LargeRange::LargeRange):
|
||
(bmalloc::LargeRange::startPhysicalSize const):
|
||
(bmalloc::LargeRange::setStartPhysicalSize):
|
||
(bmalloc::LargeRange::totalPhysicalSize const):
|
||
(bmalloc::LargeRange::setTotalPhysicalSize):
|
||
(bmalloc::merge):
|
||
(bmalloc::LargeRange::split const):
|
||
(bmalloc::LargeRange::physicalSize const): Deleted.
|
||
(bmalloc::LargeRange::setPhysicalSize): Deleted.
|
||
* bmalloc/PhysicalPageMap.h: Added.
|
||
This class is added for debugging purposes. It's useful when hacking
|
||
on the code that calculates the footprint to use this map as a sanity
|
||
check. It's just a simple implementation that has a set of all the committed pages.
|
||
|
||
(bmalloc::PhysicalPageMap::commit):
|
||
(bmalloc::PhysicalPageMap::decommit):
|
||
(bmalloc::PhysicalPageMap::footprint):
|
||
(bmalloc::PhysicalPageMap::forEachPhysicalPage):
|
||
* bmalloc/Scavenger.cpp:
|
||
(bmalloc::dumpStats):
|
||
(bmalloc::Scavenger::scavenge):
|
||
(bmalloc::Scavenger::freeableMemory):
|
||
This is here just for debugging for now. But we should implement an
|
||
efficient version of this to use when driving when to run the
|
||
scavenger.
|
||
|
||
(bmalloc::Scavenger::footprint):
|
||
(bmalloc::Scavenger::threadRunLoop):
|
||
* bmalloc/Scavenger.h:
|
||
* bmalloc/VMAllocate.h:
|
||
(bmalloc::physicalPageSizeSloppy):
|
||
* bmalloc/VMHeap.cpp:
|
||
(bmalloc::VMHeap::tryAllocateLargeChunk):
|
||
* bmalloc/bmalloc.cpp:
|
||
(bmalloc::api::commitAlignedPhysical):
|
||
(bmalloc::api::decommitAlignedPhysical):
|
||
* bmalloc/bmalloc.h:
|
||
|
||
2018-03-28 Commit Queue <commit-queue@webkit.org>
|
||
|
||
Unreviewed, rolling out r230005.
|
||
https://bugs.webkit.org/show_bug.cgi?id=184115
|
||
|
||
"it caused a huge regression on iOS" (Requested by saamyjoon
|
||
on #webkit).
|
||
|
||
Reverted changeset:
|
||
|
||
"memoryStatus() is wrong in certain testing scenarios on iOS"
|
||
https://bugs.webkit.org/show_bug.cgi?id=184050
|
||
https://trac.webkit.org/changeset/230005
|
||
|
||
2018-03-27 Saam Barati <sbarati@apple.com>
|
||
|
||
memoryStatus() is wrong in certain testing scenarios on iOS
|
||
https://bugs.webkit.org/show_bug.cgi?id=184050
|
||
<rdar://problem/37959258>
|
||
|
||
Rubber-stamped by Mark Lam.
|
||
|
||
This switches us from using "phys_footprint" to using "internal + compressed"
|
||
when computing the dirty memory in the current process. There are iOS testing
|
||
scenarios where phys_footprint doesn't give us a reliable answer. In my testing,
|
||
"internal + compressed" tracks phys_footprint closely (when phys_footprint is
|
||
working). They're usually within much less than 1% of each other. We're making
|
||
this change to ensure testing in our iOS infrastructure is valid.
|
||
|
||
I opened a bug to move back to phys_footprint when it's feasible:
|
||
https://bugs.webkit.org/show_bug.cgi?id=184050
|
||
|
||
* bmalloc/AvailableMemory.cpp:
|
||
(bmalloc::memoryStatus):
|
||
|
||
2018-03-20 Tim Horton <timothy_horton@apple.com>
|
||
|
||
Add and adopt WK_PLATFORM_NAME and adjust default feature defines
|
||
https://bugs.webkit.org/show_bug.cgi?id=183758
|
||
<rdar://problem/38017644>
|
||
|
||
Reviewed by Dan Bernstein.
|
||
|
||
* Configurations/Base.xcconfig:
|
||
|
||
2018-03-16 Filip Pizlo <fpizlo@apple.com>
|
||
|
||
Put the DOM in IsoHeaps
|
||
https://bugs.webkit.org/show_bug.cgi?id=183546
|
||
|
||
Reviewed by Simon Fraser.
|
||
|
||
Make it easy to runtime-disable IsoHeaps.
|
||
|
||
* bmalloc/Allocator.h:
|
||
* bmalloc/IsoTLS.cpp:
|
||
(bmalloc::IsoTLS::determineMallocFallbackState):
|
||
* bmalloc/IsoTLS.h:
|
||
* bmalloc/IsoTLSInlines.h:
|
||
(bmalloc::IsoTLS::allocateSlow):
|
||
(bmalloc::IsoTLS::deallocateSlow):
|
||
|
||
2018-03-16 Michael Catanzaro <mcatanzaro@igalia.com>
|
||
|
||
Improve error message when Gigacage cannot allocate virtual memory
|
||
https://bugs.webkit.org/show_bug.cgi?id=183329
|
||
|
||
Reviewed by Filip Pizlo.
|
||
|
||
We've discovered that Deja Dup monitor sets a virtual memory limit, breaking Gigacage. Since
|
||
it runs in the background on a fresh out-of-the-box install of Ubuntu, this is not good.
|
||
That will have to be fixed by Deja Dup, but there is concern that other applications might
|
||
try this, or that users will set a virtual memory limit for the entire desktop session. Of
|
||
particular concern is the possibility that users might have copypasted a ulimit line into
|
||
a session startup script without understanding it. Let's try to make it slightly easier to
|
||
understand what's going wrong.
|
||
|
||
* bmalloc/Gigacage.cpp:
|
||
(Gigacage::ensureGigacage):
|
||
|
||
2018-03-13 Tim Horton <timothy_horton@apple.com>
|
||
|
||
Add and adopt WK_ALTERNATE_FRAMEWORKS_DIR in WTF and bmalloc
|
||
https://bugs.webkit.org/show_bug.cgi?id=183576
|
||
<rdar://problem/38396766>
|
||
|
||
Reviewed by Dan Bernstein.
|
||
|
||
* Configurations/Base.xcconfig:
|
||
* Configurations/bmalloc.xcconfig:
|
||
* Configurations/mbmalloc.xcconfig:
|
||
|
||
2018-03-10 Filip Pizlo <fpizlo@apple.com>
|
||
|
||
PerProcess<> should be safe by default
|
||
https://bugs.webkit.org/show_bug.cgi?id=183545
|
||
|
||
Reviewed by Yusuke Suzuki.
|
||
|
||
This makes PerProcess<> safe by default, so we don't need SafePerProcess<>.
|
||
|
||
The new PerProcess<> design relies on a hash-consing mechanism for PerProcess<> storage based
|
||
on the __PRETTY_FUNCTION__ from inside PerProcess<>, which captures the instantiated type in
|
||
the string. Therefore, this can be used to runtime-coalesce PerProcess<> instances based on
|
||
type.
|
||
|
||
I expect this to be perf-neutral. It's an important prerequisite to more bmalloc work, since I
|
||
don't want to have more PerProcess<> vs SafePerProcess<> bugs, and SafePerProcess<> can't be
|
||
used for everything (I don't see how to use it for isoheaps).
|
||
|
||
* CMakeLists.txt:
|
||
* bmalloc.xcodeproj/project.pbxproj:
|
||
* bmalloc/Heap.cpp:
|
||
(bmalloc::Heap::Heap):
|
||
* bmalloc/IsoDirectoryInlines.h:
|
||
(bmalloc::passedNumPages>::takeFirstEligible):
|
||
(bmalloc::passedNumPages>::didBecome):
|
||
* bmalloc/PerProcess.cpp: Added.
|
||
(bmalloc::stringHash):
|
||
(bmalloc::allocate):
|
||
(bmalloc::getPerProcessData):
|
||
* bmalloc/PerProcess.h:
|
||
(bmalloc::PerProcess::mutex):
|
||
(bmalloc::PerProcess::coalesce):
|
||
(bmalloc::PerProcess::getSlowCase):
|
||
(): Deleted.
|
||
* bmalloc/Scavenger.cpp:
|
||
* bmalloc/Scavenger.h:
|
||
* bmalloc/bmalloc.cpp:
|
||
(bmalloc::api::scavenge):
|
||
(bmalloc::api::setScavengerThreadQOSClass):
|
||
|
||
2018-03-10 Commit Queue <commit-queue@webkit.org>
|
||
|
||
Unreviewed, rolling out r229436.
|
||
https://bugs.webkit.org/show_bug.cgi?id=183542
|
||
|
||
seems to have regressed wasm compile times by 10% (Requested
|
||
by pizlo-mbp on #webkit).
|
||
|
||
Reverted changeset:
|
||
|
||
"bmalloc mutex should be adaptive"
|
||
https://bugs.webkit.org/show_bug.cgi?id=177839
|
||
https://trac.webkit.org/changeset/229436
|
||
|
||
2018-03-08 Filip Pizlo <fpizlo@apple.com>
|
||
|
||
bmalloc mutex should be adaptive
|
||
https://bugs.webkit.org/show_bug.cgi?id=177839
|
||
|
||
Reviewed by Michael Saboff.
|
||
|
||
This pulls the WordLock algorithm into bmalloc, mostly by copy-pasting the code. We need to
|
||
copy paste because sometimes we build WTF without bmalloc, so WTF cannot rely on bmalloc for
|
||
anything other than malloc.
|
||
|
||
Reland after failing to reproduce the WasmBench crash that caused it to get rolled out. Maybe that fixed
|
||
itself somehow?
|
||
|
||
* bmalloc/Algorithm.h:
|
||
(bmalloc::compareExchangeWeak):
|
||
(bmalloc::compareExchangeStrong):
|
||
* bmalloc/PerThread.h:
|
||
* bmalloc/StaticMutex.cpp:
|
||
(bmalloc::StaticMutex::lockSlow):
|
||
(bmalloc::StaticMutex::unlockSlow):
|
||
(bmalloc::StaticMutex::lockSlowCase): Deleted.
|
||
* bmalloc/StaticMutex.h:
|
||
(bmalloc::StaticMutex::try_lock):
|
||
(bmalloc::StaticMutex::isLocked const):
|
||
(bmalloc::StaticMutex::init):
|
||
(bmalloc::StaticMutex::tryLock):
|
||
(bmalloc::StaticMutex::lock):
|
||
(bmalloc::StaticMutex::unlock):
|
||
(bmalloc::sleep): Deleted.
|
||
(bmalloc::waitUntilFalse): Deleted.
|
||
|
||
2018-02-16 Filip Pizlo <fpizlo@apple.com>
|
||
|
||
Unreviewed, roll out r228306 (custom memcpy/memset) because the bots say that it was not a
|
||
progression.
|
||
|
||
* bmalloc/Algorithm.h:
|
||
(bmalloc::fastCopy): Deleted.
|
||
(bmalloc::fastZeroFill): Deleted.
|
||
* bmalloc/Allocator.cpp:
|
||
(bmalloc::Allocator::reallocate):
|
||
* bmalloc/Bits.h:
|
||
(bmalloc::BitsWordOwner::operator=):
|
||
(bmalloc::BitsWordOwner::clearAll):
|
||
(bmalloc::BitsWordOwner::set):
|
||
* bmalloc/IsoPageInlines.h:
|
||
(bmalloc::IsoPage<Config>::IsoPage):
|
||
* bmalloc/Vector.h:
|
||
(bmalloc::Vector<T>::reallocateBuffer):
|
||
|
||
2018-02-09 Carlos Alberto Lopez Perez <clopez@igalia.com>
|
||
|
||
Improve of string.h include after r228317.
|
||
https://bugs.webkit.org/show_bug.cgi?id=182642
|
||
|
||
Reviewed by Mark Lam.
|
||
|
||
* bmalloc/Algorithm.h: Avoid an architecture-specific #include.
|
||
|
||
2018-02-09 Carlos Alberto Lopez Perez <clopez@igalia.com>
|
||
|
||
Fix build for !BCPU(X86_64) after r228306
|
||
https://bugs.webkit.org/show_bug.cgi?id=182563
|
||
|
||
Unreviewed build fix.
|
||
|
||
* bmalloc/Algorithm.h:
|
||
|
||
2018-02-08 Filip Pizlo <fpizlo@apple.com>
|
||
|
||
Experiment with alternative implementation of memcpy/memset
|
||
https://bugs.webkit.org/show_bug.cgi?id=182563
|
||
|
||
Reviewed by Michael Saboff and Mark Lam.
|
||
|
||
Add a faster x86_64-specific implementation of memcpy and memset. Ideally, this would just be
|
||
implemented in WTF, but we have to copy it into bmalloc since bmalloc sits below WTF on the
|
||
stack.
|
||
|
||
* bmalloc/Algorithm.h:
|
||
(bmalloc::fastCopy):
|
||
(bmalloc::fastZeroFill):
|
||
* bmalloc/Allocator.cpp:
|
||
(bmalloc::Allocator::reallocate):
|
||
* bmalloc/Bits.h:
|
||
(bmalloc::BitsWordOwner::operator=):
|
||
(bmalloc::BitsWordOwner::clearAll):
|
||
(bmalloc::BitsWordOwner::set):
|
||
* bmalloc/IsoPageInlines.h:
|
||
(bmalloc::IsoPage<Config>::IsoPage):
|
||
* bmalloc/Vector.h:
|
||
(bmalloc::Vector<T>::reallocateBuffer):
|
||
|
||
2018-02-05 JF Bastien <jfbastien@apple.com>
|
||
|
||
Gigacage: enable only for WebContent process and token executables
|
||
https://bugs.webkit.org/show_bug.cgi?id=182457
|
||
<rdar://problem/35875011>
|
||
|
||
Reviewed by Keith Miller.
|
||
|
||
Gigacage is a solid security improvement, but it's probably best
|
||
to roll it out incrementally to the most valuable targets first
|
||
and progressively try out more and more over time rather than
|
||
outright enabling it everywhere. We've gotten some reports that it
|
||
has some side-effects that weren't expected, so for now let's
|
||
enable it for the WebContent process, JSC, and other executables
|
||
we know, and then later we'll enable more gigacage uses.
|
||
|
||
For now I've chosen the following bundles:
|
||
|
||
- com.apple.WebKit.WebContent.Development
|
||
- com.apple.WebKit.WebContent
|
||
- com.apple.WebProcess
|
||
|
||
And the following processes:
|
||
|
||
- jsc
|
||
- wasm
|
||
- anything starting with "test", to match the JSC tests
|
||
|
||
I tried a different approach first, where I add a function to turn
|
||
gigacage on or off and crash if gigacage is initialized without
|
||
having been told what to do. Doing this in ChildProcess and a
|
||
bunch of the process initialization methods isn't sufficient. I
|
||
got MiniBrowser working, but some other builds use static globals
|
||
which themselves use hash and string which are allocate with
|
||
bmalloc and therefore which initialize gigacage before main is
|
||
called and before the process gets a chance to opt in our out. It
|
||
gets tricky with API calls too, because we have to do the right
|
||
thing in any entry an API user could plausibly use, even the
|
||
private ones, so I endend up having to initialize gigacage in e.g.
|
||
WebPreferencesExperimentalFeatures.cpp.erb.
|
||
|
||
Another approach could be to create a free-for-all gigacage
|
||
entitlement, and opt-in the processes we want..
|
||
|
||
As a follow-up we can also check that gigacage allocation always
|
||
succeeds if it was allowed for that process. With my change I
|
||
expect it to always succeed.
|
||
|
||
* CMakeLists.txt:
|
||
* bmalloc.xcodeproj/project.pbxproj:
|
||
* bmalloc/BPlatform.h:
|
||
* bmalloc/Gigacage.cpp:
|
||
(Gigacage::shouldBeEnabled):
|
||
* bmalloc/ProcessCheck.h: Added.
|
||
(bmalloc::gigacageEnabledForProcess):
|
||
* bmalloc/ProcessCheck.mm: Added.
|
||
(bmalloc::gigacageEnabledForProcess):
|
||
|
||
2018-02-05 Joseph Pecoraro <pecoraro@apple.com>
|
||
|
||
Multiple bmalloc scavenger threads is unexpected
|
||
https://bugs.webkit.org/show_bug.cgi?id=182474
|
||
<rdar://problem/37175526>
|
||
|
||
Reviewed by Filip Pizlo.
|
||
|
||
* bmalloc/Heap.cpp:
|
||
(bmalloc::Heap::Heap):
|
||
* bmalloc/IsoDirectoryInlines.h:
|
||
(bmalloc::passedNumPages>::takeFirstEligible):
|
||
(bmalloc::passedNumPages>::didBecome):
|
||
* bmalloc/bmalloc.cpp:
|
||
(bmalloc::api::scavenge):
|
||
(bmalloc::api::setScavengerThreadQOSClass):
|
||
Switch to SafePerProcess for Scavenger to ensure one instance
|
||
for the entire process.
|
||
|
||
* bmalloc/PerProcess.h:
|
||
(bmalloc::PerProcess::get):
|
||
(bmalloc::PerProcess::getFastCase):
|
||
(bmalloc::PerProcess::getSlowCase):
|
||
(bmalloc::SafePerProcess::get):
|
||
(bmalloc::SafePerProcess::getFastCase):
|
||
(bmalloc::SafePerProcess::getSlowCase):
|
||
Duplicate the class with a version that can ensure
|
||
single instances by requiring exporting symbols that
|
||
can be created with macros.
|
||
|
||
* bmalloc/Scavenger.cpp:
|
||
* bmalloc/Scavenger.h:
|
||
Export symbols to ensure all images get the same instance.
|
||
|
||
2018-01-31 Saam Barati <sbarati@apple.com>
|
||
|
||
Replace tryLargeMemalignVirtual with tryLargeZeroedMemalignVirtual and use it to allocate large zeroed memory in Wasm
|
||
https://bugs.webkit.org/show_bug.cgi?id=182064
|
||
<rdar://problem/36840132>
|
||
|
||
Reviewed by Geoffrey Garen.
|
||
|
||
This patch replaces the tryLargeMemalignVirtual API with tryLargeZeroedMemalignVirtual.
|
||
By doing that, we're able to remove the AllocationKind enum. To zero the memory,
|
||
tryLargeZeroedMemalignVirtual uses mmap(... MAP_ANON ...) over previously mmapped
|
||
memory. This both purges the any resident memory for the virtual range and ensures
|
||
that the pages in the range are zeroed. Most OSs should implement this by taking a
|
||
page fault and zero filling on first access. Therefore, this API is returning pages
|
||
that will result in page faults on first access. Hence, the name 'virtual' in the API.
|
||
This API differs from the old API in that users of it need not call madvise themselves.
|
||
The memory is ready to go.
|
||
|
||
* bmalloc.xcodeproj/project.pbxproj:
|
||
* bmalloc/AllocationKind.h: Removed.
|
||
* bmalloc/DebugHeap.cpp:
|
||
(bmalloc::DebugHeap::memalignLarge):
|
||
(bmalloc::DebugHeap::freeLarge):
|
||
* bmalloc/DebugHeap.h:
|
||
* bmalloc/Heap.cpp:
|
||
(bmalloc::Heap::splitAndAllocate):
|
||
(bmalloc::Heap::tryAllocateLarge):
|
||
(bmalloc::Heap::allocateLarge):
|
||
(bmalloc::Heap::shrinkLarge):
|
||
(bmalloc::Heap::deallocateLarge):
|
||
* bmalloc/Heap.h:
|
||
* bmalloc/IsoPage.cpp:
|
||
(bmalloc::IsoPageBase::allocatePageMemory):
|
||
* bmalloc/VMAllocate.h:
|
||
(bmalloc::vmZeroAndPurge):
|
||
* bmalloc/VMHeap.cpp:
|
||
(bmalloc::VMHeap::tryAllocateLargeChunk):
|
||
* bmalloc/VMHeap.h:
|
||
* bmalloc/bmalloc.cpp:
|
||
(bmalloc::api::tryLargeZeroedMemalignVirtual):
|
||
(bmalloc::api::freeLargeVirtual):
|
||
(bmalloc::api::tryLargeMemalignVirtual): Deleted.
|
||
* bmalloc/bmalloc.h:
|
||
|
||
2018-01-19 Keith Miller <keith_miller@apple.com>
|
||
|
||
HaveInternalSDK includes should be "#include?"
|
||
https://bugs.webkit.org/show_bug.cgi?id=179670
|
||
|
||
Reviewed by Dan Bernstein.
|
||
|
||
* Configurations/Base.xcconfig:
|
||
|
||
2018-01-18 Dan Bernstein <mitz@apple.com>
|
||
|
||
[Xcode] Streamline and future-proof target-macOS-version-dependent build setting definitions
|
||
https://bugs.webkit.org/show_bug.cgi?id=181803
|
||
|
||
Reviewed by Tim Horton.
|
||
|
||
* Configurations/Base.xcconfig: Updated.
|
||
* Configurations/DebugRelease.xcconfig: Ditto.
|
||
|
||
2018-01-16 Michael Catanzaro <mcatanzaro@igalia.com>
|
||
|
||
mbmalloc should only be built in developer mode
|
||
https://bugs.webkit.org/show_bug.cgi?id=181654
|
||
|
||
Reviewed by Carlos Garcia Campos.
|
||
|
||
* CMakeLists.txt:
|
||
|
||
2018-01-15 Michael Catanzaro <mcatanzaro@igalia.com>
|
||
|
||
Improve use of ExportMacros
|
||
https://bugs.webkit.org/show_bug.cgi?id=181652
|
||
|
||
Reviewed by Konstantin Tokarev.
|
||
|
||
Disable BEXPORT on Linux ports.
|
||
|
||
* bmalloc/BExport.h: Check for BUSE(EXPORT_MACROS).
|
||
* bmalloc/BPlatform.h: Add BUSE(EXPORT_MACROS) and define it on macOS and iOS.
|
||
|
||
2017-12-20 Ting-Wei Lan <lantw44@gmail.com>
|
||
|
||
Include stdio.h before using stderr and _IONBF
|
||
https://bugs.webkit.org/show_bug.cgi?id=181046
|
||
|
||
Reviewed by Alex Christensen.
|
||
|
||
* bmalloc/IsoTLS.cpp:
|
||
|
||
2017-12-14 David Kilzer <ddkilzer@apple.com>
|
||
|
||
Enable -Wstrict-prototypes for WebKit
|
||
<https://webkit.org/b/180757>
|
||
<rdar://problem/36024132>
|
||
|
||
Rubber-stamped by Joseph Pecoraro.
|
||
|
||
* Configurations/Base.xcconfig:
|
||
(CLANG_WARN_STRICT_PROTOTYPES): Add. Set to YES.
|
||
|
||
2017-12-14 Saam Barati <sbarati@apple.com>
|
||
|
||
logVMFailure should not simulate crash on iOS
|
||
https://bugs.webkit.org/show_bug.cgi?id=180790
|
||
|
||
Reviewed by JF Bastien.
|
||
|
||
The Gigacage allocation on iOS is expected to fail in certain circumstances.
|
||
Let's not simulate a crash on failure because since this is expected behavior.
|
||
|
||
* bmalloc/VMAllocate.h:
|
||
(bmalloc::tryVMAllocate):
|
||
|
||
2017-12-11 Tim Horton <timothy_horton@apple.com>
|
||
|
||
Stop using deprecated target conditional for simulator builds
|
||
https://bugs.webkit.org/show_bug.cgi?id=180662
|
||
<rdar://problem/35136156>
|
||
|
||
Reviewed by Simon Fraser.
|
||
|
||
* bmalloc/BPlatform.h:
|
||
|
||
2017-12-08 Saam Barati <sbarati@apple.com>
|
||
|
||
Enable gigacage on iOS with a 32GB runway and ensure it doesn't break WasmBench
|
||
https://bugs.webkit.org/show_bug.cgi?id=178557
|
||
|
||
Reviewed by Mark Lam.
|
||
|
||
* bmalloc/Algorithm.h:
|
||
(bmalloc::isPowerOfTwo):
|
||
* bmalloc/Gigacage.cpp:
|
||
* bmalloc/Gigacage.h:
|
||
|
||
2017-12-05 Andy Estes <aestes@apple.com>
|
||
|
||
[Darwin] Simplify use of TargetConditionals
|
||
https://bugs.webkit.org/show_bug.cgi?id=180455
|
||
<rdar://problem/35142971>
|
||
|
||
Reviewed by Tim Horton.
|
||
|
||
There's no need to check if TARGET_* macros are defined on Darwin platforms, since
|
||
TargetConditionals.h always defines them. Also, we can simplify
|
||
(TARGET_OS_EMBEDDED || TARGET_OS_IPHONE || TARGET_IPHONE_SIMULATOR) to TARGET_OS_IPHONE.
|
||
|
||
* bmalloc/BPlatform.h:
|
||
|
||
2017-12-05 Filip Pizlo <fpizlo@apple.com>
|
||
|
||
bmalloc IsoHeap needs to allow a thread to deallocate some size for the first time
|
||
https://bugs.webkit.org/show_bug.cgi?id=180443
|
||
|
||
Reviewed by Saam Barati.
|
||
|
||
It's true that we can expect a heap to already be initialized if we try to deallocate in it. But it
|
||
may not have its deallocator initialized on this thread yet.
|
||
|
||
This is easily fixed by adding a null check on the deallocate path. That's probably not going to
|
||
change perf at all. But doing that allows me to get rid of a lot of weird stuff I previously did to
|
||
avoid that null check, like creating a dummy TLS in the DebugHeap case.
|
||
|
||
* bmalloc/IsoTLS.cpp:
|
||
(bmalloc::IsoTLS::debugFree):
|
||
(bmalloc::IsoTLS::deallocateSlow): Deleted.
|
||
* bmalloc/IsoTLS.h:
|
||
* bmalloc/IsoTLSInlines.h:
|
||
(bmalloc::IsoTLS::allocateImpl):
|
||
(bmalloc::IsoTLS::allocateSlow):
|
||
(bmalloc::IsoTLS::deallocateImpl):
|
||
(bmalloc::IsoTLS::deallocateSlow):
|
||
(bmalloc::IsoTLS::ensureHeapAndEntries):
|
||
|
||
2017-12-01 Michael Saboff <msaboff@apple.com>
|
||
|
||
Gigacage should not be enabled for ARM64_32
|
||
https://bugs.webkit.org/show_bug.cgi?id=180265
|
||
|
||
Reviewed by Saam Barati.
|
||
|
||
Disabled Gigacage for ARM64_32.
|
||
In the process, restructured Gigacage::shouldBeEnabled() with GIGACAGE_ENABLED set
|
||
to 0 to avoid a dead code compiler warning.
|
||
|
||
* bmalloc/Gigacage.cpp:
|
||
(Gigacage::shouldBeEnabled):
|
||
* bmalloc/Gigacage.h:
|
||
|
||
2017-11-29 JF Bastien <jfbastien@apple.com>
|
||
|
||
WTF / bmalloc: don't write to 0xbbadbeef when ASAN is looking
|
||
https://bugs.webkit.org/show_bug.cgi?id=180175
|
||
|
||
Reviewed by Mark Lam.
|
||
|
||
ASAN knows that 0xbbadbeef is a bbad aaddress, and tells us so
|
||
when we write to it, say in an assert. That creates bbad error
|
||
reports where ASAN thinks we write to an invalid address, instead
|
||
of thinking that we hit an assertion. In some cases, tooling that
|
||
use fuzzers aggregate similar issues, and think that we just have
|
||
the one bug and not a bunch of different asserts.
|
||
|
||
At the same time, bmalloc's version of CRASH just writes to
|
||
0xbbadbeef and assumes that's invalid and will crash, which isn't
|
||
necessarily true on non-Mac platforms. WTF's version then makes
|
||
sure there's a crash, so bmalloc should do the same.
|
||
|
||
* bmalloc.xcodeproj/project.pbxproj:
|
||
* bmalloc/BAssert.h:
|
||
* bmalloc/BCompiler.h: Added.
|
||
* bmalloc/BPlatform.h:
|
||
|
||
2017-11-27 Filip Pizlo <fpizlo@apple.com>
|
||
|
||
Don't crash in forEachEntry when DebugHeap is enabled.
|
||
|
||
Unreviewed, fixing crashes on leaks bots by removing an assertion.
|
||
|
||
* bmalloc/IsoTLS.cpp:
|
||
(bmalloc::IsoTLS::forEachEntry):
|
||
* test/testbmalloc.cpp: Make this test work with DebugHeap so I can catch this kind of problem in the future.
|
||
|
||
2017-11-16 Filip Pizlo <fpizlo@apple.com>
|
||
|
||
Isolated Heaps caused an increase in reported leaks on the bots
|
||
https://bugs.webkit.org/show_bug.cgi?id=179463
|
||
|
||
Reviewed by Darin Adler.
|
||
|
||
This fixes the way isoheaps interact with system tools:
|
||
|
||
- Opts into the VMHeap API so that the leaks tool can find isoheap memory.
|
||
|
||
- Opts into the DebugHeap/Environment APIs so that we turn off isoheap allocation if memory
|
||
debugging options are in use.
|
||
|
||
* bmalloc.xcodeproj/project.pbxproj:
|
||
* bmalloc/DebugHeap.h:
|
||
* bmalloc/IsoHeap.h:
|
||
* bmalloc/IsoPage.cpp: Added.
|
||
(bmalloc::IsoPageBase::allocatePageMemory):
|
||
* bmalloc/IsoPage.h:
|
||
* bmalloc/IsoPageInlines.h:
|
||
(bmalloc::IsoPage<Config>::tryCreate):
|
||
* bmalloc/IsoTLS.cpp:
|
||
(bmalloc::IsoTLS::deallocateSlow):
|
||
(bmalloc::IsoTLS::ensureEntries):
|
||
(bmalloc::IsoTLS::isUsingDebugHeap):
|
||
(bmalloc::IsoTLS::debugMalloc):
|
||
* bmalloc/IsoTLS.h:
|
||
* bmalloc/IsoTLSInlines.h:
|
||
(bmalloc::IsoTLS::allocate):
|
||
(bmalloc::IsoTLS::deallocate):
|
||
(bmalloc::IsoTLS::allocateImpl):
|
||
(bmalloc::IsoTLS::allocateFast):
|
||
(bmalloc::IsoTLS::allocateSlow):
|
||
(bmalloc::IsoTLS::deallocateImpl):
|
||
(bmalloc::IsoTLS::deallocateFast):
|
||
(bmalloc::IsoTLS::ensureHeapAndEntries):
|
||
(bmalloc::IsoTLS::allocator): Deleted.
|
||
(bmalloc::IsoTLS::deallocator): Deleted.
|
||
* bmalloc/bmalloc.cpp:
|
||
(bmalloc::api::tryLargeMemalignVirtual):
|
||
(bmalloc::api::freeLargeVirtual):
|
||
(bmalloc::api::scavenge):
|
||
(bmalloc::api::isEnabled):
|
||
(bmalloc::api::setScavengerThreadQOSClass):
|
||
* bmalloc/bmalloc.h:
|
||
(bmalloc::api::tryLargeMemalignVirtual): Deleted.
|
||
(bmalloc::api::freeLargeVirtual): Deleted.
|
||
(bmalloc::api::scavenge): Deleted.
|
||
(bmalloc::api::isEnabled): Deleted.
|
||
(bmalloc::api::setScavengerThreadQOSClass): Deleted.
|
||
|
||
2017-11-14 Saam Barati <sbarati@apple.com>
|
||
|
||
Make the gigacage runway 32GB
|
||
https://bugs.webkit.org/show_bug.cgi?id=175062
|
||
|
||
Reviewed by Mark Lam.
|
||
|
||
Making the gigacage runway 32GB defends us against buffer overflows in the
|
||
cage reaching memory outside the cage assuming indices are 32-bit unsigned
|
||
integers and the type they're indexing into has size <= 8 bytes. This is
|
||
exactly the case for many things in JSC. For example, butterfly access in
|
||
JSC meet this criteria, as does typed array access.
|
||
|
||
The 32GB comes from 8 * 2^32 = 32GB.
|
||
|
||
* bmalloc/Gigacage.cpp:
|
||
|
||
2017-11-08 Michael Catanzaro <mcatanzaro@igalia.com>
|
||
|
||
Gigacage.cpp:44:46: warning: ‘*’ in boolean context, suggest ‘&&’ instead [-Wint-in-bool-context]
|
||
https://bugs.webkit.org/show_bug.cgi?id=179427
|
||
|
||
Reviewed by Saam Barati.
|
||
|
||
Tweak the conditional to suppress the warning.
|
||
|
||
* bmalloc/Gigacage.cpp:
|
||
(Gigacage::ensureGigacage):
|
||
|
||
2017-11-07 Saam Barati <sbarati@apple.com>
|
||
|
||
We should PROT_NONE the Gigacage runway so OOB accesses crash
|
||
https://bugs.webkit.org/show_bug.cgi?id=179392
|
||
|
||
Reviewed by Mark Lam.
|
||
|
||
If we assume that an attacker will exploit JSC and cause OOB accesses,
|
||
we should make OOB accesses in the Gigacage runway crash.
|
||
|
||
* bmalloc/Gigacage.cpp:
|
||
(Gigacage::ensureGigacage):
|
||
|
||
2017-10-31 Filip Pizlo <fpizlo@apple.com>
|
||
|
||
bmalloc should support strictly type-segregated isolated heaps
|
||
https://bugs.webkit.org/show_bug.cgi?id=178108
|
||
|
||
Reviewed by Saam Barati, Simon Fraser, and Ryosuke Niwa.
|
||
|
||
This introduces a new allocation API in bmalloc called IsoHeap. An IsoHeap is templatized by
|
||
type and created in static storage. When unused, it takes only a few words. When you do use
|
||
it, each IsoHeap gets a bag of virtual pages unique to it. This prevents use-after-free bugs
|
||
in one IsoHeap from affecting any other memory. At worst, two pointers of the same type will
|
||
point to the same object even though they should not have.
|
||
|
||
IsoHeaps allocate using a first-fit discipline that combines ideas from bmalloc and Riptide
|
||
(the JSC GC):
|
||
|
||
Like Riptide, it uses a bump'n'pop allocator. What Riptide calls blocks, IsoHeaps calls
|
||
pages. Pages are collected into directories. Directories track pages using bitvectors, so
|
||
that it's easy to quickly find a completely free page or one that has at least one free
|
||
object. I think that the bump'n'pop allocator is as fast as the bmalloc Immix-style (page and
|
||
line) allocator, but is better at allocating in holes. It's guaranteed to follow a first-fit
|
||
discipline. However, the real reason why I wrote it that was is that this is what I'm more
|
||
familiar with. This is a part of the design I want to revisit (bug 179278).
|
||
|
||
Like bmalloc, it uses a deallocation log. This means that the internal IsoHeap data
|
||
structures can be locked with a coarse-grained lock, since the deallocator only grabs it when
|
||
flushing the log. Similarly, the allocator only grabs it when refilling the bump'n'pop
|
||
FreeList.
|
||
|
||
This adds a unit test for IsoHeaps. In this change, IsoHeaps are adopted only by WebCore's
|
||
RenderObject.
|
||
|
||
Note that despite the use of GC concepts, it's not a goal to make this code directly sharable
|
||
with GC. The GC will probably have to do isolated heaps its own way (likely a special
|
||
Subspace or something like that).
|
||
|
||
* bmalloc.xcodeproj/project.pbxproj:
|
||
* bmalloc/Algorithm.h:
|
||
(bmalloc::findBitInWord):
|
||
* bmalloc/AllIsoHeaps.cpp: Added.
|
||
(bmalloc::AllIsoHeaps::AllIsoHeaps):
|
||
(bmalloc::AllIsoHeaps::add):
|
||
(bmalloc::AllIsoHeaps::head):
|
||
* bmalloc/AllIsoHeaps.h: Added.
|
||
* bmalloc/AllIsoHeapsInlines.h: Added.
|
||
(bmalloc::AllIsoHeaps::forEach):
|
||
* bmalloc/BMalloced.h: Added.
|
||
* bmalloc/Bits.h: Added.
|
||
(bmalloc::bitsArrayLength):
|
||
(bmalloc::BitsWordView::BitsWordView):
|
||
(bmalloc::BitsWordView::numBits const):
|
||
(bmalloc::BitsWordView::word const):
|
||
(bmalloc::BitsWordOwner::BitsWordOwner):
|
||
(bmalloc::BitsWordOwner::view const):
|
||
(bmalloc::BitsWordOwner::operator=):
|
||
(bmalloc::BitsWordOwner::setAll):
|
||
(bmalloc::BitsWordOwner::clearAll):
|
||
(bmalloc::BitsWordOwner::set):
|
||
(bmalloc::BitsWordOwner::numBits const):
|
||
(bmalloc::BitsWordOwner::arrayLength const):
|
||
(bmalloc::BitsWordOwner::word const):
|
||
(bmalloc::BitsWordOwner::word):
|
||
(bmalloc::BitsWordOwner::words const):
|
||
(bmalloc::BitsWordOwner::words):
|
||
(bmalloc::BitsAndWords::BitsAndWords):
|
||
(bmalloc::BitsAndWords::view const):
|
||
(bmalloc::BitsAndWords::numBits const):
|
||
(bmalloc::BitsAndWords::word const):
|
||
(bmalloc::BitsOrWords::BitsOrWords):
|
||
(bmalloc::BitsOrWords::view const):
|
||
(bmalloc::BitsOrWords::numBits const):
|
||
(bmalloc::BitsOrWords::word const):
|
||
(bmalloc::BitsNotWords::BitsNotWords):
|
||
(bmalloc::BitsNotWords::view const):
|
||
(bmalloc::BitsNotWords::numBits const):
|
||
(bmalloc::BitsNotWords::word const):
|
||
(bmalloc::BitsImpl::BitsImpl):
|
||
(bmalloc::BitsImpl::numBits const):
|
||
(bmalloc::BitsImpl::size const):
|
||
(bmalloc::BitsImpl::arrayLength const):
|
||
(bmalloc::BitsImpl::operator== const):
|
||
(bmalloc::BitsImpl::operator!= const):
|
||
(bmalloc::BitsImpl::at const):
|
||
(bmalloc::BitsImpl::operator[] const):
|
||
(bmalloc::BitsImpl::isEmpty const):
|
||
(bmalloc::BitsImpl::operator& const):
|
||
(bmalloc::BitsImpl::operator| const):
|
||
(bmalloc::BitsImpl::operator~ const):
|
||
(bmalloc::BitsImpl::forEachSetBit const):
|
||
(bmalloc::BitsImpl::forEachClearBit const):
|
||
(bmalloc::BitsImpl::forEachBit const):
|
||
(bmalloc::BitsImpl::findBit const):
|
||
(bmalloc::BitsImpl::findSetBit const):
|
||
(bmalloc::BitsImpl::findClearBit const):
|
||
(bmalloc::BitsImpl::wordView const):
|
||
(bmalloc::BitsImpl::atImpl const):
|
||
(bmalloc::Bits::Bits):
|
||
(bmalloc::Bits::operator=):
|
||
(bmalloc::Bits::resize):
|
||
(bmalloc::Bits::setAll):
|
||
(bmalloc::Bits::clearAll):
|
||
(bmalloc::Bits::setAndCheck):
|
||
(bmalloc::Bits::operator|=):
|
||
(bmalloc::Bits::operator&=):
|
||
(bmalloc::Bits::at const):
|
||
(bmalloc::Bits::operator[] const):
|
||
(bmalloc::Bits::BitReference::BitReference):
|
||
(bmalloc::Bits::BitReference::operator bool const):
|
||
(bmalloc::Bits::BitReference::operator=):
|
||
(bmalloc::Bits::at):
|
||
(bmalloc::Bits::operator[]):
|
||
* bmalloc/CryptoRandom.cpp: Replaced with Source/bmalloc/bmalloc/CryptoRandom.cpp.
|
||
(bmalloc::cryptoRandom):
|
||
* bmalloc/CryptoRandom.h: Replaced with Source/bmalloc/bmalloc/CryptoRandom.h.
|
||
* bmalloc/DeferredDecommit.h: Added.
|
||
* bmalloc/DeferredDecommitInlines.h: Added.
|
||
(bmalloc::DeferredDecommit::DeferredDecommit):
|
||
* bmalloc/DeferredTrigger.h: Added.
|
||
(bmalloc::DeferredTrigger::DeferredTrigger):
|
||
* bmalloc/DeferredTriggerInlines.h: Added.
|
||
(bmalloc::DeferredTrigger<trigger>::didBecome):
|
||
(bmalloc::DeferredTrigger<trigger>::handleDeferral):
|
||
* bmalloc/EligibilityResult.h: Added.
|
||
(bmalloc::EligibilityResult::EligibilityResult):
|
||
* bmalloc/EligibilityResultInlines.h: Added.
|
||
(bmalloc::EligibilityResult<Config>::EligibilityResult):
|
||
* bmalloc/FixedVector.h:
|
||
* bmalloc/FreeList.cpp: Added.
|
||
(bmalloc::FreeList::FreeList):
|
||
(bmalloc::FreeList::~FreeList):
|
||
(bmalloc::FreeList::clear):
|
||
(bmalloc::FreeList::initializeList):
|
||
(bmalloc::FreeList::initializeBump):
|
||
(bmalloc::FreeList::contains const):
|
||
* bmalloc/FreeList.h: Added.
|
||
(bmalloc::FreeCell::scramble):
|
||
(bmalloc::FreeCell::descramble):
|
||
(bmalloc::FreeCell::setNext):
|
||
(bmalloc::FreeCell::next const):
|
||
(bmalloc::FreeList::allocationWillFail const):
|
||
(bmalloc::FreeList::allocationWillSucceed const):
|
||
(bmalloc::FreeList::originalSize const):
|
||
(bmalloc::FreeList::head const):
|
||
* bmalloc/FreeListInlines.h: Added.
|
||
(bmalloc::FreeList::allocate):
|
||
(bmalloc::FreeList::forEach const):
|
||
* bmalloc/IsoAllocator.h: Added.
|
||
* bmalloc/IsoAllocatorInlines.h: Added.
|
||
(bmalloc::IsoAllocator<Config>::IsoAllocator):
|
||
(bmalloc::IsoAllocator<Config>::~IsoAllocator):
|
||
(bmalloc::IsoAllocator<Config>::allocate):
|
||
(bmalloc::IsoAllocator<Config>::allocateSlow):
|
||
(bmalloc::IsoAllocator<Config>::scavenge):
|
||
* bmalloc/IsoConfig.h: Added.
|
||
* bmalloc/IsoDeallocator.h: Added.
|
||
* bmalloc/IsoDeallocatorInlines.h: Added.
|
||
(bmalloc::IsoDeallocator<Config>::IsoDeallocator):
|
||
(bmalloc::IsoDeallocator<Config>::~IsoDeallocator):
|
||
(bmalloc::IsoDeallocator<Config>::deallocate):
|
||
(bmalloc::IsoDeallocator<Config>::scavenge):
|
||
* bmalloc/IsoDirectory.h: Added.
|
||
(bmalloc::IsoDirectoryBaseBase::IsoDirectoryBaseBase):
|
||
(bmalloc::IsoDirectoryBaseBase::~IsoDirectoryBaseBase):
|
||
(bmalloc::IsoDirectoryBase::heap):
|
||
* bmalloc/IsoDirectoryInlines.h: Added.
|
||
(bmalloc::IsoDirectoryBase<Config>::IsoDirectoryBase):
|
||
(bmalloc::passedNumPages>::IsoDirectory):
|
||
(bmalloc::passedNumPages>::takeFirstEligible):
|
||
(bmalloc::passedNumPages>::didBecome):
|
||
(bmalloc::passedNumPages>::didDecommit):
|
||
(bmalloc::passedNumPages>::scavenge):
|
||
(bmalloc::passedNumPages>::forEachCommittedPage):
|
||
* bmalloc/IsoDirectoryPage.h: Added.
|
||
(bmalloc::IsoDirectoryPage::index const):
|
||
* bmalloc/IsoDirectoryPageInlines.h: Added.
|
||
(bmalloc::IsoDirectoryPage<Config>::IsoDirectoryPage):
|
||
(bmalloc::IsoDirectoryPage<Config>::pageFor):
|
||
* bmalloc/IsoHeap.h: Added.
|
||
(bmalloc::api::IsoHeap::allocatorOffset):
|
||
(bmalloc::api::IsoHeap::setAllocatorOffset):
|
||
(bmalloc::api::IsoHeap::deallocatorOffset):
|
||
(bmalloc::api::IsoHeap::setDeallocatorOffset):
|
||
* bmalloc/IsoHeapImpl.cpp: Added.
|
||
(bmalloc::IsoHeapImplBase::IsoHeapImplBase):
|
||
(bmalloc::IsoHeapImplBase::~IsoHeapImplBase):
|
||
(bmalloc::IsoHeapImplBase::scavengeNow):
|
||
(bmalloc::IsoHeapImplBase::finishScavenging):
|
||
* bmalloc/IsoHeapImpl.h: Added.
|
||
* bmalloc/IsoHeapImplInlines.h: Added.
|
||
(bmalloc::IsoHeapImpl<Config>::IsoHeapImpl):
|
||
(bmalloc::IsoHeapImpl<Config>::takeFirstEligible):
|
||
(bmalloc::IsoHeapImpl<Config>::didBecomeEligible):
|
||
(bmalloc::IsoHeapImpl<Config>::scavenge):
|
||
(bmalloc::IsoHeapImpl<Config>::allocatorOffset):
|
||
(bmalloc::IsoHeapImpl<Config>::deallocatorOffset):
|
||
(bmalloc::IsoHeapImpl<Config>::numLiveObjects):
|
||
(bmalloc::IsoHeapImpl<Config>::numCommittedPages):
|
||
(bmalloc::IsoHeapImpl<Config>::forEachDirectory):
|
||
(bmalloc::IsoHeapImpl<Config>::forEachCommittedPage):
|
||
(bmalloc::IsoHeapImpl<Config>::forEachLiveObject):
|
||
* bmalloc/IsoHeapInlines.h: Added.
|
||
(bmalloc::api::IsoHeap<Type>::allocate):
|
||
(bmalloc::api::IsoHeap<Type>::tryAllocate):
|
||
(bmalloc::api::IsoHeap<Type>::deallocate):
|
||
(bmalloc::api::IsoHeap<Type>::scavenge):
|
||
(bmalloc::api::IsoHeap<Type>::isInitialized):
|
||
(bmalloc::api::IsoHeap<Type>::impl):
|
||
* bmalloc/IsoPage.h: Added.
|
||
(bmalloc::IsoPage::index const):
|
||
(bmalloc::IsoPage::directory):
|
||
(bmalloc::IsoPage::isInUseForAllocation const):
|
||
(bmalloc::IsoPage::indexOfFirstObject):
|
||
* bmalloc/IsoPageInlines.h: Added.
|
||
(bmalloc::IsoPage<Config>::tryCreate):
|
||
(bmalloc::IsoPage<Config>::IsoPage):
|
||
(bmalloc::IsoPage<Config>::free):
|
||
(bmalloc::IsoPage<Config>::startAllocating):
|
||
(bmalloc::IsoPage<Config>::stopAllocating):
|
||
(bmalloc::IsoPage<Config>::forEachLiveObject):
|
||
* bmalloc/IsoPageTrigger.h: Added.
|
||
* bmalloc/IsoTLS.cpp: Added.
|
||
(bmalloc::IsoTLS::scavenge):
|
||
(bmalloc::IsoTLS::IsoTLS):
|
||
(bmalloc::IsoTLS::ensureEntries):
|
||
(bmalloc::IsoTLS::destructor):
|
||
(bmalloc::IsoTLS::sizeForCapacity):
|
||
(bmalloc::IsoTLS::capacityForSize):
|
||
(bmalloc::IsoTLS::size):
|
||
(bmalloc::IsoTLS::forEachEntry):
|
||
* bmalloc/IsoTLS.h: Added.
|
||
* bmalloc/IsoTLSAllocatorEntry.h: Added.
|
||
* bmalloc/IsoTLSAllocatorEntryInlines.h: Added.
|
||
(bmalloc::IsoTLSAllocatorEntry<Config>::IsoTLSAllocatorEntry):
|
||
(bmalloc::IsoTLSAllocatorEntry<Config>::~IsoTLSAllocatorEntry):
|
||
(bmalloc::IsoTLSAllocatorEntry<Config>::construct):
|
||
* bmalloc/IsoTLSDeallocatorEntry.h: Added.
|
||
* bmalloc/IsoTLSDeallocatorEntryInlines.h: Added.
|
||
(bmalloc::IsoTLSDeallocatorEntry<Config>::IsoTLSDeallocatorEntry):
|
||
(bmalloc::IsoTLSDeallocatorEntry<Config>::~IsoTLSDeallocatorEntry):
|
||
(bmalloc::IsoTLSDeallocatorEntry<Config>::construct):
|
||
* bmalloc/IsoTLSEntry.cpp: Added.
|
||
(bmalloc::IsoTLSEntry::IsoTLSEntry):
|
||
(bmalloc::IsoTLSEntry::~IsoTLSEntry):
|
||
* bmalloc/IsoTLSEntry.h: Added.
|
||
(bmalloc::IsoTLSEntry::offset const):
|
||
(bmalloc::IsoTLSEntry::alignment const):
|
||
(bmalloc::IsoTLSEntry::size const):
|
||
(bmalloc::IsoTLSEntry::extent const):
|
||
* bmalloc/IsoTLSEntryInlines.h: Added.
|
||
(bmalloc::IsoTLSEntry::walkUpToInclusive):
|
||
(bmalloc::DefaultIsoTLSEntry<EntryType>::DefaultIsoTLSEntry):
|
||
(bmalloc::DefaultIsoTLSEntry<EntryType>::~DefaultIsoTLSEntry):
|
||
(bmalloc::DefaultIsoTLSEntry<EntryType>::move):
|
||
(bmalloc::DefaultIsoTLSEntry<EntryType>::destruct):
|
||
(bmalloc::DefaultIsoTLSEntry<EntryType>::scavenge):
|
||
* bmalloc/IsoTLSInlines.h: Added.
|
||
(bmalloc::IsoTLS::allocate):
|
||
(bmalloc::IsoTLS::deallocate):
|
||
(bmalloc::IsoTLS::scavenge):
|
||
(bmalloc::IsoTLS::allocator):
|
||
(bmalloc::IsoTLS::deallocator):
|
||
(bmalloc::IsoTLS::get):
|
||
(bmalloc::IsoTLS::set):
|
||
(bmalloc::IsoTLS::ensureHeap):
|
||
(bmalloc::IsoTLS::ensureHeapAndEntries):
|
||
* bmalloc/IsoTLSLayout.cpp: Added.
|
||
(bmalloc::IsoTLSLayout::IsoTLSLayout):
|
||
(bmalloc::IsoTLSLayout::add):
|
||
* bmalloc/IsoTLSLayout.h: Added.
|
||
(bmalloc::IsoTLSLayout::head const):
|
||
* bmalloc/PerHeapKind.h:
|
||
* bmalloc/PerProcess.h:
|
||
(bmalloc::PerProcess<T>::getFastCase):
|
||
* bmalloc/Scavenger.cpp:
|
||
(bmalloc::Scavenger::scavenge):
|
||
* bmalloc/Scavenger.h:
|
||
* bmalloc/bmalloc.h:
|
||
(bmalloc::api::scavengeThisThread):
|
||
* test: Added.
|
||
* test/testbmalloc.cpp: Added.
|
||
(hiddenTruthBecauseNoReturnIsStupid):
|
||
(usage):
|
||
(assertEmptyPointerSet):
|
||
(assertHasObjects):
|
||
(assertHasOnlyObjects):
|
||
(assertClean):
|
||
(testIsoSimple):
|
||
(testIsoSimpleScavengeBeforeDealloc):
|
||
(testIsoFlipFlopFragmentedPages):
|
||
(testIsoFlipFlopFragmentedPagesScavengeInMiddle):
|
||
(BisoMalloced::BisoMalloced):
|
||
(testBisoMalloced):
|
||
(BisoMallocedInline::BisoMallocedInline):
|
||
(testBisoMallocedInline):
|
||
(run):
|
||
(main):
|
||
|
||
2017-10-30 Zan Dobersek <zdobersek@igalia.com>
|
||
|
||
[ARM64][Linux] Re-enable Gigacage
|
||
https://bugs.webkit.org/show_bug.cgi?id=178130
|
||
|
||
Reviewed by Michael Catanzaro.
|
||
|
||
* bmalloc/Gigacage.h: Re-enable Gigacage on ARM64 Linux.
|
||
|
||
2017-10-25 Commit Queue <commit-queue@webkit.org>
|
||
|
||
Unreviewed, rolling out r222945.
|
||
https://bugs.webkit.org/show_bug.cgi?id=178818
|
||
|
||
"It made WasmBench crash" (Requested by saamyjoon on #webkit).
|
||
|
||
Reverted changeset:
|
||
|
||
"bmalloc mutex should be adaptive"
|
||
https://bugs.webkit.org/show_bug.cgi?id=177839
|
||
https://trac.webkit.org/changeset/222945
|
||
|
||
2017-10-24 Zan Dobersek <zdobersek@igalia.com>
|
||
|
||
[Linux] Enable Gigacage in x64 Linux environment
|
||
https://bugs.webkit.org/show_bug.cgi?id=177745
|
||
<rdar://problem/34773148>
|
||
|
||
Reviewed by Yusuke Suzuki.
|
||
|
||
Re-enable Gigacage on x86_64 Linux platforms after it was disabled in 223877.
|
||
|
||
The cause for the revert was problems with huge coredumps being generated
|
||
while Gigacage was enabled. The feature virtually allocates about 80GB of
|
||
memory at the beginning of the process lifetime. This is not a problem in
|
||
itself since the memory range is marked as not needed through madvise(),
|
||
but all this memory was still included upon core dump generation on Linux.
|
||
Since there are reasonable limits enforced upon core dumps, these were
|
||
being truncated every time, not yielding any useful information.
|
||
|
||
To avoid this, on Linux, invocations of madvise() with the MADV_NORMAL and
|
||
MADV_DONTNEED advice parameters should be accompanied with respectively
|
||
matching MADV_DODUMP and MADV_DONTDUMP madvise() calls. This correctly
|
||
avoids core-dumping any memory that's not yet been physically allocated.
|
||
|
||
* bmalloc/Gigacage.h:
|
||
* bmalloc/VMAllocate.h:
|
||
(bmalloc::vmDeallocatePhysicalPages):
|
||
(bmalloc::vmAllocatePhysicalPages):
|
||
|
||
2017-10-24 David Kilzer <ddkilzer@apple.com>
|
||
|
||
Need to pass non-nil argument to SimulateCrash() in bmalloc::logVMFailure()
|
||
<https://webkit.org/b/178740>
|
||
<rdar://problem/35154943>
|
||
|
||
Reviewed by Saam Barati.
|
||
|
||
* bmalloc/BPlatform.h:
|
||
(BUNUSED_PARAM): Define macro.
|
||
* bmalloc/Logging.cpp:
|
||
(SimulateCrash): Change third argument of SimulateCrash() to
|
||
CFStringRef since it's an NSString * in Objective-C.
|
||
(bmalloc::logVMFailure): Create a CFStringRef to use as a
|
||
description string. Use new vmSize parameter to log size.
|
||
* bmalloc/Logging.h:
|
||
(bmalloc::logVMFailure): Update function signature to take a
|
||
size_t parameter representing vmSize.
|
||
* bmalloc/VMAllocate.h:
|
||
(bmalloc::tryVMAllocate): Pass vmSize into logVMFailure().
|
||
|
||
2017-10-23 Michael Catanzaro <mcatanzaro@igalia.com>
|
||
|
||
Unreviewed, roll out r222731
|
||
https://bugs.webkit.org/show_bug.cgi?id=177745
|
||
<rdar://problem/34773148>
|
||
|
||
Unfortunately Gigacage has broken core dump generation.
|
||
|
||
* bmalloc/Gigacage.h:
|
||
|
||
2017-10-23 Zan Dobersek <zdobersek@igalia.com>
|
||
|
||
bmalloc::api::tryLargeMemalignVirtual() shouldn't assert on a failed allocation
|
||
https://bugs.webkit.org/show_bug.cgi?id=178654
|
||
|
||
Reviewed by Geoffrey Garen.
|
||
|
||
* bmalloc/bmalloc.h:
|
||
(bmalloc::api::tryLargeMemalignVirtual): Call Heap::tryAllocateLarge()
|
||
instead of Heap::allocateLarge(). The former will return a null pointer
|
||
upon a failed allocation, allowing the caller to fail gracefully just as
|
||
the API entrypoint implies, while the latter currently provokes a crash
|
||
in these circumstances.
|
||
|
||
2017-10-19 Saam Barati <sbarati@apple.com>
|
||
|
||
Runtime disable gigacage on iOS because it broke WasmBench
|
||
https://bugs.webkit.org/show_bug.cgi?id=178556
|
||
|
||
Reviewed by Keith Miller.
|
||
|
||
* bmalloc/Gigacage.cpp:
|
||
(Gigacage::shouldBeEnabled):
|
||
|
||
2017-10-17 Filip Pizlo <fpizlo@apple.com>
|
||
|
||
You can't vmDeallocate null
|
||
<rdar://problem/35038926>
|
||
|
||
Reviewed by Michael Saboff.
|
||
|
||
After failing allocation, we would try to deallocate the thing we failed to allocate. The fix is to
|
||
not try to deallocate something that is obviously null.
|
||
|
||
* bmalloc/Gigacage.cpp:
|
||
(Gigacage::ensureGigacage):
|
||
|
||
2017-09-29 Filip Pizlo <fpizlo@apple.com>
|
||
|
||
Enable gigacage on iOS
|
||
https://bugs.webkit.org/show_bug.cgi?id=177586
|
||
|
||
Reviewed by JF Bastien.
|
||
|
||
Introduce the ability to disable gigacage at runtime if allocation fails. If any step of gigacage
|
||
allocation fails, we free all of the gigacages and turn off gigacage support.
|
||
|
||
Roll this back in after discussion.
|
||
|
||
* CMakeLists.txt:
|
||
* bmalloc.xcodeproj/project.pbxproj:
|
||
* bmalloc/Cache.cpp:
|
||
(bmalloc::Cache::scavenge):
|
||
* bmalloc/Cache.h:
|
||
(bmalloc::Cache::tryAllocate):
|
||
(bmalloc::Cache::allocate):
|
||
(bmalloc::Cache::deallocate):
|
||
(bmalloc::Cache::reallocate):
|
||
* bmalloc/Gigacage.cpp:
|
||
(Gigacage::ensureGigacage):
|
||
(Gigacage::runway):
|
||
(Gigacage::totalSize):
|
||
(Gigacage::shouldBeEnabled):
|
||
(): Deleted.
|
||
(Gigacage::Callback::Callback): Deleted.
|
||
(Gigacage::Callback::function): Deleted.
|
||
(Gigacage::PrimitiveDisableCallbacks::PrimitiveDisableCallbacks): Deleted.
|
||
* bmalloc/Gigacage.h:
|
||
(Gigacage::wasEnabled):
|
||
(Gigacage::isEnabled):
|
||
(Gigacage::runway): Deleted.
|
||
(Gigacage::totalSize): Deleted.
|
||
* bmalloc/HeapKind.cpp: Added.
|
||
(bmalloc::isActiveHeapKind):
|
||
(bmalloc::mapToActiveHeapKind):
|
||
* bmalloc/HeapKind.h:
|
||
(bmalloc::isActiveHeapKindAfterEnsuringGigacage):
|
||
(bmalloc::mapToActiveHeapKindAfterEnsuringGigacage):
|
||
* bmalloc/Scavenger.cpp:
|
||
(bmalloc::Scavenger::scavenge):
|
||
* bmalloc/bmalloc.h:
|
||
(bmalloc::api::tryLargeMemalignVirtual):
|
||
(bmalloc::api::freeLargeVirtual):
|
||
(bmalloc::api::isEnabled):
|
||
|
||
2017-10-11 Commit Queue <commit-queue@webkit.org>
|
||
|
||
Unreviewed, rolling out r223113 and r223121.
|
||
https://bugs.webkit.org/show_bug.cgi?id=178182
|
||
|
||
Reintroduced 20% regression on Kraken (Requested by rniwa on
|
||
#webkit).
|
||
|
||
Reverted changesets:
|
||
|
||
"Enable gigacage on iOS"
|
||
https://bugs.webkit.org/show_bug.cgi?id=177586
|
||
https://trac.webkit.org/changeset/223113
|
||
|
||
"Use one virtual allocation for all gigacages and their
|
||
runways"
|
||
https://bugs.webkit.org/show_bug.cgi?id=178050
|
||
https://trac.webkit.org/changeset/223121
|
||
|
||
2017-10-07 Filip Pizlo <fpizlo@apple.com>
|
||
|
||
Use one virtual allocation for all gigacages and their runways
|
||
https://bugs.webkit.org/show_bug.cgi?id=178050
|
||
|
||
Reviewed by Saam Barati.
|
||
|
||
* bmalloc/Gigacage.cpp:
|
||
(Gigacage::ensureGigacage):
|
||
(Gigacage::runway): Deleted.
|
||
(Gigacage::totalSize): Deleted.
|
||
* bmalloc/Gigacage.h:
|
||
|
||
2017-09-29 Filip Pizlo <fpizlo@apple.com>
|
||
|
||
Enable gigacage on iOS
|
||
https://bugs.webkit.org/show_bug.cgi?id=177586
|
||
|
||
Reviewed by JF Bastien.
|
||
|
||
Introduce the ability to disable gigacage at runtime if allocation fails. If any step of gigacage
|
||
allocation fails, we free all of the gigacages and turn off gigacage support.
|
||
|
||
Reland this after confirming that the 20% Kraken regression was a one-bot fluke. Local testing on the
|
||
same kind of system did not show the regression. Saam and I both tried independently.
|
||
|
||
* CMakeLists.txt:
|
||
* bmalloc.xcodeproj/project.pbxproj:
|
||
* bmalloc/Cache.cpp:
|
||
(bmalloc::Cache::scavenge):
|
||
* bmalloc/Cache.h:
|
||
(bmalloc::Cache::tryAllocate):
|
||
(bmalloc::Cache::allocate):
|
||
(bmalloc::Cache::deallocate):
|
||
(bmalloc::Cache::reallocate):
|
||
* bmalloc/Gigacage.cpp:
|
||
(Gigacage::ensureGigacage):
|
||
(Gigacage::runway):
|
||
(Gigacage::totalSize):
|
||
(Gigacage::shouldBeEnabled):
|
||
(): Deleted.
|
||
(Gigacage::Callback::Callback): Deleted.
|
||
(Gigacage::Callback::function): Deleted.
|
||
(Gigacage::PrimitiveDisableCallbacks::PrimitiveDisableCallbacks): Deleted.
|
||
* bmalloc/Gigacage.h:
|
||
(Gigacage::wasEnabled):
|
||
(Gigacage::isEnabled):
|
||
(Gigacage::runway): Deleted.
|
||
(Gigacage::totalSize): Deleted.
|
||
* bmalloc/HeapKind.cpp: Added.
|
||
(bmalloc::isActiveHeapKind):
|
||
(bmalloc::mapToActiveHeapKind):
|
||
* bmalloc/HeapKind.h:
|
||
(bmalloc::isActiveHeapKindAfterEnsuringGigacage):
|
||
(bmalloc::mapToActiveHeapKindAfterEnsuringGigacage):
|
||
* bmalloc/Scavenger.cpp:
|
||
(bmalloc::Scavenger::scavenge):
|
||
* bmalloc/bmalloc.h:
|
||
(bmalloc::api::tryLargeMemalignVirtual):
|
||
(bmalloc::api::freeLargeVirtual):
|
||
(bmalloc::api::isEnabled):
|
||
|
||
2017-10-09 Commit Queue <commit-queue@webkit.org>
|
||
|
||
Unreviewed, rolling out r223015 and r223025.
|
||
https://bugs.webkit.org/show_bug.cgi?id=178093
|
||
|
||
Regressed Kraken on iOS by 20% (Requested by keith_mi_ on
|
||
#webkit).
|
||
|
||
Reverted changesets:
|
||
|
||
"Enable gigacage on iOS"
|
||
https://bugs.webkit.org/show_bug.cgi?id=177586
|
||
http://trac.webkit.org/changeset/223015
|
||
|
||
"Unreviewed, disable Gigacage on ARM64 Linux"
|
||
https://bugs.webkit.org/show_bug.cgi?id=177586
|
||
http://trac.webkit.org/changeset/223025
|
||
|
||
2017-10-07 Yusuke Suzuki <utatane.tea@gmail.com>
|
||
|
||
Unreviewed, disable Gigacage on ARM64 Linux
|
||
https://bugs.webkit.org/show_bug.cgi?id=177586
|
||
|
||
Gigacage's LLInt change breaks ARM64 Linux.
|
||
Currently we do not have maintainers for this.
|
||
Let's simply disable it.
|
||
|
||
* bmalloc/Gigacage.h:
|
||
|
||
2017-09-29 Filip Pizlo <fpizlo@apple.com>
|
||
|
||
Enable gigacage on iOS
|
||
https://bugs.webkit.org/show_bug.cgi?id=177586
|
||
|
||
Reviewed by JF Bastien.
|
||
|
||
Introduce the ability to disable gigacage at runtime if allocation fails. If any step of gigacage
|
||
allocation fails, we free all of the gigacages and turn off gigacage support.
|
||
|
||
* CMakeLists.txt:
|
||
* bmalloc.xcodeproj/project.pbxproj:
|
||
* bmalloc/Cache.cpp:
|
||
(bmalloc::Cache::scavenge):
|
||
* bmalloc/Cache.h:
|
||
(bmalloc::Cache::tryAllocate):
|
||
(bmalloc::Cache::allocate):
|
||
(bmalloc::Cache::deallocate):
|
||
(bmalloc::Cache::reallocate):
|
||
* bmalloc/Gigacage.cpp:
|
||
(Gigacage::ensureGigacage):
|
||
(Gigacage::runway):
|
||
(Gigacage::totalSize):
|
||
(Gigacage::shouldBeEnabled):
|
||
(): Deleted.
|
||
(Gigacage::Callback::Callback): Deleted.
|
||
(Gigacage::Callback::function): Deleted.
|
||
(Gigacage::PrimitiveDisableCallbacks::PrimitiveDisableCallbacks): Deleted.
|
||
* bmalloc/Gigacage.h:
|
||
(Gigacage::wasEnabled):
|
||
(Gigacage::isEnabled):
|
||
(Gigacage::runway): Deleted.
|
||
(Gigacage::totalSize): Deleted.
|
||
* bmalloc/HeapKind.cpp: Added.
|
||
(bmalloc::isActiveHeapKind):
|
||
(bmalloc::mapToActiveHeapKind):
|
||
* bmalloc/HeapKind.h:
|
||
(bmalloc::isActiveHeapKindAfterEnsuringGigacage):
|
||
(bmalloc::mapToActiveHeapKindAfterEnsuringGigacage):
|
||
* bmalloc/Scavenger.cpp:
|
||
(bmalloc::Scavenger::scavenge):
|
||
* bmalloc/bmalloc.h:
|
||
(bmalloc::api::tryLargeMemalignVirtual):
|
||
(bmalloc::api::freeLargeVirtual):
|
||
(bmalloc::api::isEnabled):
|
||
|
||
2017-10-05 Filip Pizlo <fpizlo@apple.com>
|
||
|
||
Use one Scavenger thread for all Heaps
|
||
https://bugs.webkit.org/show_bug.cgi?id=174973
|
||
|
||
Reviewed by JF Bastien.
|
||
|
||
This combines the scavengers from all Heap instances into a single scavenger. It also combines
|
||
the accounting for deciding when to run. Each Heap still controls what it means to scavenge
|
||
itself (it's all in Heap::scavenge) but the policy decisions are all controlled by Scavenger.
|
||
Because Scavenger is also the only thing that needs an AsyncTask, this removes AsyncTask and
|
||
moves all of AsyncTask's logic into Scavenger.
|
||
|
||
This appears to be a 1% progression on JetStream (with high statistical confidence: p = 0.0049).
|
||
|
||
* bmalloc.xcodeproj/project.pbxproj:
|
||
* bmalloc/AsyncTask.h: Removed.
|
||
* bmalloc/Heap.cpp:
|
||
(bmalloc::Heap::Heap):
|
||
(bmalloc::Heap::allocateSmallChunk):
|
||
(bmalloc::Heap::allocateSmallPage):
|
||
(bmalloc::Heap::deallocateSmallLine):
|
||
(bmalloc::Heap::splitAndAllocate):
|
||
(bmalloc::Heap::tryAllocateLarge):
|
||
(bmalloc::Heap::shrinkLarge):
|
||
(bmalloc::Heap::deallocateLarge):
|
||
(bmalloc::Heap::concurrentScavenge): Deleted.
|
||
(bmalloc::Heap::scheduleScavengerIfUnderMemoryPressure): Deleted.
|
||
(bmalloc::Heap::scheduleScavenger): Deleted.
|
||
* bmalloc/Heap.h:
|
||
* bmalloc/Scavenger.cpp:
|
||
(bmalloc::Scavenger::Scavenger):
|
||
(bmalloc::Scavenger::run):
|
||
(bmalloc::Scavenger::runHoldingLock):
|
||
(bmalloc::Scavenger::runSoon):
|
||
(bmalloc::Scavenger::runSoonHoldingLock):
|
||
(bmalloc::Scavenger::didStartGrowing):
|
||
(bmalloc::Scavenger::scheduleIfUnderMemoryPressure):
|
||
(bmalloc::Scavenger::scheduleIfUnderMemoryPressureHoldingLock):
|
||
(bmalloc::Scavenger::schedule):
|
||
(bmalloc::Scavenger::threadEntryPoint):
|
||
(bmalloc::Scavenger::threadRunLoop):
|
||
(bmalloc::Scavenger::setSelfQOSClass):
|
||
* bmalloc/Scavenger.h:
|
||
(bmalloc::Scavenger::willRun):
|
||
(bmalloc::Scavenger::willRunSoon):
|
||
|
||
2017-10-04 Filip Pizlo <fpizlo@apple.com>
|
||
|
||
bmalloc mutex should be adaptive
|
||
https://bugs.webkit.org/show_bug.cgi?id=177839
|
||
|
||
Reviewed by Michael Saboff.
|
||
|
||
This pulls the WordLock algorithm into bmalloc, mostly by copy-pasting the code. We need to
|
||
copy paste because sometimes we build WTF without bmalloc, so WTF cannot rely on bmalloc for
|
||
anything other than malloc.
|
||
|
||
Reland after fixing ancient WordLock bug: the notify_one has to happen with the lock held
|
||
to ensure it doesn't run after that thread has died.
|
||
|
||
* bmalloc/Algorithm.h:
|
||
(bmalloc::compareExchangeWeak):
|
||
(bmalloc::compareExchangeStrong):
|
||
* bmalloc/PerThread.h:
|
||
* bmalloc/StaticMutex.cpp:
|
||
(bmalloc::StaticMutex::lockSlow):
|
||
(bmalloc::StaticMutex::unlockSlow):
|
||
(bmalloc::StaticMutex::lockSlowCase): Deleted.
|
||
* bmalloc/StaticMutex.h:
|
||
(bmalloc::StaticMutex::try_lock):
|
||
(bmalloc::StaticMutex::isLocked const):
|
||
(bmalloc::StaticMutex::init):
|
||
(bmalloc::StaticMutex::tryLock):
|
||
(bmalloc::StaticMutex::lock):
|
||
(bmalloc::StaticMutex::unlock):
|
||
(bmalloc::sleep): Deleted.
|
||
(bmalloc::waitUntilFalse): Deleted.
|
||
|
||
2017-10-05 Matt Lewis <jlewis3@apple.com>
|
||
|
||
Unreviewed, rolling out r222893.
|
||
|
||
This caused multiple API failures.
|
||
|
||
Reverted changeset:
|
||
|
||
"bmalloc mutex should be adaptive"
|
||
https://bugs.webkit.org/show_bug.cgi?id=177839
|
||
http://trac.webkit.org/changeset/222893
|
||
|
||
2017-10-05 Yusuke Suzuki <utatane.tea@gmail.com>
|
||
|
||
[Linux] Port MallocBench
|
||
https://bugs.webkit.org/show_bug.cgi?id=177856
|
||
|
||
Reviewed by Filip Pizlo.
|
||
|
||
* CMakeLists.txt:
|
||
|
||
2017-10-04 Filip Pizlo <fpizlo@apple.com>
|
||
|
||
bmalloc mutex should be adaptive
|
||
https://bugs.webkit.org/show_bug.cgi?id=177839
|
||
|
||
Reviewed by Michael Saboff.
|
||
|
||
This pulls the WordLock algorithm into bmalloc, mostly by copy-pasting the code. We need to
|
||
copy paste because sometimes we build WTF without bmalloc, so WTF cannot rely on bmalloc for
|
||
anything other than malloc.
|
||
|
||
* bmalloc/Algorithm.h:
|
||
(bmalloc::compareExchangeWeak):
|
||
(bmalloc::compareExchangeStrong):
|
||
* bmalloc/PerThread.h:
|
||
* bmalloc/StaticMutex.cpp:
|
||
(bmalloc::StaticMutex::lockSlow):
|
||
(bmalloc::StaticMutex::unlockSlow):
|
||
(bmalloc::StaticMutex::lockSlowCase): Deleted.
|
||
* bmalloc/StaticMutex.h:
|
||
(bmalloc::StaticMutex::try_lock):
|
||
(bmalloc::StaticMutex::isLocked const):
|
||
(bmalloc::StaticMutex::init):
|
||
(bmalloc::StaticMutex::tryLock):
|
||
(bmalloc::StaticMutex::lock):
|
||
(bmalloc::StaticMutex::unlock):
|
||
(bmalloc::sleep): Deleted.
|
||
(bmalloc::waitUntilFalse): Deleted.
|
||
|
||
2017-10-02 Yusuke Suzuki <utatane.tea@gmail.com>
|
||
|
||
[Linux] Enable Gigacage in x64 Linux environment
|
||
https://bugs.webkit.org/show_bug.cgi?id=177745
|
||
|
||
Reviewed by Carlos Garcia Campos.
|
||
|
||
This patch enables Gigacage in x64 Linux environment.
|
||
Gigacage enforces a caged pointer to reference to the
|
||
specific memory region. This reduces the effectiveness
|
||
of some types of attacks setting a pointer to ArrayBuffer
|
||
and modifying arbitrary memory region.
|
||
|
||
* bmalloc/Gigacage.h:
|
||
|
||
2017-09-29 Commit Queue <commit-queue@webkit.org>
|
||
|
||
Unreviewed, rolling out r222625.
|
||
https://bugs.webkit.org/show_bug.cgi?id=177664
|
||
|
||
causes crashes on iOS (Requested by pizlo-mbp on #webkit).
|
||
|
||
Reverted changeset:
|
||
|
||
"Enable gigacage on iOS"
|
||
https://bugs.webkit.org/show_bug.cgi?id=177586
|
||
http://trac.webkit.org/changeset/222625
|
||
|
||
2017-09-28 Filip Pizlo <fpizlo@apple.com>
|
||
|
||
Enable gigacage on iOS
|
||
https://bugs.webkit.org/show_bug.cgi?id=177586
|
||
|
||
Reviewed by Michael Saboff.
|
||
|
||
This enables Gigacage on iOS using a much smaller cage size. It's not necessary for it to be so
|
||
small, but this is a good conservative starting point to start to exercise the code.
|
||
|
||
* bmalloc/Gigacage.h:
|
||
|
||
2017-09-26 Filip Pizlo <fpizlo@apple.com>
|
||
|
||
Put g_gigacageBasePtr into its own page and make it read-only
|
||
https://bugs.webkit.org/show_bug.cgi?id=174972
|
||
|
||
Reviewed by Michael Saboff.
|
||
|
||
This puts the gigacage base pointers into their own page and makes that page read-only.
|
||
|
||
* bmalloc/Gigacage.cpp:
|
||
(Gigacage::ensureGigacage):
|
||
(Gigacage::disablePrimitiveGigacage):
|
||
(Gigacage::addPrimitiveDisableCallback):
|
||
* bmalloc/Gigacage.h:
|
||
(Gigacage::basePtr):
|
||
(Gigacage::basePtrs):
|
||
|
||
2017-09-04 Adrian Perez de Castro <aperez@igalia.com>
|
||
|
||
Unreviewed build fix for Clang with libc++
|
||
|
||
Fixes a build failure when building with Clang, -stdlib=libc++, and gigacage
|
||
support enabled, which resulted in "stderr" being undefined.
|
||
|
||
* bmalloc/Gigacage.cpp: Add missing <ctsdio> include to pull the definition.
|
||
|
||
2017-09-03 Yusuke Suzuki <utatane.tea@gmail.com>
|
||
|
||
Large virtual memory region allocation requires MMAP_NORESERVE in Linux
|
||
https://bugs.webkit.org/show_bug.cgi?id=176211
|
||
|
||
Reviewed by Geoffrey Garen.
|
||
|
||
In Linux, we cannot allocate very large memory region without MMAP_NORESERVE.
|
||
Linux kernel needs to reserve swap area for allocated memory region. If the
|
||
swap area is exhausted, kernel fails to allocate the memory region with ENOMEM.
|
||
|
||
This patch adds MMAP_NORESERVE to mmap flags in Linux. By adding this flag,
|
||
mmap does not need to reserve swap area for the reserved memory region.
|
||
This allows us to reserve very large memory region that is necessary for Gigacage.
|
||
|
||
* bmalloc/BPlatform.h:
|
||
* bmalloc/VMAllocate.h:
|
||
(bmalloc::tryVMAllocate):
|
||
|
||
2017-08-22 Filip Pizlo <fpizlo@apple.com>
|
||
|
||
Strings need to be in some kind of gigacage
|
||
https://bugs.webkit.org/show_bug.cgi?id=174924
|
||
|
||
Reviewed by Oliver Hunt.
|
||
|
||
This adds a StringGigacage.
|
||
|
||
* bmalloc/Gigacage.cpp:
|
||
* bmalloc/Gigacage.h:
|
||
(Gigacage::name):
|
||
(Gigacage::basePtr):
|
||
(Gigacage::forEachKind):
|
||
* bmalloc/HeapKind.h:
|
||
(bmalloc::isGigacage):
|
||
(bmalloc::gigacageKind):
|
||
(bmalloc::heapKind):
|
||
|
||
2017-08-30 Matt Lewis <jlewis3@apple.com>
|
||
|
||
Unreviewed, rolling out r221384.
|
||
|
||
This patch caused multiple 32-bit JSC test failures.
|
||
|
||
Reverted changeset:
|
||
|
||
"Strings need to be in some kind of gigacage"
|
||
https://bugs.webkit.org/show_bug.cgi?id=174924
|
||
http://trac.webkit.org/changeset/221384
|
||
|
||
2017-08-22 Filip Pizlo <fpizlo@apple.com>
|
||
|
||
Strings need to be in some kind of gigacage
|
||
https://bugs.webkit.org/show_bug.cgi?id=174924
|
||
|
||
Reviewed by Oliver Hunt.
|
||
|
||
This adds a StringGigacage.
|
||
|
||
* bmalloc/Gigacage.cpp:
|
||
* bmalloc/Gigacage.h:
|
||
(Gigacage::name):
|
||
(Gigacage::basePtr):
|
||
(Gigacage::forEachKind):
|
||
* bmalloc/HeapKind.h:
|
||
(bmalloc::isGigacage):
|
||
(bmalloc::gigacageKind):
|
||
(bmalloc::heapKind):
|
||
|
||
2017-08-25 Daniel Bates <dabates@apple.com>
|
||
|
||
Demarcate code added due to lack of NSDMI for aggregates
|
||
https://bugs.webkit.org/show_bug.cgi?id=175990
|
||
|
||
Reviewed by Andy Estes.
|
||
|
||
* bmalloc/BPlatform.h:
|
||
* bmalloc/List.h: Be explicit when initializing m_node to improve readability.
|
||
(bmalloc::ListNode::ListNode):
|
||
|
||
2017-08-23 Filip Pizlo <fpizlo@apple.com>
|
||
|
||
Reduce Gigacage sizes
|
||
https://bugs.webkit.org/show_bug.cgi?id=175920
|
||
|
||
Reviewed by Mark Lam.
|
||
|
||
This introduces the ability to have different gigacage sizes for different gigacages, and uses it to reduce the size of both
|
||
gigacages, but to different extents: Primitive gets 32GB with a 16GB runway and JSValue gets 16GB.
|
||
|
||
This is a ~10% membuster progression on my Mac Pro.
|
||
|
||
* bmalloc/Gigacage.cpp:
|
||
(Gigacage::ensureGigacage):
|
||
* bmalloc/Gigacage.h:
|
||
(Gigacage::size):
|
||
(Gigacage::alignment):
|
||
(Gigacage::mask):
|
||
(Gigacage::runway):
|
||
(Gigacage::totalSize):
|
||
(Gigacage::caged):
|
||
* bmalloc/Heap.cpp:
|
||
(bmalloc::Heap::Heap):
|
||
(bmalloc::Heap::gigacageSize):
|
||
* bmalloc/Heap.h:
|
||
|
||
2017-08-08 Filip Pizlo <fpizlo@apple.com>
|
||
|
||
Baseline JIT should do caging
|
||
https://bugs.webkit.org/show_bug.cgi?id=175037
|
||
|
||
Reviewed by Mark Lam.
|
||
|
||
This centralizes the notion of permanently enabling the primitive gigacage, which we only do in jsc
|
||
and WebProcess.
|
||
|
||
This saves the baseline JIT from emitting some code. Otherwise it would always have to emit enabled
|
||
checks on each typed array access.
|
||
|
||
* bmalloc/Gigacage.cpp:
|
||
(Gigacage::primitiveGigacageDisabled):
|
||
(Gigacage::disableDisablingPrimitiveGigacageIfShouldBeEnabled):
|
||
(Gigacage::isDisablingPrimitiveGigacageDisabled):
|
||
* bmalloc/Gigacage.h:
|
||
(Gigacage::isPrimitiveGigacagePermanentlyEnabled):
|
||
(Gigacage::canPrimitiveGigacageBeDisabled):
|
||
|
||
2017-08-08 Ryan Haddad <ryanhaddad@apple.com>
|
||
|
||
Unreviewed, rolling out r220368.
|
||
|
||
This change caused WK1 tests to exit early with crashes.
|
||
|
||
Reverted changeset:
|
||
|
||
"Baseline JIT should do caging"
|
||
https://bugs.webkit.org/show_bug.cgi?id=175037
|
||
http://trac.webkit.org/changeset/220368
|
||
|
||
2017-08-07 Filip Pizlo <fpizlo@apple.com>
|
||
|
||
Baseline JIT should do caging
|
||
https://bugs.webkit.org/show_bug.cgi?id=175037
|
||
|
||
Reviewed by Mark Lam.
|
||
|
||
This centralizes the notion of permanently enabling the primitive gigacage, which we only do in jsc
|
||
and WebProcess.
|
||
|
||
This saves the baseline JIT from emitting some code. Otherwise it would always have to emit enabled
|
||
checks on each typed array access.
|
||
|
||
* bmalloc/Gigacage.cpp:
|
||
(Gigacage::primitiveGigacageDisabled):
|
||
(Gigacage::disableDisablingPrimitiveGigacageIfShouldBeEnabled):
|
||
(Gigacage::isDisablingPrimitiveGigacageDisabled):
|
||
* bmalloc/Gigacage.h:
|
||
(Gigacage::isPrimitiveGigacagePermanentlyEnabled):
|
||
(Gigacage::canPrimitiveGigacageBeDisabled):
|
||
|
||
2017-08-06 Filip Pizlo <fpizlo@apple.com>
|
||
|
||
Primitive auxiliaries and JSValue auxiliaries should have separate gigacages
|
||
https://bugs.webkit.org/show_bug.cgi?id=174919
|
||
|
||
Reviewed by Keith Miller.
|
||
|
||
This introduces two kinds of Gigacage, Primitive and JSValue. This translates to two kinds of
|
||
HeapKind, PrimitiveGigacage and JSValueGigacage.
|
||
|
||
The new support functionality required turning Inline.h into BInline.h, and INLINE into BINLINE, and
|
||
NO_INLINE into BNO_INLINE.
|
||
|
||
* bmalloc.xcodeproj/project.pbxproj:
|
||
* bmalloc/Allocator.cpp:
|
||
(bmalloc::Allocator::refillAllocatorSlowCase):
|
||
(bmalloc::Allocator::refillAllocator):
|
||
(bmalloc::Allocator::allocateLarge):
|
||
(bmalloc::Allocator::allocateLogSizeClass):
|
||
* bmalloc/AsyncTask.h:
|
||
* bmalloc/BInline.h: Copied from Source/bmalloc/bmalloc/Inline.h.
|
||
* bmalloc/Cache.cpp:
|
||
(bmalloc::Cache::tryAllocateSlowCaseNullCache):
|
||
(bmalloc::Cache::allocateSlowCaseNullCache):
|
||
(bmalloc::Cache::deallocateSlowCaseNullCache):
|
||
(bmalloc::Cache::reallocateSlowCaseNullCache):
|
||
* bmalloc/Deallocator.cpp:
|
||
* bmalloc/Gigacage.cpp:
|
||
(Gigacage::PrimitiveDisableCallbacks::PrimitiveDisableCallbacks):
|
||
(Gigacage::ensureGigacage):
|
||
(Gigacage::disablePrimitiveGigacage):
|
||
(Gigacage::addPrimitiveDisableCallback):
|
||
(Gigacage::removePrimitiveDisableCallback):
|
||
(Gigacage::Callbacks::Callbacks): Deleted.
|
||
(Gigacage::disableGigacage): Deleted.
|
||
(Gigacage::addDisableCallback): Deleted.
|
||
(Gigacage::removeDisableCallback): Deleted.
|
||
* bmalloc/Gigacage.h:
|
||
(Gigacage::name):
|
||
(Gigacage::basePtr):
|
||
(Gigacage::forEachKind):
|
||
(Gigacage::caged):
|
||
(Gigacage::isCaged):
|
||
* bmalloc/Heap.cpp:
|
||
(bmalloc::Heap::Heap):
|
||
(bmalloc::Heap::usingGigacage):
|
||
(bmalloc::Heap::gigacageBasePtr):
|
||
* bmalloc/Heap.h:
|
||
* bmalloc/HeapKind.h:
|
||
(bmalloc::isGigacage):
|
||
(bmalloc::gigacageKind):
|
||
(bmalloc::heapKind):
|
||
* bmalloc/Inline.h: Removed.
|
||
* bmalloc/Map.h:
|
||
* bmalloc/PerProcess.h:
|
||
(bmalloc::PerProcess<T>::getFastCase):
|
||
(bmalloc::PerProcess<T>::get):
|
||
(bmalloc::PerProcess<T>::getSlowCase):
|
||
* bmalloc/PerThread.h:
|
||
(bmalloc::PerThread<T>::getFastCase):
|
||
* bmalloc/Vector.h:
|
||
(bmalloc::Vector<T>::push):
|
||
(bmalloc::Vector<T>::shrinkCapacity):
|
||
(bmalloc::Vector<T>::growCapacity):
|
||
|
||
2017-08-02 Filip Pizlo <fpizlo@apple.com>
|
||
|
||
If Gigacage is disabled, bmalloc should service large aligned memory allocation requests through vmAllocate
|
||
https://bugs.webkit.org/show_bug.cgi?id=175085
|
||
|
||
Reviewed by Saam Barati.
|
||
|
||
This fixes a problem where if we used gmalloc, WebAssembly memory allocations would still use
|
||
bmalloc's large allocator.
|
||
|
||
We want to use the page allocator for those "large" allocations when the Gigacage is disabled.
|
||
|
||
* bmalloc/DebugHeap.cpp:
|
||
(bmalloc::DebugHeap::DebugHeap):
|
||
(bmalloc::DebugHeap::memalignLarge):
|
||
(bmalloc::DebugHeap::freeLarge):
|
||
* bmalloc/DebugHeap.h:
|
||
* bmalloc/Heap.cpp:
|
||
(bmalloc::Heap::tryAllocateLarge):
|
||
(bmalloc::Heap::deallocateLarge):
|
||
|
||
2017-08-02 Filip Pizlo <fpizlo@apple.com>
|
||
|
||
We should be OK with the gigacage being disabled on gmalloc
|
||
https://bugs.webkit.org/show_bug.cgi?id=175082
|
||
|
||
Reviewed by Michael Saboff.
|
||
|
||
This adds Gigacage::shouldBeEnabled(), which returns false when we're using gmalloc or other things
|
||
that enable DebugHeap.
|
||
|
||
* bmalloc/Environment.cpp:
|
||
(bmalloc::Environment::Environment):
|
||
* bmalloc/Environment.h:
|
||
* bmalloc/Gigacage.cpp:
|
||
(Gigacage::ensureGigacage):
|
||
(Gigacage::shouldBeEnabled):
|
||
* bmalloc/Gigacage.h:
|
||
* bmalloc/Heap.cpp:
|
||
(bmalloc::Heap::Heap):
|
||
* bmalloc/Heap.h:
|
||
|
||
2017-08-01 Filip Pizlo <fpizlo@apple.com>
|
||
|
||
Bmalloc and GC should put auxiliaries (butterflies, typed array backing stores) in a gigacage (separate multi-GB VM region)
|
||
https://bugs.webkit.org/show_bug.cgi?id=174727
|
||
|
||
Reviewed by Mark Lam.
|
||
|
||
This adds a mechanism for managing multiple isolated heaps in bmalloc. For now, these isoheaps
|
||
(isolated heaps) have a very simple relationship with each other and with the rest of bmalloc:
|
||
|
||
- You have to choose how many isoheaps you will have statically. See numHeaps in HeapKind.h.
|
||
|
||
- Because numHeaps is static, each isoheap gets fast thread-local allocation. Basically, we have a
|
||
Cache for each heap kind.
|
||
|
||
- Each isoheap gets its own Heap.
|
||
|
||
- Each Heap gets a scavenger thread.
|
||
|
||
- Some things, like Zone/VMHeap/Scavenger, are per-process.
|
||
|
||
Most of the per-HeapKind functionality is handled by PerHeapKind<>.
|
||
|
||
This approach is ideal for supporting special per-HeapKind behaviors. For now we have two heaps:
|
||
the Primary heap for normal malloc and the Gigacage. The gigacage is a 64GB-aligned 64GB virtual
|
||
region that we now use for variable-length random-access allocations. No Primary allocations will
|
||
go into the Gigacage.
|
||
|
||
* CMakeLists.txt:
|
||
* bmalloc.xcodeproj/project.pbxproj:
|
||
* bmalloc/AllocationKind.h: Added.
|
||
* bmalloc/Allocator.cpp:
|
||
(bmalloc::Allocator::Allocator):
|
||
(bmalloc::Allocator::tryAllocate):
|
||
(bmalloc::Allocator::allocateImpl):
|
||
(bmalloc::Allocator::reallocate):
|
||
(bmalloc::Allocator::refillAllocatorSlowCase):
|
||
(bmalloc::Allocator::allocateLarge):
|
||
* bmalloc/Allocator.h:
|
||
* bmalloc/BExport.h: Added.
|
||
* bmalloc/Cache.cpp:
|
||
(bmalloc::Cache::scavenge):
|
||
(bmalloc::Cache::Cache):
|
||
(bmalloc::Cache::tryAllocateSlowCaseNullCache):
|
||
(bmalloc::Cache::allocateSlowCaseNullCache):
|
||
(bmalloc::Cache::deallocateSlowCaseNullCache):
|
||
(bmalloc::Cache::reallocateSlowCaseNullCache):
|
||
(bmalloc::Cache::operator new): Deleted.
|
||
(bmalloc::Cache::operator delete): Deleted.
|
||
* bmalloc/Cache.h:
|
||
(bmalloc::Cache::tryAllocate):
|
||
(bmalloc::Cache::allocate):
|
||
(bmalloc::Cache::deallocate):
|
||
(bmalloc::Cache::reallocate):
|
||
* bmalloc/Deallocator.cpp:
|
||
(bmalloc::Deallocator::Deallocator):
|
||
(bmalloc::Deallocator::scavenge):
|
||
(bmalloc::Deallocator::processObjectLog):
|
||
(bmalloc::Deallocator::deallocateSlowCase):
|
||
* bmalloc/Deallocator.h:
|
||
* bmalloc/Gigacage.cpp: Added.
|
||
(Gigacage::Callback::Callback):
|
||
(Gigacage::Callback::function):
|
||
(Gigacage::Callbacks::Callbacks):
|
||
(Gigacage::ensureGigacage):
|
||
(Gigacage::disableGigacage):
|
||
(Gigacage::addDisableCallback):
|
||
(Gigacage::removeDisableCallback):
|
||
* bmalloc/Gigacage.h: Added.
|
||
(Gigacage::caged):
|
||
(Gigacage::isCaged):
|
||
* bmalloc/Heap.cpp:
|
||
(bmalloc::Heap::Heap):
|
||
(bmalloc::Heap::usingGigacage):
|
||
(bmalloc::Heap::concurrentScavenge):
|
||
(bmalloc::Heap::splitAndAllocate):
|
||
(bmalloc::Heap::tryAllocateLarge):
|
||
(bmalloc::Heap::allocateLarge):
|
||
(bmalloc::Heap::shrinkLarge):
|
||
(bmalloc::Heap::deallocateLarge):
|
||
* bmalloc/Heap.h:
|
||
(bmalloc::Heap::mutex):
|
||
(bmalloc::Heap::kind const):
|
||
(bmalloc::Heap::setScavengerThreadQOSClass): Deleted.
|
||
* bmalloc/HeapKind.h: Added.
|
||
* bmalloc/ObjectType.cpp:
|
||
(bmalloc::objectType):
|
||
* bmalloc/ObjectType.h:
|
||
* bmalloc/PerHeapKind.h: Added.
|
||
(bmalloc::PerHeapKindBase::PerHeapKindBase):
|
||
(bmalloc::PerHeapKindBase::size):
|
||
(bmalloc::PerHeapKindBase::at):
|
||
(bmalloc::PerHeapKindBase::at const):
|
||
(bmalloc::PerHeapKindBase::operator[]):
|
||
(bmalloc::PerHeapKindBase::operator[] const):
|
||
(bmalloc::StaticPerHeapKind::StaticPerHeapKind):
|
||
(bmalloc::PerHeapKind::PerHeapKind):
|
||
(bmalloc::PerHeapKind::~PerHeapKind):
|
||
* bmalloc/PerThread.h:
|
||
(bmalloc::PerThread<T>::destructor):
|
||
(bmalloc::PerThread<T>::getSlowCase):
|
||
(bmalloc::PerThreadStorage<Cache>::get): Deleted.
|
||
(bmalloc::PerThreadStorage<Cache>::init): Deleted.
|
||
* bmalloc/Scavenger.cpp: Added.
|
||
(bmalloc::Scavenger::Scavenger):
|
||
(bmalloc::Scavenger::scavenge):
|
||
* bmalloc/Scavenger.h: Added.
|
||
(bmalloc::Scavenger::setScavengerThreadQOSClass):
|
||
(bmalloc::Scavenger::requestedScavengerThreadQOSClass const):
|
||
* bmalloc/VMHeap.cpp:
|
||
(bmalloc::VMHeap::VMHeap):
|
||
(bmalloc::VMHeap::tryAllocateLargeChunk):
|
||
* bmalloc/VMHeap.h:
|
||
* bmalloc/Zone.cpp:
|
||
(bmalloc::Zone::Zone):
|
||
* bmalloc/Zone.h:
|
||
* bmalloc/bmalloc.h:
|
||
(bmalloc::api::tryMalloc):
|
||
(bmalloc::api::malloc):
|
||
(bmalloc::api::tryMemalign):
|
||
(bmalloc::api::memalign):
|
||
(bmalloc::api::realloc):
|
||
(bmalloc::api::tryLargeMemalignVirtual):
|
||
(bmalloc::api::free):
|
||
(bmalloc::api::freeLargeVirtual):
|
||
(bmalloc::api::scavengeThisThread):
|
||
(bmalloc::api::scavenge):
|
||
(bmalloc::api::isEnabled):
|
||
(bmalloc::api::setScavengerThreadQOSClass):
|
||
* bmalloc/mbmalloc.cpp:
|
||
|
||
2017-08-01 Daewoong Jang <daewoong.jang@navercorp.com>
|
||
|
||
Implement __builtin_clzl for MSVC
|
||
https://bugs.webkit.org/show_bug.cgi?id=174232
|
||
|
||
Reviewed by Geoffrey Garen.
|
||
|
||
* bmalloc/Algorithm.h:
|
||
(bmalloc::clzl):
|
||
(bmalloc::clzl<1>):
|
||
(bmalloc::__builtin_clzl):
|
||
* bmalloc/BPlatform.h:
|
||
|
||
2017-07-31 Mark Lam <mark.lam@apple.com>
|
||
|
||
Fixed some comment typos.
|
||
|
||
Not reviewed.
|
||
|
||
* bmalloc/PerProcess.h:
|
||
|
||
2017-07-14 Filip Pizlo <fpizlo@apple.com>
|
||
|
||
It should be easy to decide how WebKit yields
|
||
https://bugs.webkit.org/show_bug.cgi?id=174298
|
||
|
||
Reviewed by Saam Barati.
|
||
|
||
Use sched_yield() explicitly.
|
||
|
||
* bmalloc/StaticMutex.cpp:
|
||
(bmalloc::StaticMutex::lockSlowCase):
|
||
|
||
2017-07-20 Chris Dumez <cdumez@apple.com>
|
||
|
||
Replace calls to Vector::resize() with calls to more efficient shrink() / grow() when applicable
|
||
https://bugs.webkit.org/show_bug.cgi?id=174660
|
||
|
||
Reviewed by Geoffrey Garen.
|
||
|
||
Replace calls to Vector::resize() with calls to more efficient shrink() / grow() when applicable.
|
||
This essentially replaces a branch to figure out if the new size is less or greater than the
|
||
current size by an assertion.
|
||
|
||
* bmalloc/Map.h:
|
||
(bmalloc::Hash>::rehash):
|
||
|
||
2017-07-18 Andy Estes <aestes@apple.com>
|
||
|
||
[Xcode] Enable CLANG_WARN_RANGE_LOOP_ANALYSIS
|
||
https://bugs.webkit.org/show_bug.cgi?id=174631
|
||
|
||
Reviewed by Tim Horton.
|
||
|
||
* Configurations/Base.xcconfig:
|
||
|
||
2017-07-18 Andy Estes <aestes@apple.com>
|
||
|
||
[Xcode] Enable CLANG_WARN_OBJC_LITERAL_CONVERSION
|
||
https://bugs.webkit.org/show_bug.cgi?id=174631
|
||
|
||
Reviewed by Sam Weinig.
|
||
|
||
* Configurations/Base.xcconfig:
|
||
|
||
2017-07-18 Andy Estes <aestes@apple.com>
|
||
|
||
[Xcode] Enable CLANG_WARN_NON_LITERAL_NULL_CONVERSION
|
||
https://bugs.webkit.org/show_bug.cgi?id=174631
|
||
|
||
Reviewed by Dan Bernstein.
|
||
|
||
* Configurations/Base.xcconfig:
|
||
|
||
2017-07-18 Andy Estes <aestes@apple.com>
|
||
|
||
[Xcode] Enable CLANG_WARN_BLOCK_CAPTURE_AUTORELEASING
|
||
https://bugs.webkit.org/show_bug.cgi?id=174631
|
||
|
||
Reviewed by Darin Adler.
|
||
|
||
* Configurations/Base.xcconfig:
|
||
|
||
2017-07-12 Adrian Perez de Castro <aperez@igalia.com>
|
||
|
||
bmalloc: Failure to build when the compiler specifically targets ARMv8-A / defines __ARM_ARCH_8A__
|
||
https://bugs.webkit.org/show_bug.cgi?id=174424
|
||
|
||
Reviewed by Michael Catanzaro.
|
||
|
||
* bmalloc/BPlatform.h: Also check for __ARCH_ARM_8A__ to detect ARMv8.
|
||
|
||
2017-07-05 Daewoong Jang <daewoong.jang@navercorp.com>
|
||
|
||
reinterpret_cast does not evaluate to constexpr
|
||
https://bugs.webkit.org/show_bug.cgi?id=173622
|
||
|
||
Reviewed by Yusuke Suzuki.
|
||
|
||
* bmalloc/Algorithm.h:
|
||
(bmalloc::mask):
|
||
(bmalloc::roundUpToMultipleOf):
|
||
|
||
2017-07-03 Andy Estes <aestes@apple.com>
|
||
|
||
[Xcode] Add an experimental setting to build with ccache
|
||
https://bugs.webkit.org/show_bug.cgi?id=173875
|
||
|
||
Reviewed by Tim Horton.
|
||
|
||
* Configurations/DebugRelease.xcconfig: Included ccache.xcconfig.
|
||
|
||
2017-07-01 Dan Bernstein <mitz@apple.com>
|
||
|
||
[iOS] Remove code only needed when building for iOS 9.x
|
||
https://bugs.webkit.org/show_bug.cgi?id=174068
|
||
|
||
Reviewed by Tim Horton.
|
||
|
||
* bmalloc/BPlatform.h:
|
||
* bmalloc/VMAllocate.h:
|
||
(bmalloc::vmPageSizePhysical):
|
||
|
||
2017-07-01 Dan Bernstein <mitz@apple.com>
|
||
|
||
[macOS] Remove code only needed when building for OS X Yosemite
|
||
https://bugs.webkit.org/show_bug.cgi?id=174067
|
||
|
||
Reviewed by Tim Horton.
|
||
|
||
* Configurations/Base.xcconfig:
|
||
* Configurations/DebugRelease.xcconfig:
|
||
|
||
2017-06-30 Ryosuke Niwa <rniwa@webkit.org>
|
||
|
||
Ran sort-Xcode-project-file.
|
||
|
||
* bmalloc.xcodeproj/project.pbxproj:
|
||
|
||
2017-06-19 Geoffrey Garen <ggaren@apple.com>
|
||
|
||
bmalloc: Add a per-thread line cache
|
||
https://bugs.webkit.org/show_bug.cgi?id=173552
|
||
|
||
Reviewed by Darin Adler.
|
||
|
||
Previously, any thread could allocate out of any page with free lines.
|
||
Now, the first thread to free a line in a page owns that page's free
|
||
lines until the whole page becomes free.
|
||
|
||
This patch is a big speedup on multi-threaded benchmarks.
|
||
tree_churn --parallel gets 14% faster on a 2-core (4-hyper-core) MacBook
|
||
Air and 2.85X faster on 12-core (24-hyper-core) Mac Pro. Other parallel
|
||
benchmarks show significant but smaller speedups.
|
||
|
||
Thread affinity is a great predictor of object lifetime. The per-thread
|
||
line cache avoids the pathology of shuffling pages between threads,
|
||
turning predictable lifetimes into unpredictable lifetimes, increasing
|
||
fragmentation. On tree_churn --parallel, the per-thread line cache
|
||
increases free memory found per page scanned by 2.85X.
|
||
|
||
Free line scanning in fragmented pages is pretty expensive relative to
|
||
other allocate / initialize / free operations. According to Instruments,
|
||
on tree_churn --parallel, scanning is about 10X more expensive than
|
||
freeing. This explains why a 2.85X improvement in scanning efficiency
|
||
translates into a 2.85X overall speedup on tree_churn --parallel.
|
||
|
||
* bmalloc/Allocator.cpp:
|
||
(bmalloc::Allocator::refillAllocatorSlowCase): Pass through our line
|
||
cache so the Heap can fill it.
|
||
|
||
* bmalloc/Deallocator.cpp:
|
||
(bmalloc::Deallocator::scavenge): Scavenge our line cache.
|
||
|
||
(bmalloc::Deallocator::processObjectLog): Deleted.
|
||
|
||
* bmalloc/Deallocator.h:
|
||
(bmalloc::Deallocator::lineCache): Added a line cache.
|
||
|
||
* bmalloc/Heap.cpp:
|
||
(bmalloc::Heap::deallocateLineCache): Deallocation function for thread
|
||
destruction.
|
||
|
||
(bmalloc::Heap::allocateSmallPage):
|
||
(bmalloc::Heap::deallocateSmallLine):
|
||
(bmalloc::Heap::allocateSmallBumpRangesByMetadata):
|
||
(bmalloc::Heap::allocateSmallBumpRangesByObject): Consult the new per-thread line
|
||
cache for allocation and deallocation.
|
||
|
||
* bmalloc/Heap.h:
|
||
(bmalloc::Heap::allocateSmallBumpRanges):
|
||
(bmalloc::Heap::derefSmallLine):
|
||
|
||
* bmalloc/List.h:
|
||
(bmalloc::List::remove): Remove has always been a logically static
|
||
operation. Declare it static now so that the Heap can remove a page from
|
||
a thread's line cache without holding a direct pointer to the cache.
|
||
|
||
* bmalloc/SmallPage.h:
|
||
|
||
2017-06-10 Dan Bernstein <mitz@apple.com>
|
||
|
||
Reverted r218056 because it made the IDE reindex constantly.
|
||
|
||
* Configurations/DebugRelease.xcconfig:
|
||
|
||
2017-06-10 Dan Bernstein <mitz@apple.com>
|
||
|
||
[Xcode] With Xcode 9 developer beta, everything rebuilds when switching between command-line and IDE
|
||
https://bugs.webkit.org/show_bug.cgi?id=173223
|
||
|
||
Reviewed by Sam Weinig.
|
||
|
||
The rebuilds were happening due to a difference in the compiler options that the IDE and
|
||
xcodebuild were specifying. Only the IDE was passing the -index-store-path option. To make
|
||
xcodebuild pass that option, too, set CLANG_INDEX_STORE_ENABLE to YES if it is unset, and
|
||
specify an appropriate path in CLANG_INDEX_STORE_PATH.
|
||
|
||
* Configurations/DebugRelease.xcconfig:
|
||
|
||
2017-06-07 Geoffrey Garen <ggaren@apple.com>
|
||
|
||
bmalloc: memory APIs don't need to be heap members
|
||
https://bugs.webkit.org/show_bug.cgi?id=173076
|
||
|
||
Reviewed by Sam Weinig.
|
||
|
||
Asking the OS about memory use is unrelated to the state of bmalloc's
|
||
heap, so it's a better separation of concerns if related code is not
|
||
part of the heap.
|
||
|
||
* bmalloc/AvailableMemory.cpp:
|
||
(bmalloc::memoryStatus):
|
||
* bmalloc/AvailableMemory.h:
|
||
(bmalloc::MemoryStatus::MemoryStatus):
|
||
(bmalloc::isUnderMemoryPressure):
|
||
(bmalloc::memoryFootprint):
|
||
(bmalloc::percentAvailableMemoryInUse):
|
||
* bmalloc/Heap.cpp:
|
||
(bmalloc::Heap::Heap):
|
||
(bmalloc::Heap::updateMemoryInUseParameters): Deleted.
|
||
* bmalloc/Heap.h:
|
||
(bmalloc::Heap::isUnderMemoryPressure): Deleted.
|
||
(bmalloc::Heap::memoryFootprint): Deleted.
|
||
(bmalloc::Heap::percentAvailableMemoryInUse): Deleted.
|
||
|
||
2017-06-06 Yusuke Suzuki <utatane.tea@gmail.com>
|
||
|
||
struct does not accept initializer-form if member has initializers in GCC 4.9
|
||
https://bugs.webkit.org/show_bug.cgi?id=172974
|
||
|
||
Reviewed by Carlos Garcia Campos.
|
||
|
||
struct cannot accept initializer-form constructor (like, `ListNode<T> t { ... }`) if
|
||
the member of the struct has a default initializer.
|
||
Here is a simple snippet.
|
||
|
||
template<typename T>
|
||
struct Pair {
|
||
T* prev { nullptr };
|
||
T* next { nullptr };
|
||
};
|
||
|
||
Pair<int> pair { nullptr, nullptr }; // compile erorr in GCC 4.9.
|
||
|
||
Instead, we define a default constructor (to invoke default initializers) and a constructor
|
||
to accept the above initialization.
|
||
|
||
* bmalloc/List.h:
|
||
(bmalloc::ListNode::ListNode):
|
||
(bmalloc::List::iterator::iterator):
|
||
|
||
2017-06-06 Geoffrey Garen <ggaren@apple.com>
|
||
|
||
Try to fix the GTK build.
|
||
|
||
Unreviewed.
|
||
|
||
* bmalloc/List.h:
|
||
(bmalloc::List::List):
|
||
|
||
2017-06-05 Geoffrey Garen <ggaren@apple.com>
|
||
|
||
Try to fix the GTK build.
|
||
|
||
Unreviewed.
|
||
|
||
* bmalloc/List.h:
|
||
|
||
2017-06-02 Geoffrey Garen <ggaren@apple.com>
|
||
|
||
bmalloc: Small and large objects should share memory
|
||
https://bugs.webkit.org/show_bug.cgi?id=172880
|
||
<rdar://problem/31494732>
|
||
|
||
Reviewed by Sam Weinig.
|
||
|
||
This reduces our high water mark memory usage on JetStream on macOS
|
||
by 10%-20%. It also has the nice side effect that we can free small
|
||
object metadata after returning from a high water mark.
|
||
|
||
No change in throughput.
|
||
|
||
Our old algorithm allocated small object chunks and large objects in
|
||
segregated virtual memory and never recycled addresses between them.
|
||
This provided a slight security benefit because we could apply guard
|
||
pages between the segregated ranges and we would never reuse the same
|
||
virtual address for object and metadata memory.
|
||
|
||
Our new algorithm allocates small object chunks from the large object
|
||
allocator. This naturally recycles memory between small chunks and large
|
||
objects, and between small chunks of different page classes. This allows
|
||
us to shift memory between allocation types as a program moves between
|
||
different phases of allocation, and to delete small object chunk metadata
|
||
when a program shrinks back from a high water mark.
|
||
|
||
Two intuitions I had about memory use turned out to be backwards in
|
||
this context:
|
||
|
||
(1) I thought that this optimization would work because it allowed you to
|
||
allocate and free a 4MB object and then reuse that large allocation to
|
||
service small allocations. In practice, the common benefit seems to be
|
||
the opposite: After you allocate and free many small objects, you can
|
||
stitch them together to allocate a large object without growing the heap.
|
||
|
||
(2) I thought that it would be more memory-efficient to allocate
|
||
fine-grained pages from the large object allocator. In practice, giving
|
||
the large object allocator too many arbitrarily-sized ranges to manage
|
||
leads to fragmentation. Meanwhile, segregated fit is a powerful memory
|
||
optimization. So, it's best to return small object memory to the large
|
||
allocator only when a whole small object chunk is free.
|
||
|
||
* bmalloc/Chunk.h:
|
||
(bmalloc::Chunk::ref):
|
||
(bmalloc::Chunk::deref):
|
||
(bmalloc::Chunk::refCount):
|
||
(bmalloc::Chunk::freePages): We keep a free list per chunk and refcount
|
||
each chunk so we can notice when a chunk becomes empty, and return it
|
||
to the large allocator.
|
||
|
||
(bmalloc::forEachPage): A new helper function for iterating the pages
|
||
in a Chunk.
|
||
|
||
(bmalloc::Chunk::Chunk): Use forEachPage instead of manual iteration.
|
||
Use { } initialization because we don't get zero-initialized by the OS
|
||
anymore.
|
||
|
||
* bmalloc/Heap.cpp:
|
||
(bmalloc::Heap::Heap):
|
||
(bmalloc::Heap::concurrentScavenge):
|
||
(bmalloc::Heap::scavenge): Don't bother unlocking while scavenging. I
|
||
wasn't able to show it to be a consistent speedup. A more promising
|
||
approach, if we find a motivating example, is for the scavenger to give
|
||
up and return early if any other client is waiting on the lock.
|
||
|
||
(bmalloc::Heap::allocateSmallChunk): New helper function for allocating
|
||
a small chunk. It allocates through the large allocator to facilitate
|
||
sharing. We still allocate a chunk at a time instead of a page at a time.
|
||
Surprisingly, more precise page-at-a-time allocation is worse for memory
|
||
use because of fragmentation. Segregated fit is a powerful optimization.
|
||
|
||
(bmalloc::Heap::deallocateSmallChunk): New helper function for deallocating
|
||
a small chunk.
|
||
|
||
(bmalloc::Heap::allocateSmallPage): Updated for new APIs.
|
||
|
||
(bmalloc::Heap::deallocateSmallLine): Updated for new APIs. Note that
|
||
we cache one free chunk per page class. This avoids churn in the large
|
||
allocator when you free(malloc(X)).
|
||
|
||
(bmalloc::Heap::allocateSmallBumpRangesByMetadata):
|
||
(bmalloc::Heap::allocateSmallBumpRangesByObject):
|
||
(bmalloc::Heap::tryAllocateLarge):
|
||
(bmalloc::Heap::scavengeSmallPages): Deleted.
|
||
(bmalloc::Heap::scavengeLargeObjects): Deleted.
|
||
* bmalloc/Heap.h:
|
||
|
||
* bmalloc/LargeMap.h:
|
||
(bmalloc::LargeMap::begin):
|
||
(bmalloc::LargeMap::end): Added iteration helpers for scavenging.
|
||
|
||
* bmalloc/LargeRange.h:
|
||
(bmalloc::LargeRange::physicalSize): Added a comment about something
|
||
that I confused myself about in this patch.
|
||
|
||
* bmalloc/List.h:
|
||
(bmalloc::List::iterator::operator*):
|
||
(bmalloc::List::iterator::operator->):
|
||
(bmalloc::List::iterator::operator!=):
|
||
(bmalloc::List::iterator::operator++):
|
||
(bmalloc::List::begin):
|
||
(bmalloc::List::end):
|
||
(bmalloc::List::pushFront):
|
||
(bmalloc::List::remove):
|
||
(bmalloc::ListNode::ListNode): Deleted. Added iteration helpers for
|
||
scavenging. Changed the default state of a Node to null pointers instead
|
||
of self pointers to distinguish the null node from the empty node for
|
||
easier debugging.
|
||
|
||
* bmalloc/Sizes.h: Changed the chunk size to 1MB to increase the chances
|
||
of a chunk becoming free and recyclable.
|
||
|
||
* bmalloc/SmallPage.h:
|
||
(bmalloc::SmallPage::hasPhysicalPages):
|
||
(bmalloc::SmallPage::setHasPhysicalPages): Track physical state by page
|
||
instead of implicitly by which list a page is in. It's simpler not
|
||
to have to move chunks and pages between physical vs virtual lists.
|
||
|
||
(bmalloc::SmallPage::SmallPage): Deleted.
|
||
|
||
* bmalloc/VMHeap.cpp:
|
||
(bmalloc::VMHeap::tryAllocateLargeChunk):
|
||
(bmalloc::VMHeap::allocateSmallChunk): Deleted.
|
||
* bmalloc/VMHeap.h:
|
||
(bmalloc::VMHeap::allocateSmallPage): Deleted.
|
||
(bmalloc::VMHeap::deallocateSmallPage): Deleted. Small chunk allocation
|
||
just forwards to the large allocator now.
|
||
|
||
* bmalloc/bmalloc.h:
|
||
(bmalloc::api::scavenge):
|
||
|
||
2017-05-28 Dan Bernstein <mitz@apple.com>
|
||
|
||
[Xcode] ALWAYS_SEARCH_USER_PATHS is set to YES
|
||
https://bugs.webkit.org/show_bug.cgi?id=172691
|
||
|
||
Reviewed by Tim Horton.
|
||
|
||
* Configurations/Base.xcconfig: Set ALWAYS_SEARCH_USER_PATHS to NO.
|
||
|
||
2017-05-25 Geoffrey Garen <ggaren@apple.com> and Michael Saboff <msaboff@apple.com>
|
||
|
||
bmalloc: scavenger runs too much on JetStream
|
||
https://bugs.webkit.org/show_bug.cgi?id=172373
|
||
|
||
Reviewed by Geoffrey Garen.
|
||
|
||
Instruments says that JetStream on macOS spends about 3% of its time in
|
||
madvise.
|
||
|
||
In <https://bugs.webkit.org/show_bug.cgi?id=160098>, Ben saw some
|
||
evidence that madvise was the reason that switching to bmalloc for
|
||
DFG::Node allocations was a slowdown the first time around.
|
||
|
||
In <https://bugs.webkit.org/show_bug.cgi?id=172124>, Michael saw that
|
||
scavening policy can affect JetStream.
|
||
|
||
Intuitively, it seems wrong for the heap to idle shrink during hardcore
|
||
benchmarking.
|
||
|
||
The strategy here is to back off in response to any heap growth event,
|
||
and to wait 2s instead of 0.5s for heap growth to take place -- but we
|
||
scavenge immediately in response to critical memory pressure, to avoid
|
||
jetsam.
|
||
|
||
One hole in this strategy is that a workload with a perfectly
|
||
unfragmented heap that allocates and deallocates ~16kB every 2s will
|
||
never shrink its heap. This doesn't seem to be a problem in practice.
|
||
|
||
This looks like a 2% - 4% speedup on JetStream on Mac Pro and MacBook Air.
|
||
|
||
* bmalloc/AsyncTask.h:
|
||
(bmalloc::AsyncTask::willRun):
|
||
(bmalloc::AsyncTask::willRunSoon):
|
||
(bmalloc::Function>::AsyncTask):
|
||
(bmalloc::Function>::run):
|
||
(bmalloc::Function>::runSoon):
|
||
(bmalloc::Function>::threadRunLoop):
|
||
(bmalloc::Function>::runSlowCase): Deleted. Added a "run soon" state
|
||
so that execution delay is modeled directly instead of implicitly
|
||
through sleep events. This enables the Heap to issue a "run now" event
|
||
at any moment in response ot memory pressure.
|
||
|
||
* bmalloc/Heap.cpp:
|
||
(bmalloc::Heap::Heap): Don't call into our own API -- that's a layering
|
||
violation.
|
||
|
||
(bmalloc::Heap::updateMemoryInUseParameters): No need for
|
||
m_scavengeSleepDuration anymore.
|
||
|
||
(bmalloc::Heap::concurrentScavenge): Added a back-off policy when the
|
||
heap is growing.
|
||
(bmalloc::Heap::scavenge):
|
||
|
||
(bmalloc::Heap::scavengeSmallPages):
|
||
(bmalloc::Heap::scavengeLargeObjects): Don't try to give up in the middle
|
||
of a scavenge event. Our new backoff policy supplants that design. Also,
|
||
it's easier to profile and understand scavenging behavior if it always
|
||
runs to completion once started.
|
||
|
||
(bmalloc::Heap::scheduleScavenger):
|
||
(bmalloc::Heap::scheduleScavengerIfUnderMemoryPressure): Added a
|
||
synchronous amortized check for memory pressure. This check has the
|
||
benefit that it runs immediately during high rates of heap activity,
|
||
so we can detect memory pressure right away and wake the scavenger
|
||
instead of waiting for the scavenger to wake up.
|
||
|
||
(bmalloc::Heap::allocateSmallPage):
|
||
(bmalloc::Heap::deallocateSmallLine):
|
||
(bmalloc::Heap::splitAndAllocate):
|
||
(bmalloc::Heap::tryAllocateLarge):
|
||
(bmalloc::Heap::shrinkLarge):
|
||
(bmalloc::Heap::deallocateLarge):
|
||
* bmalloc/Heap.h:
|
||
(bmalloc::Heap::isUnderMemoryPressure):
|
||
* bmalloc/Sizes.h:
|
||
* bmalloc/VMHeap.h:
|
||
(bmalloc::VMHeap::deallocateSmallPage):
|
||
* bmalloc/bmalloc.h:
|
||
(bmalloc::api::scavenge): Updated for API changes above.
|
||
|
||
2017-05-17 Michael Saboff <msaboff@apple.com>
|
||
|
||
[iOS] The Garbage Collector shouldn't rely on the bmalloc scavenger for up to date memory footprint info (172186)
|
||
https://bugs.webkit.org/show_bug.cgi?id=172186
|
||
|
||
Reviewed by Geoffrey Garen.
|
||
|
||
The calls memoryFootprint() and percentAvailableMemoryInUse() now make a system call to get
|
||
the current memory footprint value.
|
||
|
||
* bmalloc/Heap.h:
|
||
(bmalloc::Heap::memoryFootprint):
|
||
(bmalloc::Heap::percentAvailableMemoryInUse):
|
||
|
||
2017-05-16 Michael Saboff <msaboff@apple.com>
|
||
|
||
REGRESSION(r216763): JetStream is 1% slower on Mac
|
||
https://bugs.webkit.org/show_bug.cgi?id=172124
|
||
|
||
Reviewed by Filip Pizlo.
|
||
|
||
It appears that changing maxScavengeSleepDuration from 512 to 250ms in r216763 is
|
||
responsible for the regression.
|
||
|
||
* bmalloc/Sizes.h:
|
||
|
||
2017-05-15 Geoffrey Garen <ggaren@apple.com>
|
||
|
||
bmalloc: Bump the size of the deallocator log to 512
|
||
https://bugs.webkit.org/show_bug.cgi?id=172143
|
||
|
||
Reviewed by Michael Saboff.
|
||
|
||
This is a speedup on parallel workloads for machines with lots of CPUs.
|
||
|
||
* bmalloc/Sizes.h:
|
||
|
||
2017-05-12 Michael Saboff <msaboff@apple.com>
|
||
|
||
[iOS] Use memory footprint to dynamically adjust behavior of allocators
|
||
https://bugs.webkit.org/show_bug.cgi?id=171944
|
||
|
||
Reviewed by Filip Pizlo.
|
||
|
||
This change is iOS only.
|
||
|
||
After the scavenger thread completes scavenging, it asks the OS for how much total memory the
|
||
process is using. This information is used to update the sleep delay for the scanvenger thread,
|
||
as well as to provide memory in use data for other parts of the system.
|
||
|
||
The scavenger sleep time is calculated using the following quadradic equation.
|
||
|
||
scavengerSleep = 1.2*percentFreeMemory^2 - percentFreeMemory + 2
|
||
|
||
Where percentFreeMemory is between 0 and 100. The result is constrained to the values 2 and 250.
|
||
|
||
This equation empirically works out to providing a 2ms sleep time when we have less than 10%
|
||
memory available, 30ms when 20% is available and 250ms when 50% or more is available. In testing,
|
||
this exponentially agressive scavenging delay by itself reduced memory usage and made it much
|
||
more deterministic when used without the corresponding change in the JSC Heap.
|
||
|
||
Changed the scavenger thread to use the User Initiated QOS priority to ensure it doesn't
|
||
get starved.
|
||
|
||
Moved the non-Windows functionality of WTF::RAMSize() to new files AvailableMemory.{cpp,h}
|
||
and implemented in the function availableMemory(). That functions limits the value returned
|
||
on iOS to a maximum of 840MB as that is the jetsam soft memory limit.
|
||
Added a new API availableMemory() so that WTF::RAMSize() will use this value.
|
||
|
||
* CMakeLists.txt:
|
||
* bmalloc.xcodeproj/project.pbxproj:
|
||
* bmalloc/BPlatform.h:
|
||
* bmalloc/Heap.cpp:
|
||
(bmalloc::Heap::Heap):
|
||
(bmalloc::Heap::updateMemoryInUseParameters):
|
||
(bmalloc::Heap::concurrentScavenge):
|
||
(bmalloc::Heap::scavenge):
|
||
* bmalloc/Heap.h:
|
||
(bmalloc::Heap::memoryFootprint):
|
||
(bmalloc::Heap::percentAvailableMemoryInUse):
|
||
* bmalloc/Sizes.h:
|
||
* bmalloc/bmalloc.h:
|
||
(bmalloc::api::availableMemory):
|
||
(bmalloc::api::memoryFootprint):
|
||
(bmalloc::api::percentAvailableMemoryInUse):
|
||
* bmalloc/AvailableMemory.cpp: Added.
|
||
(bmalloc::computeAvailableMemory):
|
||
(bmalloc::availableMemory):
|
||
* bmalloc/AvailableMemory.h: Added.
|
||
|
||
2017-05-05 Joseph Pecoraro <pecoraro@apple.com>
|
||
|
||
Leaks always reports "WebKit Malloc Memory Pressure Handler" dispatch_queue/source as leaking
|
||
https://bugs.webkit.org/show_bug.cgi?id=171532
|
||
|
||
Reviewed by Geoffrey Garen.
|
||
|
||
* bmalloc/Heap.cpp:
|
||
(bmalloc::Heap::Heap):
|
||
* bmalloc/Heap.h:
|
||
Store the dispatch_source_t in a member to avoid a false positive leak.
|
||
|
||
2017-04-27 Michael Saboff <msaboff@apple.com>
|
||
|
||
bmalloc scavenger should know what page classes are allocating
|
||
https://bugs.webkit.org/show_bug.cgi?id=171384
|
||
|
||
Reviewed by Geoffrey Garen.
|
||
|
||
This change replaces m_isAllocatingPages with a per page class flag to track which page
|
||
classes are currently allocating. When scavenging, we skip page classes that are actively
|
||
allocating and come back to them on a subsequent pass. This reduces the amount of time it
|
||
takes for scavenger to free up pages as well as the total time it takes to handle all
|
||
page classes.
|
||
|
||
* bmalloc/Heap.cpp:
|
||
(bmalloc::Heap::Heap):
|
||
(bmalloc::Heap::concurrentScavenge):
|
||
(bmalloc::Heap::scavenge):
|
||
(bmalloc::Heap::scavengeSmallPages):
|
||
(bmalloc::Heap::scavengeLargeObjects):
|
||
(bmalloc::Heap::allocateSmallPage):
|
||
(bmalloc::Heap::splitAndAllocate):
|
||
(bmalloc::Heap::deallocateLarge):
|
||
* bmalloc/Heap.h:
|
||
(bmalloc::Heap::takeRequestedScavengerThreadQOSClass): Deleted.
|
||
* bmalloc/VMHeap.h:
|
||
(bmalloc::VMHeap::deallocateSmallPage):
|
||
* bmalloc/bmalloc.h:
|
||
(bmalloc::api::scavenge):
|
||
|
||
2017-04-25 Michael Saboff <msaboff@apple.com>
|
||
|
||
Call bmalloc scavenger first when handling a memory pressure event
|
||
https://bugs.webkit.org/show_bug.cgi?id=171289
|
||
|
||
Reviewed by Geoffrey Garen.
|
||
|
||
Registered a critical memory pressure handler. We add this handler in addition to the
|
||
call to release bmalloc memory in the WebCore releaseMemory handler for the case of
|
||
JSC API users that don't use WebCore. When both handlers are in the process, it is
|
||
basically a race. One will win, but the loser won't do any more work, so it is harmless.
|
||
|
||
* bmalloc/Heap.cpp:
|
||
(bmalloc::Heap::Heap):
|
||
|
||
2017-04-14 Mark Lam <mark.lam@apple.com>
|
||
|
||
Update architectures in xcconfig files.
|
||
https://bugs.webkit.org/show_bug.cgi?id=170867
|
||
<rdar://problem/31628104>
|
||
|
||
Reviewed by Joseph Pecoraro.
|
||
|
||
* Configurations/Base.xcconfig:
|
||
|
||
2017-04-12 Dan Bernstein <mitz@apple.com>
|
||
|
||
[Mac] Future-proof .xcconfig files
|
||
https://bugs.webkit.org/show_bug.cgi?id=170802
|
||
|
||
Reviewed by Tim Horton.
|
||
|
||
* Configurations/Base.xcconfig:
|
||
* Configurations/DebugRelease.xcconfig:
|
||
|
||
2017-02-03 Ting-Wei Lan <lantw44@gmail.com>
|
||
|
||
Include cstdlib before using ::malloc and posix_memalign
|
||
https://bugs.webkit.org/show_bug.cgi?id=167800
|
||
|
||
Reviewed by Geoffrey Garen.
|
||
|
||
* bmalloc/DebugHeap.cpp:
|
||
|
||
2017-02-01 Andreas Kling <akling@apple.com>
|
||
|
||
Implement the alwaysRunsAtBackgroundPriority WK2 setting using thread QoS.
|
||
<https://webkit.org/b/167387>
|
||
<rdar://problem/29711409>
|
||
|
||
Reviewed by Antti Koivisto.
|
||
|
||
Support changing the QoS level of the scavenger thread asynchronously through
|
||
a request variable. This is not the most elegant thing in the world, but since
|
||
threads are only allowed to change their own QoS class, our options are limited.
|
||
|
||
* bmalloc/Heap.cpp:
|
||
(bmalloc::Heap::concurrentScavenge):
|
||
* bmalloc/Heap.h:
|
||
(bmalloc::Heap::takeRequestedScavengerThreadQOSClass):
|
||
(bmalloc::Heap::setScavengerThreadQOSClass):
|
||
* bmalloc/bmalloc.h:
|
||
(bmalloc::api::setScavengerThreadQOSClass):
|
||
|
||
2017-01-13 Geoffrey Garen <ggaren@apple.com>
|
||
|
||
bmalloc: Use a separate zone when using system malloc
|
||
https://bugs.webkit.org/show_bug.cgi?id=167014
|
||
|
||
Reviewed by Filip Pizlo.
|
||
|
||
Harris asked for this so he could separate Safari and WebKit memory use
|
||
when doing memory analysis.
|
||
|
||
This patch adds an explicit DebugHeap class that contains all our
|
||
code for specialized allocation with debugging.
|
||
|
||
* bmalloc.xcodeproj/project.pbxproj:
|
||
|
||
* bmalloc/Allocator.cpp:
|
||
(bmalloc::Allocator::Allocator):
|
||
(bmalloc::Allocator::tryAllocate):
|
||
(bmalloc::Allocator::allocateImpl):
|
||
(bmalloc::Allocator::reallocate):
|
||
(bmalloc::Allocator::allocateSlowCase):
|
||
* bmalloc/Allocator.h: Forward to DebugHeap instead of inlining all the
|
||
code. This is required for our new interface, and it is also a nice
|
||
simplification that moves some not-very-important code out of the way.
|
||
|
||
* bmalloc/Deallocator.cpp:
|
||
(bmalloc::Deallocator::Deallocator):
|
||
(bmalloc::Deallocator::scavenge):
|
||
(bmalloc::Deallocator::deallocateSlowCase):
|
||
* bmalloc/Deallocator.h: Ditto.
|
||
|
||
* bmalloc/DebugHeap.cpp: Added.
|
||
(bmalloc::DebugHeap::DebugHeap):
|
||
(bmalloc::DebugHeap::malloc):
|
||
(bmalloc::DebugHeap::memalign):
|
||
(bmalloc::DebugHeap::realloc):
|
||
(bmalloc::DebugHeap::free):
|
||
* bmalloc/DebugHeap.h: Added. New class for overriding normal heap
|
||
behavior. Right now, it just adds a malloc zone and then forwards to
|
||
system malloc -- but we can add lots more kinds of debug heaps in the
|
||
future if we find them useful.
|
||
|
||
* bmalloc/Environment.cpp:
|
||
(bmalloc::Environment::Environment):
|
||
(bmalloc::Environment::computeIsDebugHeapEnabled):
|
||
(bmalloc::Environment::computeIsBmallocEnabled): Deleted.
|
||
* bmalloc/Environment.h:
|
||
(bmalloc::Environment::isDebugHeapEnabled):
|
||
(bmalloc::Environment::isBmallocEnabled): Deleted. Renamed everything to
|
||
reflect our new use of DebugHeap.
|
||
|
||
* bmalloc/Heap.cpp:
|
||
(bmalloc::Heap::Heap):
|
||
* bmalloc/Heap.h:
|
||
(bmalloc::Heap::debugHeap): Updated to use DebugHeap.
|
||
(bmalloc::Heap::environment): Deleted.
|
||
|
||
* bmalloc/bmalloc.h:
|
||
(bmalloc::api::isEnabled): Updated to use DebugHeap.
|
||
|
||
2016-12-15 Myles C. Maxfield <mmaxfield@apple.com>
|
||
|
||
Sort Xcode project files
|
||
https://bugs.webkit.org/show_bug.cgi?id=165937
|
||
|
||
Reviewed by Simon Fraser.
|
||
|
||
* bmalloc.xcodeproj/project.pbxproj:
|
||
|
||
2016-12-08 David Kilzer <ddkilzer@apple.com>
|
||
|
||
Always check the return value of pthread_key_create()
|
||
<https://webkit.org/b/165274>
|
||
|
||
Reviewed by Darin Adler.
|
||
|
||
* bmalloc/PerThread.h:
|
||
(bmalloc::PerThreadStorage::init): Call BCRASH() if
|
||
pthread_key_create() returns an error. The error code will be
|
||
stored in a register available in a crash log, so no need to log
|
||
the value explicitly.
|
||
|
||
2016-12-06 Alexey Proskuryakov <ap@apple.com>
|
||
|
||
Correct SDKROOT values in xcconfig files
|
||
https://bugs.webkit.org/show_bug.cgi?id=165487
|
||
rdar://problem/29539209
|
||
|
||
Reviewed by Dan Bernstein.
|
||
|
||
Fix suggested by Dan Bernstein.
|
||
|
||
* Configurations/DebugRelease.xcconfig:
|
||
|
||
2016-11-29 Andy Estes <aestes@apple.com>
|
||
|
||
[Cocoa] Enable two clang warnings recommended by Xcode
|
||
https://bugs.webkit.org/show_bug.cgi?id=164498
|
||
|
||
Reviewed by Mark Lam.
|
||
|
||
* Configurations/Base.xcconfig: Enabled CLANG_WARN_INFINITE_RECURSION and CLANG_WARN_SUSPICIOUS_MOVE.
|
||
|
||
2016-11-10 Geoffrey Garen <ggaren@apple.com>
|
||
|
||
bmalloc threads should have a non-default QoS
|
||
https://bugs.webkit.org/show_bug.cgi?id=164612
|
||
|
||
Reviewed by Filip Pizlo.
|
||
|
||
* bmalloc/AsyncTask.h:
|
||
(bmalloc::Function>::threadEntryPoint): Request user-interactive quality
|
||
of service because user-interactive tasks use malloc.
|
||
|
||
2016-10-20 Mark Lam <mark.lam@apple.com>
|
||
|
||
bmalloc api should crash on failure to allocate when !isBmallocEnabled.
|
||
https://bugs.webkit.org/show_bug.cgi?id=163766
|
||
|
||
Reviewed by Keith Miller and Filip Pizlo.
|
||
|
||
We want to crash in bmalloc on failure to allocate even when !isBmallocEnabled.
|
||
This is so that failures to allocate memory will manifest as crashes with a
|
||
unique signature (i.e. as a SIGTRAP on release builds, or as a write to illegal
|
||
address 0xbbadbeef on debug builds) and the crash will manifest inside bmalloc.
|
||
This distinguishes allocation failures from other crashing bugs that manifest as
|
||
SIGSEGVs due to random pointer dereferences in the clients of bmalloc.
|
||
|
||
* bmalloc/Allocator.cpp:
|
||
(bmalloc::Allocator::allocateImpl):
|
||
(bmalloc::Allocator::reallocate):
|
||
(bmalloc::Allocator::allocateSlowCase):
|
||
|
||
2016-09-26 Yoshiaki Jitsukawa <Yoshiaki.Jitsukawa@sony.com>
|
||
|
||
Avoid implicit conversion from iterator to pointer
|
||
https://bugs.webkit.org/show_bug.cgi?id=162482
|
||
|
||
Reviewed by Geoffrey Garen.
|
||
|
||
Not every STL supporting such conversion, we should get a pointer explicitly.
|
||
|
||
* bmalloc/Chunk.h:
|
||
(bmalloc::Chunk::lines):
|
||
(bmalloc::Chunk::pages):
|
||
* bmalloc/FixedVector.h:
|
||
(bmalloc::FixedVector::begin):
|
||
|
||
2016-08-31 Filip Pizlo <fpizlo@apple.com>
|
||
|
||
Butterflies should be allocated in Auxiliary MarkedSpace instead of CopiedSpace and we should rewrite as much of the GC as needed to make this not a regression
|
||
https://bugs.webkit.org/show_bug.cgi?id=160125
|
||
|
||
Reviewed by Geoffrey Garen and Keith Miller.
|
||
|
||
I needed to tryMemalign, so I added such a thing.
|
||
|
||
* bmalloc/Allocator.cpp:
|
||
(bmalloc::Allocator::allocate):
|
||
(bmalloc::Allocator::tryAllocate):
|
||
(bmalloc::Allocator::allocateImpl):
|
||
* bmalloc/Allocator.h:
|
||
* bmalloc/Cache.h:
|
||
(bmalloc::Cache::tryAllocate):
|
||
* bmalloc/bmalloc.h:
|
||
(bmalloc::api::tryMemalign):
|
||
|
||
2016-08-30 Yusuke Suzuki <utatane.tea@gmail.com>
|
||
|
||
Unreviewed, build fix for GCC ports
|
||
|
||
std::forward is declared in <utility> header.
|
||
|
||
* bmalloc/ScopeExit.h:
|
||
|
||
2016-08-30 Geoffrey Garen <ggaren@apple.com>
|
||
|
||
bmalloc: speed up the lock slow path
|
||
https://bugs.webkit.org/show_bug.cgi?id=161058
|
||
|
||
Unreviewed roll-in - with regression fixed.
|
||
|
||
Revert to using yield() instead of swtch() because very low priority
|
||
background tasks can cause priority inversion and deadlock. In the
|
||
network process, that happened with com.apple.WebKit.Cache.Storage.serialBackground.
|
||
|
||
Still a big speedup on MallocBench.
|
||
|
||
* bmalloc.xcodeproj/project.pbxproj:
|
||
* bmalloc/ScopeExit.h: Added.
|
||
(bmalloc::ScopeExit::ScopeExit):
|
||
(bmalloc::ScopeExit::~ScopeExit):
|
||
(bmalloc::makeScopeExit):
|
||
* bmalloc/StaticMutex.cpp:
|
||
(bmalloc::StaticMutex::lockSlowCase):
|
||
* bmalloc/StaticMutex.h:
|
||
(bmalloc::StaticMutex::init):
|
||
|
||
2016-08-26 Geoffrey Garen <ggaren@apple.com>
|
||
|
||
Unreviewed build fix.
|
||
|
||
Fix the CMake build.
|
||
|
||
* CMakeLists.txt:
|
||
|
||
2016-08-26 Geoffrey Garen <ggaren@apple.com>
|
||
|
||
Renamed XLarge* => Large*
|
||
https://bugs.webkit.org/show_bug.cgi?id=161261
|
||
|
||
Reviewed by Andreas Kling.
|
||
|
||
XLarge is not a thing anymore: We just have Small and Large.
|
||
|
||
* bmalloc.xcodeproj/project.pbxproj:
|
||
* bmalloc/Heap.cpp:
|
||
(bmalloc::Heap::splitAndAllocate):
|
||
(bmalloc::Heap::tryAllocateLarge):
|
||
(bmalloc::Heap::shrinkLarge):
|
||
(bmalloc::Heap::deallocateLarge):
|
||
* bmalloc/Heap.h:
|
||
* bmalloc/LargeMap.cpp: Copied from Source/bmalloc/bmalloc/XLargeMap.cpp.
|
||
(bmalloc::LargeMap::remove):
|
||
(bmalloc::LargeMap::add):
|
||
(bmalloc::XLargeMap::remove): Deleted.
|
||
(bmalloc::XLargeMap::add): Deleted.
|
||
* bmalloc/LargeMap.h: Copied from Source/bmalloc/bmalloc/XLargeMap.h.
|
||
(bmalloc::LargeMap::ranges):
|
||
(bmalloc::XLargeMap::ranges): Deleted.
|
||
* bmalloc/LargeRange.h: Copied from Source/bmalloc/bmalloc/XLargeRange.h.
|
||
(bmalloc::LargeRange::LargeRange):
|
||
(bmalloc::LargeRange::operator<):
|
||
(bmalloc::canMerge):
|
||
(bmalloc::merge):
|
||
(bmalloc::LargeRange::split):
|
||
(bmalloc::XLargeRange::XLargeRange): Deleted.
|
||
(bmalloc::XLargeRange::operator<): Deleted.
|
||
(bmalloc::XLargeRange::split): Deleted.
|
||
* bmalloc/VMHeap.cpp:
|
||
(bmalloc::VMHeap::tryAllocateLargeChunk):
|
||
* bmalloc/VMHeap.h:
|
||
* bmalloc/XLargeMap.cpp: Removed.
|
||
* bmalloc/XLargeMap.h: Removed.
|
||
* bmalloc/XLargeRange.h: Removed.
|
||
|
||
2016-08-26 Gavin Barraclough <barraclough@apple.com>
|
||
|
||
bmalloc: speed up the lock slow path
|
||
https://bugs.webkit.org/show_bug.cgi?id=161058
|
||
|
||
Unreviewed rollout - this caused regressions <rdar://problem/28026089>.
|
||
|
||
* bmalloc.xcodeproj/project.pbxproj:
|
||
* bmalloc/ScopeExit.h: Removed.
|
||
* bmalloc/StaticMutex.cpp:
|
||
(bmalloc::StaticMutex::lockSlowCase):
|
||
* bmalloc/StaticMutex.h:
|
||
(bmalloc::StaticMutex::init):
|
||
* bmalloc/ThreadSwitch.h: Removed.
|
||
|
||
2016-08-24 Andreas Kling <akling@apple.com>
|
||
|
||
Add bmalloc::api::isEnabled().
|
||
<https://webkit.org/b/160534>
|
||
|
||
Reviewed by Joseph Pecoraro.
|
||
|
||
* bmalloc/bmalloc.h:
|
||
(bmalloc::api::isEnabled):
|
||
|
||
2016-08-24 Filip Pizlo <fpizlo@apple.com>
|
||
|
||
Unreviewed, roll out r204901, r204897, r204866, r204856, r204854.
|
||
|
||
* bmalloc/Allocator.cpp:
|
||
(bmalloc::Allocator::allocate):
|
||
(bmalloc::Allocator::tryAllocate): Deleted.
|
||
(bmalloc::Allocator::allocateImpl): Deleted.
|
||
* bmalloc/Allocator.h:
|
||
* bmalloc/Cache.h:
|
||
(bmalloc::Cache::tryAllocate): Deleted.
|
||
* bmalloc/bmalloc.h:
|
||
(bmalloc::api::tryMemalign): Deleted.
|
||
|
||
2016-08-12 Filip Pizlo <fpizlo@apple.com>
|
||
|
||
Butterflies should be allocated in Auxiliary MarkedSpace instead of CopiedSpace and we should rewrite as much of the GC as needed to make this not a regression
|
||
https://bugs.webkit.org/show_bug.cgi?id=160125
|
||
|
||
Reviewed by Geoffrey Garen.
|
||
|
||
I needed to tryMemalign, so I added such a thing.
|
||
|
||
* bmalloc/Allocator.cpp:
|
||
(bmalloc::Allocator::allocate):
|
||
(bmalloc::Allocator::tryAllocate):
|
||
(bmalloc::Allocator::allocateImpl):
|
||
(bmalloc::Allocator::reallocate):
|
||
* bmalloc/Allocator.h:
|
||
* bmalloc/Cache.h:
|
||
(bmalloc::Cache::allocate):
|
||
(bmalloc::Cache::tryAllocate):
|
||
* bmalloc/bmalloc.h:
|
||
(bmalloc::api::malloc):
|
||
(bmalloc::api::tryMemalign):
|
||
(bmalloc::api::memalign):
|
||
|
||
2016-08-22 Yusuke Suzuki <utatane.tea@gmail.com>
|
||
|
||
Unreviewed, build fix on GCC environment
|
||
https://bugs.webkit.org/show_bug.cgi?id=161058
|
||
|
||
std::forward is declared in <utility> header.
|
||
|
||
* bmalloc/ScopeExit.h:
|
||
|
||
2016-08-22 Geoffrey Garen <ggaren@apple.com>
|
||
|
||
bmalloc: speed up the lock slow path
|
||
https://bugs.webkit.org/show_bug.cgi?id=161058
|
||
|
||
Reviewed by Filip Pizlo.
|
||
|
||
It is generally accepted practice that a lock should yield instead of
|
||
spinning when a lock acquisition fails, to avoid wasting CPU and power.
|
||
|
||
There are two problems with this generally accepted practice:
|
||
|
||
(1) It's a fallacy that yielding is free. In reality, yielding itself
|
||
consumes CPU and power -- by performing a syscall, running the OS
|
||
scheduler, and possibly performing a context switch. (Instruments
|
||
traces of MallocBench show the cost of yielding.) Therefore, spinning a
|
||
little to avoid yielding can actually *save* CPU and power.
|
||
|
||
(2) std::this_thread_yield() on Darwin is way too aggressive: It not only
|
||
yields but also depresses your priority to absolute zero for 10ms. A
|
||
recent PLT trace showed a few spots where the main thread just gave up
|
||
on loading and rendering a page for 10ms so an unimportant background
|
||
task could run.
|
||
|
||
To correct these problems, this patch adds a little bit of spinning to
|
||
the bmalloc lock slow path.
|
||
|
||
Below are performance results on various CPUs.
|
||
|
||
Mac Pro (12 hyperthreaded cores = 24 threads):
|
||
|
||
Baseline Patch Δ
|
||
Execution Time:
|
||
message_one 173ms 173ms
|
||
message_many 953ms 927ms ^ 1.03x faster
|
||
churn --parallel 60ms 41ms ^ 1.46x faster
|
||
list_allocate --parallel 224ms 143ms ^ 1.57x faster
|
||
tree_allocate --parallel 1,190ms 758ms ^ 1.57x faster
|
||
tree_churn --parallel 1,517ms 906ms ^ 1.67x faster
|
||
facebook --parallel 6,519ms 4,580ms ^ 1.42x faster
|
||
reddit --parallel 5,097ms 3,411ms ^ 1.49x faster
|
||
flickr --parallel 4,903ms 3,501ms ^ 1.4x faster
|
||
theverge --parallel 6,641ms 4,505ms ^ 1.47x faster
|
||
|
||
<geometric mean> 1,158ms 832ms ^ 1.39x faster
|
||
<arithmetic mean> 2,728ms 1,895ms ^ 1.44x faster
|
||
<harmonic mean> 332ms 240ms ^ 1.38x faster
|
||
|
||
MacBook Air (2 hyperthreaded cores = 4 threads):
|
||
|
||
Baseline Patch Δ
|
||
Execution Time:
|
||
message_one 911ms 907ms ^ 1.0x faster
|
||
message_many 515ms 513ms ^ 1.0x faster
|
||
churn --parallel 132ms 134ms ! 1.02x slower
|
||
list_allocate --parallel 104ms 102ms ^ 1.02x faster
|
||
tree_allocate --parallel 117ms 111ms ^ 1.05x faster
|
||
tree_churn --parallel 154ms 151ms ^ 1.02x faster
|
||
facebook --parallel 719ms 687ms ^ 1.05x faster
|
||
reddit --parallel 382ms 341ms ^ 1.12x faster
|
||
flickr --parallel 372ms 345ms ^ 1.08x faster
|
||
theverge --parallel 489ms 444ms ^ 1.1x faster
|
||
|
||
<geometric mean> 299ms 287ms ^ 1.04x faster
|
||
<arithmetic mean> 390ms 374ms ^ 1.04x faster
|
||
<harmonic mean> 227ms 220ms ^ 1.03x faster
|
||
|
||
iPad (2 cores = 2 threads):
|
||
|
||
[ Doesn't run Ruby, so no pretty subtest output. ]
|
||
|
||
Baseline Patch Δ
|
||
Execution Time: 174.14ms 171.5ms ^ 1.02x faster
|
||
|
||
* bmalloc.xcodeproj/project.pbxproj:
|
||
|
||
* bmalloc/ScopeExit.h: Added. A barebones very wimpy version of
|
||
WTF::ScopeExit.
|
||
(bmalloc::ScopeExit::ScopeExit):
|
||
(bmalloc::ScopeExit::~ScopeExit):
|
||
(bmalloc::makeScopeExit):
|
||
|
||
* bmalloc/StaticMutex.cpp:
|
||
(bmalloc::StaticMutex::lockSlowCase): Spin before yielding -- that's the
|
||
speedup. Don't spin if another CPU is already spinning. In theory, more
|
||
than one spinner accomplishes nothing, and I found that there's a cutoff
|
||
around 8 or 16 spinners that becomes performance negative on Mac Pro.
|
||
|
||
(Note: Another way to accomplish a similar result, if you don't want to
|
||
use a bit of state in the lock, is to spin for a random duration between
|
||
0 and aLot. I tested a version of WTF::WeakRandom with unsynchronized
|
||
static state and it worked great. But I ultimately opted for the explicit
|
||
bit because I thought it was clearer.)
|
||
|
||
* bmalloc/StaticMutex.h:
|
||
(bmalloc::StaticMutex::init): Initialize our new bit.
|
||
|
||
* bmalloc/ThreadSwitch.h: Added.
|
||
(bmalloc::threadSwitch): Don't call yield() on Darwin because it's too
|
||
aggressive. swtch() does what we want: Go run something else, without
|
||
any other side-effects.
|
||
|
||
2016-08-03 Geoffrey Garen <ggaren@apple.com>
|
||
|
||
[bmalloc] Merging of XLargeRanges can leak the upper range
|
||
https://bugs.webkit.org/show_bug.cgi?id=160403
|
||
|
||
Reviewed by Michael Saboff.
|
||
|
||
* bmalloc/Heap.cpp:
|
||
(bmalloc::Heap::scavengeLargeObjects): Don't use removePhysical().
|
||
Recorded physical size is a performance optimization. It is not the
|
||
truth. So it might be zero even if a range contains physical pages.
|
||
|
||
Instead, iterate each range in the map unconditionally.
|
||
|
||
The map can shrink when we release the lock, so we must clamp our
|
||
iterator each time through the loop.
|
||
|
||
The map can grow when we release the lock, but we don't care because
|
||
growth restarts the scavenger from the beginning.
|
||
|
||
* bmalloc/XLargeMap.cpp:
|
||
(bmalloc::XLargeMap::removePhysical): Deleted. Not used anymore.
|
||
|
||
* bmalloc/XLargeMap.h:
|
||
(bmalloc::XLargeMap::ranges): Added direct access for the sake of
|
||
scavengeLargeObjects. (This violates our naming conventions -- I'll do
|
||
a rename in a follow-up patch.)
|
||
|
||
2016-07-13 Enrica Casucci <enrica@apple.com>
|
||
|
||
Update supported platforms in xcconfig files to match the sdk names.
|
||
https://bugs.webkit.org/show_bug.cgi?id=159728
|
||
|
||
Reviewed by Tim Horton.
|
||
|
||
* Configurations/Base.xcconfig:
|
||
|
||
2016-07-11 Geoffrey Garen <ggaren@apple.com>
|
||
|
||
Crash due to abort() calling libc++.1.dylib: std::__1::thread::detach()
|
||
https://bugs.webkit.org/show_bug.cgi?id=159655
|
||
|
||
Reviewed by Sam Weinig.
|
||
|
||
It's not entirely clear what was happening in these crashes, but our
|
||
use of detach() was 100% forward-looking, so we can just remove it for
|
||
now.
|
||
|
||
This patch removes the ability for the scavenger owner to die before
|
||
the scavenger thread dies (which was unused) and also removes the
|
||
ability for the scavenger thread to exit (which was used, but we
|
||
messed up and did thread joining lazily, so we never got any benefit
|
||
from thread exit.)
|
||
|
||
We can add these features back when we need them, and make them work then.
|
||
|
||
* bmalloc/AsyncTask.h:
|
||
(bmalloc::Function>::AsyncTask): We start out in the running state now
|
||
because we know that starting our thread will run it.
|
||
|
||
(bmalloc::Function>::~AsyncTask): We don't support destruction anymore.
|
||
|
||
(bmalloc::Function>::runSlowCase): I removed the Exited state.
|
||
|
||
(bmalloc::Function>::threadRunLoop): I removed the Exited and
|
||
ExitRequested states.
|
||
|
||
* bmalloc/Heap.h:
|
||
|
||
* bmalloc/VMHeap.h:
|
||
|
||
2016-06-12 David Kilzer <ddkilzer@apple.com>
|
||
|
||
Crash in com.apple.WebKit.WebContent at std::__1::__call_once_proxy<std::__1::tuple<CrashReporterSupportLibrary()::$_0&&> >
|
||
<https://webkit.org/b/158660>
|
||
<rdar://problem/25652686>
|
||
|
||
Reviewed by Darin Adler.
|
||
|
||
* bmalloc/Logging.cpp: Switch to use
|
||
BSOFT_LINK_PRIVATE_FRAMEWORK() to link
|
||
CrashReporterSupport.framework.
|
||
* bmalloc/darwin/BSoftLinking.h:
|
||
(BSOFT_LINK_PRIVATE_FRAMEWORK): Rename from BSOFT_LINK_FRAMEWORK.
|
||
Switch to use /System/Library/PrivateFrameworks/.
|
||
|
||
2016-06-11 David Kilzer <ddkilzer@apple.com>
|
||
|
||
Implement logging for RELEASE_BASSERT_WITH_MESSAGE() in BAssert.h
|
||
<http://webkit.org/b/155992>
|
||
|
||
Reviewed by Geoff Garen.
|
||
|
||
* bmalloc/BAssert.h:
|
||
(BLOG_ERROR): Add method to always log error messages.
|
||
(RELEASE_BASSERT_WITH_MESSAGE): Use BLOG_ERROR() to implement
|
||
logging in Debug builds.
|
||
* bmalloc/BPlatform.h:
|
||
(BPLATFORM_MAC): Add.
|
||
(BUSE): Add BUSE() macro.
|
||
(BATTRIBUTE_PRINTF): Add.
|
||
(BUSE_OS_LOG): Add.
|
||
* bmalloc/Logging.cpp:
|
||
(bmalloc::reportAssertionFailureWithMessage): Add. Logs to
|
||
stderr.
|
||
* bmalloc/Logging.h:
|
||
(bmalloc::reportAssertionFailureWithMessage): Add declaration.
|
||
|
||
2016-06-07 Pranjal Jumde <pjumde@apple.com>
|
||
|
||
Prevents integer overflow in Vector.h
|
||
https://bugs.webkit.org/show_bug.cgi?id=158455
|
||
<rdar://problem/20235469>
|
||
|
||
Reviewed by Mark Lam.
|
||
|
||
* bmalloc/Vector.h:
|
||
(bmalloc::Vector<T>::reallocateBuffer):
|
||
|
||
2016-05-27 Konstantin Tokarev <annulen@yandex.ru>
|
||
|
||
[cmake] Deduplicated bmalloc/Zone.cpp handling.
|
||
https://bugs.webkit.org/show_bug.cgi?id=158154
|
||
|
||
Reviewed by Alex Christensen.
|
||
|
||
File bmalloc/Zone.cpp is required on Darwin irrespectively from what
|
||
port is being built.
|
||
|
||
Also I removed WEBKIT_INCLUDE_CONFIG_FILES_IF_EXISTS() because it's
|
||
unlikely that bmalloc will ever need port-specific customizations (as
|
||
opposed to OS-specific customizations which should be done in
|
||
CMakeLists.txt).
|
||
|
||
* CMakeLists.txt: Added bmalloc/Zone.cpp for Darwin.
|
||
* PlatformGTK.cmake: Removed.
|
||
* PlatformMac.cmake: Removed.
|
||
|
||
2016-05-22 Brady Eidson <beidson@apple.com>
|
||
|
||
Move to C++14.
|
||
https://bugs.webkit.org/show_bug.cgi?id=157948
|
||
|
||
Reviewed by Michael Catanzaro.
|
||
|
||
* Configurations/Base.xcconfig:
|
||
|
||
2016-05-17 Geoffrey Garen <ggaren@apple.com>
|
||
|
||
REGRESSION: JetStream crashes on some iPhones
|
||
https://bugs.webkit.org/show_bug.cgi?id=157814
|
||
|
||
Reviewed by Michael Saboff.
|
||
|
||
* bmalloc/Sizes.h: Reduce smallMax to 32kB.
|
||
|
||
Previous justification for 64kB was:
|
||
|
||
* bmalloc/Sizes.h: Upped smallMax to 64kB. Upping to 32kB is pretty
|
||
reasonable, since sizes between 16kB and 32kB share page sizes. I went
|
||
all the way up to 64kB because the GC uses 64kB blocks, and also just
|
||
for extra padding to ensure that large allocations are indeed rare.
|
||
|
||
It turns out that the bump to 64kB substantially increases our memory
|
||
high water mark on JetStream, leading to jetsam crashes. Also, there
|
||
doesn't seem to be a practical performance problem to putting objects in
|
||
the (32kB - 64kB) range in the large allocator.
|
||
|
||
2016-05-16 Geoffrey Garen <ggaren@apple.com>
|
||
|
||
REGRESSION (200035): changes in "WebKit Malloc" VM regions are causing 'leaks' to spew "Failed to map remote region" messages
|
||
https://bugs.webkit.org/show_bug.cgi?id=157764
|
||
|
||
Reviewed by Gavin Barraclough.
|
||
|
||
We need to allow for guard pages and only report unguarded pages to the
|
||
leaks tool -- otherwise, it will try to remote map our guarded pages,
|
||
and crash.
|
||
|
||
* bmalloc/VMHeap.cpp:
|
||
(bmalloc::VMHeap::tryAllocateLargeChunk):
|
||
(bmalloc::VMHeap::allocateSmallChunk): Adopt the new API for reporting
|
||
a range instead of a Chunk*, and report the unguarded range.
|
||
|
||
This also fixes a separate bug -- very large allocations would not
|
||
fully participate in pointer scanning because they would only report 2MB
|
||
(chunkSize) in size. This could cause false-positive leak reports.
|
||
|
||
* bmalloc/Zone.cpp:
|
||
(bmalloc::enumerator): Updated to scan ranges instead of fixed-sized
|
||
Chunk pointers.
|
||
|
||
* bmalloc/Zone.h:
|
||
(bmalloc::Zone::ranges):
|
||
(bmalloc::Zone::addRange): Store ranges instead of fixed-sized Chunk
|
||
pointers because our VM ranges have variable sizes -- both due to guard
|
||
pages and due to large allocations.
|
||
|
||
(bmalloc::Zone::chunks): Deleted.
|
||
(bmalloc::Zone::addChunk): Deleted.
|
||
|
||
2016-05-10 David Kilzer <ddkilzer@apple.com>
|
||
|
||
bmalloc should automatically disable itself when ThreadSanitizer is used
|
||
<https://webkit.org/b/157527>
|
||
|
||
Reviewed by Michael Catanzaro.
|
||
|
||
* bmalloc/Environment.cpp:
|
||
(bmalloc::isASanEnabled): Rename to isSanitizerEnabled.
|
||
(bmalloc::isSanitizerEnabled): Rename from isASanEnabled. Add
|
||
support for detecting ThreadSanitizer.
|
||
(bmalloc::Environment::computeIsBmallocEnabled): Switch from
|
||
isASanEnabled to isSanitizerEnabled.
|
||
|
||
2016-05-03 Geoffrey Garen <ggaren@apple.com>
|
||
|
||
Assertion failure in bmalloc::vmRevokePermissions(void*, unsigned long).
|
||
https://bugs.webkit.org/show_bug.cgi?id=157047
|
||
|
||
Reviewed by Filip Pizlo.
|
||
|
||
Renamed roundUpToMultipleOfSloppy => roundUpToMultipleOfNonPowerOfTwo.
|
||
|
||
* bmalloc/Algorithm.h:
|
||
(bmalloc::roundUpToMultipleOfNonPowerOfTwo):
|
||
(bmalloc::roundUpToMultipleOfSloppy): Deleted.
|
||
* bmalloc/VMHeap.cpp:
|
||
(bmalloc::VMHeap::allocateSmallChunk):
|
||
|
||
2016-05-03 Geoffrey Garen <ggaren@apple.com>
|
||
|
||
Assertion failure in bmalloc::vmRevokePermissions(void*, unsigned long).
|
||
https://bugs.webkit.org/show_bug.cgi?id=157047
|
||
|
||
Reviewed by Filip Pizlo.
|
||
|
||
The previous fix aligned the guard page sizes correctly but forgot to
|
||
align the guard page start address correctly.
|
||
|
||
* bmalloc/Algorithm.h:
|
||
(bmalloc::roundUpToMultipleOfSloppy): Use a new helper method to round
|
||
up when not working with a power of two, instead of writing out the
|
||
math by hand.
|
||
|
||
* bmalloc/VMHeap.cpp:
|
||
(bmalloc::VMHeap::allocateSmallChunk): Make sure to round up the guard
|
||
page start address in addition to its size. Assert at the very end to
|
||
try to catch more bugs.
|
||
|
||
2016-04-27 Geoffrey Garen <ggaren@apple.com>
|
||
|
||
Assertion failure in bmalloc::vmRevokePermissions(void*, unsigned long).
|
||
https://bugs.webkit.org/show_bug.cgi?id=157047
|
||
|
||
Reviewed by Darin Adler.
|
||
|
||
* bmalloc/Chunk.h:
|
||
(bmalloc::Chunk::Chunk):
|
||
(bmalloc::Chunk::get):
|
||
(bmalloc::Chunk::offset):
|
||
(bmalloc::Chunk::address):
|
||
(bmalloc::Object::Object):
|
||
(bmalloc::Object::address):
|
||
(bmalloc::Object::line):
|
||
(bmalloc::Chunk::object): Deleted.
|
||
(bmalloc::Object::begin): Deleted.
|
||
* bmalloc/Heap.cpp:
|
||
(bmalloc::Heap::allocateSmallBumpRangesByObject):
|
||
* bmalloc/Object.h:
|
||
(bmalloc::Object::chunk):
|
||
(bmalloc::Object::offset): Renamed begin() to address() because this is
|
||
not an iterator.
|
||
|
||
* bmalloc/VMHeap.cpp:
|
||
(bmalloc::VMHeap::allocateSmallChunk): Round up pageSize to a vmPageSize
|
||
multiple because pageSize might be smaller than vmPageSize, but we
|
||
think the VM system requires vmPageSize-aligned values.
|
||
|
||
2016-04-25 Geoffrey Garen <ggaren@apple.com>
|
||
|
||
bmalloc: vm allocations should plant guard pages
|
||
https://bugs.webkit.org/show_bug.cgi?id=156937
|
||
|
||
Rolling back in r199936 with a fix for the memory regression.
|
||
|
||
2016-04-23 Gavin Barraclough <barraclough@apple.com>
|
||
|
||
bmalloc: vm allocations should plant guard pages
|
||
https://bugs.webkit.org/show_bug.cgi?id=156937
|
||
|
||
Rolling out - looks like this is memory regression.
|
||
|
||
* bmalloc/Object.h:
|
||
(bmalloc::Object::operator+):
|
||
(bmalloc::Object::operator<=):
|
||
(bmalloc::Object::operator-): Deleted.
|
||
* bmalloc/VMAllocate.h:
|
||
(bmalloc::vmDeallocate):
|
||
(bmalloc::vmRevokePermissions): Deleted.
|
||
* bmalloc/VMHeap.cpp:
|
||
(bmalloc::VMHeap::allocateSmallChunk):
|
||
|
||
2016-04-22 Geoffrey Garen <ggaren@apple.com>
|
||
|
||
bmalloc: vm allocations should plant guard pages
|
||
https://bugs.webkit.org/show_bug.cgi?id=156937
|
||
|
||
Reviewed by Michael Saboff.
|
||
|
||
* bmalloc/Object.h:
|
||
(bmalloc::Object::operator-): Added a - helper.
|
||
|
||
* bmalloc/VMAllocate.h:
|
||
(bmalloc::vmRevokePermissions): Added a helper to revoke permissions on
|
||
a VM region. We use this for guard pages.
|
||
|
||
* bmalloc/VMHeap.cpp:
|
||
(bmalloc::VMHeap::allocateSmallChunk): Add guard pages to the start and
|
||
end of the chunk.
|
||
|
||
Note that we don't guard large chunks becuase we need to be able to merge
|
||
them. Otherwise, we will run out of virtual addresses.
|
||
|
||
2016-04-22 Geoffrey Garen <ggaren@apple.com>
|
||
|
||
bmalloc: Constify introspect function pointer table
|
||
https://bugs.webkit.org/show_bug.cgi?id=156936
|
||
|
||
Reviewed by Michael Saboff.
|
||
|
||
* bmalloc/Zone.cpp:
|
||
(bmalloc::Zone::Zone): Declaring this function pointer table const puts
|
||
it in the read-only section of the binary, providing a little hardening
|
||
against overwriting the function pointers at runtime. (We have to
|
||
const_cast when assigning because the API declares a pointer to non-const,
|
||
but we happen to know it will never try to write through that pointer.
|
||
This is not my favorite API.)
|
||
|
||
2016-04-19 Geoffrey Garen <ggaren@apple.com>
|
||
|
||
bmalloc: fix up overflow checks
|
||
https://bugs.webkit.org/show_bug.cgi?id=156780
|
||
|
||
Reviewed by Mark Lam.
|
||
|
||
We used to try to avoid overflow in large object math by setting a very
|
||
high limit on the largest large object. But that's a bit error-prone
|
||
since the check is far away from the math that might overflow -- and
|
||
we were missing some cases.
|
||
|
||
This patch removes the limit and instead checks at each math site.
|
||
|
||
* bmalloc/Allocator.cpp:
|
||
(bmalloc::Allocator::tryAllocate):
|
||
(bmalloc::Allocator::allocate):
|
||
(bmalloc::Allocator::reallocate):
|
||
(bmalloc::Allocator::allocateSlowCase): Remove the limit. tryAllocateLarge
|
||
will check for overflow for us.
|
||
|
||
* bmalloc/Chunk.h: This ASSERT was just totally wrong.
|
||
|
||
* bmalloc/Heap.cpp:
|
||
(bmalloc::Heap::tryAllocateLarge): Check for overflow when adding.
|
||
|
||
* bmalloc/Sizes.h:
|
||
|
||
* bmalloc/VMAllocate.h:
|
||
(bmalloc::tryVMAllocate): Check for overflow when adding.
|
||
|
||
* bmalloc/VMHeap.cpp:
|
||
(bmalloc::VMHeap::tryAllocateLargeChunk): Check for overflow when adding.
|
||
|
||
2016-04-19 Geoffrey Garen <ggaren@apple.com>
|
||
|
||
Unreviewed, try to fix an ASSERT seen on the bots.
|
||
|
||
* bmalloc/Heap.cpp:
|
||
(bmalloc::Heap::tryAllocateLarge): This ASSERT is supposed to be about
|
||
alignment, not size. Oops.
|
||
|
||
2016-04-19 Geoffrey Garen <ggaren@apple.com>
|
||
|
||
bmalloc: Merge the large and xlarge allocators
|
||
https://bugs.webkit.org/show_bug.cgi?id=156734
|
||
|
||
Reviewed by Andreas Kling.
|
||
|
||
This give us better defense against worst case memory usage:
|
||
|
||
Baseline Patch Δ
|
||
Peak Memory:
|
||
nimlang 198,132kB 181,468kB ^ 1.09x smaller
|
||
|
||
It also eliminates inline metadata for large objects, fixing the
|
||
regression introduced in r198675, and more:
|
||
|
||
run-malloc-benchmarks Baseline:~/OpenSource/WebKitBuildBaseline/Release/ Patch:~/OpenSource/WebKitBuild/Release/
|
||
|
||
Baseline Patch Δ
|
||
Memory at End:
|
||
big 10,880kB 3,328kB ^ 3.27x smaller
|
||
facebook 3,112kB 2,868kB ^ 1.09x smaller
|
||
fragment --parallel 1,848kB 760kB ^ 2.43x smaller
|
||
fragment_iterate --parallel 4,908kB 776kB ^ 6.32x smaller
|
||
big --parallel 48,076kB 11,892kB ^ 4.04x smaller
|
||
|
||
Overall memory use looks OK:
|
||
|
||
run-malloc-benchmarks --memory_warning Baseline:~/OpenSource/WebKitBuildBaseline/Release/ Patch:~/OpenSource/WebKitBuild/Release/
|
||
|
||
Baseline Patch Δ
|
||
Memory at End:
|
||
<arithmetic mean> 13,992kB 13,987kB ^ 1.0x smaller
|
||
|
||
Overall throughput looks OK:
|
||
|
||
run-malloc-benchmarks Baseline:~/OpenSource/WebKitBuildBaseline/Release/ Patch:~/OpenSource/WebKitBuild/Release/
|
||
|
||
Baseline Patch Δ
|
||
Execution Time:
|
||
<arithmetic mean> 103ms 104ms ! 1.01x slower
|
||
|
||
We're a bit slower on the "all-out large allocations on all cores"
|
||
benchmark, but I think that's an OK price to pay:
|
||
|
||
Baseline Patch Δ
|
||
Execution Time:
|
||
big --parallel 125ms 136ms ! 1.09x slower
|
||
|
||
This patch net removes 1.5k lines of code. It turns out that large
|
||
allocations are rare, and free memory fragments are also rare, so the
|
||
combination is super rare, and a simple O(n) algorithm that ensures good
|
||
memory behavior is the best option.
|
||
|
||
Fun fact: In practice, the odds that the old code would save memory
|
||
were *worse* than the odds that it would contain a bug that wasted
|
||
memory. :)
|
||
|
||
* bmalloc.xcodeproj/project.pbxproj:
|
||
|
||
* bmalloc/Allocator.cpp:
|
||
(bmalloc::Allocator::tryAllocate): largeMax is the new xLargeMax since
|
||
xLargeMax is gone now.
|
||
|
||
(bmalloc::Allocator::allocate): I moved the rounding code into allocateLarge,
|
||
so we don't have to do it here.
|
||
|
||
(bmalloc::Allocator::reallocate):
|
||
(bmalloc::Allocator::allocateSlowCase):
|
||
(bmalloc::Allocator::allocateXLarge): Deleted. No more XLarge case.
|
||
|
||
* bmalloc/Allocator.h:
|
||
|
||
* bmalloc/BeginTag.h: Removed.
|
||
* bmalloc/BoundaryTag.h: Removed.
|
||
|
||
* bmalloc/Chunk.h:
|
||
(bmalloc::ChunkHash::hash): Added a hash function. The best hash function
|
||
is a unique and monotonically increasing integer, and that's exactly what
|
||
we typically get from the high bits of a Chunk, since the OS allocates
|
||
Chunks at unique and increasing addresses.
|
||
(bmalloc::Chunk::boundaryTags): Deleted.
|
||
(bmalloc::Chunk::objectType): Deleted.
|
||
(bmalloc::Chunk::beginTag): Deleted.
|
||
(bmalloc::Chunk::endTag): Deleted.
|
||
|
||
* bmalloc/Deallocator.cpp:
|
||
(bmalloc::Deallocator::deallocateSlowCase): We no longer know for sure,
|
||
by looking at its bit pattern, whether a pointer is small or large.
|
||
Instead, any pointer with large alignment *might* be large, and when
|
||
we occasionally encounter such an object, we have to consult a hash
|
||
table in the Heap to find out for sure. This turns out to be just as
|
||
cheap in practice.
|
||
|
||
We don't deallocate large objects on the fast path anymore. We can't,
|
||
because large objects have out-of-line metadata now.
|
||
|
||
(bmalloc::Deallocator::deallocateXLarge): Deleted.
|
||
|
||
* bmalloc/Deallocator.h:
|
||
(bmalloc::Deallocator::deallocateFastCase): See deallocateSlowCase.
|
||
|
||
* bmalloc/EndTag.h: Removed.
|
||
* bmalloc/FreeList.cpp: Removed.
|
||
* bmalloc/FreeList.h: Removed.
|
||
|
||
* bmalloc/Heap.cpp:
|
||
(bmalloc::Heap::allocateSmallPage): Be sure to track each chunk in
|
||
the object type map, so we can distinguish small vs large objects.
|
||
|
||
(bmalloc::Heap::deallocateSmallLine): No need to check object type
|
||
because we know object type now by virtue of being on the small object
|
||
path.
|
||
|
||
(bmalloc::Heap::splitAndAllocate): Be sure to track each chunk in
|
||
the object type map, so we can distinguish small vs large objects. Large
|
||
objects can split across chunks, so we need to add each large object's
|
||
chunk as it is allocated.
|
||
|
||
(bmalloc::Heap::tryAllocateLarge):
|
||
(bmalloc::Heap::allocateLarge):
|
||
(bmalloc::Heap::isLarge):
|
||
(bmalloc::Heap::largeSize):
|
||
(bmalloc::Heap::shrinkLarge):
|
||
(bmalloc::Heap::deallocateLarge): Merged in existing XLarge logic for
|
||
large objects.
|
||
|
||
(bmalloc::Heap::scavengeXLargeObjects): Deleted.
|
||
(bmalloc::Heap::allocateXLarge): Deleted.
|
||
(bmalloc::Heap::tryAllocateXLarge): Deleted.
|
||
(bmalloc::Heap::xLargeSize): Deleted.
|
||
(bmalloc::Heap::shrinkXLarge): Deleted.
|
||
(bmalloc::Heap::deallocateXLarge): Deleted.
|
||
|
||
* bmalloc/Heap.h:
|
||
(bmalloc::Heap::LargeObjectHash::hash):
|
||
|
||
* bmalloc/LargeObject.h: Removed.
|
||
|
||
* bmalloc/Map.h: Added.
|
||
(bmalloc::Map::size):
|
||
(bmalloc::Map::capacity):
|
||
(bmalloc::Map::get):
|
||
(bmalloc::Map::set):
|
||
(bmalloc::Map::remove):
|
||
(bmalloc::Map::shouldGrow):
|
||
(bmalloc::Map::shouldShrink):
|
||
(bmalloc::Map::find):
|
||
(bmalloc::Hash>::rehash): Simple hash table.
|
||
|
||
* bmalloc/Object.h:
|
||
|
||
* bmalloc/ObjectType.cpp:
|
||
(bmalloc::objectType):
|
||
* bmalloc/ObjectType.h:
|
||
(bmalloc::mightBeLarge): See deallocateSlowCase.
|
||
(bmalloc::isXLarge): Deleted.
|
||
|
||
* bmalloc/SegregatedFreeList.cpp: Removed.
|
||
* bmalloc/SegregatedFreeList.h: Removed.
|
||
|
||
* bmalloc/Sizes.h: Upped smallMax to 64kB. Upping to 32kB is pretty
|
||
reasonable, since sizes between 16kB and 32kB share page sizes. I went
|
||
all the way up to 64kB because the GC uses 64kB blocks, and also just
|
||
for extra padding to ensure that large allocations are indeed rare.
|
||
|
||
* bmalloc/SortedVector.h: Removed.
|
||
|
||
* bmalloc/VMHeap.cpp:
|
||
(bmalloc::VMHeap::tryAllocateLargeChunk):
|
||
(bmalloc::VMHeap::allocateSmallChunk):
|
||
(bmalloc::VMHeap::VMHeap): Deleted.
|
||
(bmalloc::VMHeap::allocateChunk): Deleted.
|
||
* bmalloc/VMHeap.h:
|
||
(bmalloc::VMHeap::deallocateSmallPage):
|
||
(bmalloc::VMHeap::allocateLargeObject): Deleted.
|
||
(bmalloc::VMHeap::deallocateLargeObject): Deleted. Nixed all the boundary
|
||
tag logic since metadata is out of line now.
|
||
|
||
* bmalloc/VMState.h: Removed. Instead of an abstract state, we track
|
||
the precise amount of committed physical pages at the head of a VM
|
||
range. This allows us to merge aggressively without triggering an madvise
|
||
storm most of the time.
|
||
|
||
* bmalloc/Vector.h:
|
||
(bmalloc::Vector<T>::Vector):
|
||
(bmalloc::Vector<T>::insert):
|
||
(bmalloc::Vector<T>::remove):
|
||
(bmalloc::Vector<T>::resize): Filled out some missing helpers.
|
||
|
||
* bmalloc/XLargeMap.cpp:
|
||
(bmalloc::XLargeMap::remove):
|
||
(bmalloc::XLargeMap::add):
|
||
(bmalloc::XLargeMap::removePhysical):
|
||
(bmalloc::XLargeMap::takeFree): Deleted.
|
||
(bmalloc::XLargeMap::addFree): Deleted.
|
||
(bmalloc::XLargeMap::addAllocated): Deleted.
|
||
(bmalloc::XLargeMap::getAllocated): Deleted.
|
||
(bmalloc::XLargeMap::takeAllocated): Deleted.
|
||
(bmalloc::XLargeMap::shrinkToFit): Deleted.
|
||
(bmalloc::XLargeMap::takePhysical): Deleted.
|
||
(bmalloc::XLargeMap::addVirtual): Deleted.
|
||
* bmalloc/XLargeMap.h:
|
||
(bmalloc::XLargeMap::Allocation::operator<): Deleted. We don't track
|
||
object sizes anymore -- just free space. (The Heap tracks object sizes.)
|
||
We use plain old linear search for free space. (See intro.)
|
||
|
||
* bmalloc/XLargeRange.h:
|
||
(bmalloc::XLargeRange::physicalSize):
|
||
(bmalloc::XLargeRange::setPhysicalSize):
|
||
(bmalloc::merge):
|
||
(bmalloc::XLargeRange::split):
|
||
(bmalloc::XLargeRange::vmState): Deleted.
|
||
(bmalloc::XLargeRange::setVMState): Deleted. See VMState.h.
|
||
|
||
2016-04-11 Fujii Hironori <Hironori.Fujii@jp.sony.com>
|
||
|
||
[CMake] Make FOLDER property INHERITED
|
||
https://bugs.webkit.org/show_bug.cgi?id=156460
|
||
|
||
Reviewed by Brent Fulgham.
|
||
|
||
* CMakeLists.txt:
|
||
Set FOLDER property as a directory property not a target property
|
||
|
||
2016-04-08 Geoffrey Garen <ggaren@apple.com>
|
||
|
||
bmalloc: stress_aligned test fails if you increase smallMax
|
||
https://bugs.webkit.org/show_bug.cgi?id=156414
|
||
|
||
Reviewed by Oliver Hunt.
|
||
|
||
When size exceeds alignment and is a multiple of alignment and is not
|
||
a power of two, such as 24kB with 8kB alignment, the small allocator
|
||
did not always guarantee alignment. Let's fix that.
|
||
|
||
* bmalloc/Algorithm.h:
|
||
(bmalloc::divideRoundingUp): Math is hard.
|
||
|
||
* bmalloc/Allocator.cpp:
|
||
(bmalloc::Allocator::allocate): Align to the page size unconditionally.
|
||
Even if the page size is not a power of two, it might be a multiple of
|
||
a power of two, and we want alignment to that smaller power of two to
|
||
be guaranteed.
|
||
|
||
2016-04-06 Geoffrey Garen <ggaren@apple.com>
|
||
|
||
bmalloc: handle aligned allocations on the fast path
|
||
https://bugs.webkit.org/show_bug.cgi?id=156302
|
||
|
||
Reviewed by Michael Saboff.
|
||
|
||
This helps keep the JavaScriptCore GC on the fast path, and it also
|
||
helps avoid fragmentation on our website stress test:
|
||
|
||
nimlang 209,584kB 198,076kB ^ 1.06x smaller
|
||
|
||
* bmalloc/Allocator.cpp:
|
||
(bmalloc::Allocator::allocate): Because we arrange for power-of-two size
|
||
classes to allocate at power-of-two alignments, we can allocate any
|
||
small aligned request on the small path.
|
||
|
||
* bmalloc/Chunk.h:
|
||
(bmalloc::Chunk::bytes):
|
||
(bmalloc::Chunk::lines):
|
||
(bmalloc::Chunk::pages):
|
||
(bmalloc::Chunk::boundaryTags):
|
||
(bmalloc::Chunk::objectType): Moved some code around to provide better
|
||
API.
|
||
|
||
(bmalloc::Chunk::Chunk): Moved this code to VMHeap.
|
||
|
||
(bmalloc::Chunk::offset):
|
||
(bmalloc::Chunk::object): Use our new bytes() helper function.
|
||
|
||
* bmalloc/VMHeap.cpp:
|
||
(bmalloc::VMHeap::allocateChunk): Moved code here from Chunk.
|
||
|
||
(bmalloc::VMHeap::allocateSmallChunk): Ensure that power-of-two page
|
||
sizes always begin allocation at the same alignment. Power-of-two object
|
||
sizes always request power-of-two page sizes (since that's the least
|
||
wasteful option), so if we also ensure that power-of-two page sizes get
|
||
power-of-two alignment, then everything is aligned for all small objects.
|
||
|
||
2016-04-03 Geoffrey Garen <ggaren@apple.com>
|
||
|
||
bmalloc: segregate small and large objects again, and allocate more objects on the small path
|
||
https://bugs.webkit.org/show_bug.cgi?id=156152
|
||
|
||
Reviewed by Sam Weinig.
|
||
|
||
Microbenchmark data suggested that it was a good idea for small and large
|
||
objects to share memory. But r198675 did not improve memory use in
|
||
full browser benchmarks.
|
||
|
||
This patch reverts to segregating small and large objects -- but without
|
||
going back to doubled VM usage -- in order to capture a few benefits:
|
||
|
||
(*) Small pages fragment the large heap. Separating them out saves a lot
|
||
of memory in our worst case fragmentation recording:
|
||
|
||
nimlang 276,076kB 209,636kB ^ 1.32x smaller
|
||
|
||
(*) Small objects are common enough that even their slow paths benefit
|
||
from simpler code:
|
||
|
||
Execution Time:
|
||
...
|
||
facebook 234ms 216ms ^ 1.08x faster
|
||
reddit 114ms 108ms ^ 1.06x faster
|
||
flickr 118ms 111ms ^ 1.06x faster
|
||
theverge 146ms 140ms ^ 1.04x faster
|
||
...
|
||
<arithmetic mean> 107ms 102ms ^ 1.04x faster
|
||
|
||
(*) We can use less metadata:
|
||
|
||
Memory at End:
|
||
...
|
||
list_allocate 460kB 384kB ^ 1.2x smaller
|
||
tree_allocate 492kB 424kB ^ 1.16x smaller
|
||
tree_churn 480kB 404kB ^ 1.19x smaller
|
||
fragment 532kB 452kB ^ 1.18x smaller
|
||
fragment_iterate 712kB 588kB ^ 1.21x smaller
|
||
medium 15,152kB 11,796kB ^ 1.28x smaller
|
||
big 15,044kB 10,976kB ^ 1.37x smaller
|
||
...
|
||
<arithmetic mean> 7,724kB 7,190kB ^ 1.07x smaller
|
||
|
||
This patch also takes advantage of our support for varying the page size
|
||
at runtime by allocating more objects on the small object path:
|
||
|
||
medium 178ms 150ms ^ 1.19x faster
|
||
|
||
Some microbenchmarks report memory use increases from this change -- like
|
||
they reported memory use decreases from r198675 -- but I'm ignoring them
|
||
for now because I expect our full browser memory benchmarks to confirm
|
||
that this patch is fine.
|
||
|
||
* bmalloc/BumpAllocator.h:
|
||
(bmalloc::BumpAllocator::BumpAllocator): Use a full unsigned because we
|
||
can allocate objects larger than 16kB - 1, and a full unsigned does not
|
||
make BumpAllocator any larger on 64bit systems.
|
||
|
||
* bmalloc/Chunk.h:
|
||
(bmalloc::Chunk::begin):
|
||
(bmalloc::Chunk::end):
|
||
(bmalloc::Chunk::size):
|
||
(bmalloc::Chunk::objectType): Store ObjectType in the Chunk, since it only
|
||
varies by Chunk now, and not from page to page within a Chunk. Also,
|
||
union together small and large object metadata, since we will only use
|
||
one or the other. This saves memory.
|
||
|
||
(bmalloc::Chunk::Chunk): Conditionalize initialization based on object
|
||
type, since only one kind of metadata or the other can be used at runtime.
|
||
|
||
(bmalloc::Object::Object):
|
||
(bmalloc::Object::begin):
|
||
(bmalloc::SmallPage::end): Deleted.
|
||
|
||
* bmalloc/Heap.cpp:
|
||
(bmalloc::Heap::Heap):
|
||
(bmalloc::Heap::initializeLineMetadata): Save a little space, since we
|
||
know that lines are only 256 bytes long.
|
||
|
||
(bmalloc::Heap::initializePageMetadata): Store a dynamic page size for
|
||
each size class. We used to use only one page size (the system page size)
|
||
but that limited our ability to allocate objects larger than 1kB on the
|
||
small object path. Now we can handle any object size we want by storing
|
||
objects of that size in a custom page size.
|
||
|
||
(bmalloc::Heap::concurrentScavenge):
|
||
(bmalloc::Heap::scavenge):
|
||
(bmalloc::Heap::scavengeSmallPages): Revert to our old linked list
|
||
strategy for storing small pages.
|
||
|
||
(bmalloc::Heap::splitAndAllocate): Object type is per Chunk now.
|
||
|
||
(bmalloc::Heap::allocateLarge): Don't nuke the small page list when
|
||
allocating a large object because the two don't share memory anymore.
|
||
|
||
(bmalloc::Heap::allocateSmallPage): Revert to our old linked list
|
||
strategy for storing small pages.
|
||
|
||
(bmalloc::Heap::deallocateSmallLine): Don't return early in the case
|
||
where this is the first free object in the page. In the case of large-ish
|
||
objects, the first free object might also be the last free object,
|
||
since there's one object per page.
|
||
|
||
(bmalloc::Heap::allocateSmallBumpRangesByMetadata): Split out some helper
|
||
lambdas to make this code clearer.
|
||
|
||
(bmalloc::Heap::allocateSmallBumpRangesByObject): Added a fast scan
|
||
for objects larger than the line size. When multiple objects fit in
|
||
a single line, it's an optimization to scan a line at a time. But when
|
||
it's one object per line, or one object per 64 lines, it's better just
|
||
to scan an object at a time.
|
||
|
||
* bmalloc/Heap.h:
|
||
(bmalloc::Heap::allocateSmallBumpRanges):
|
||
(bmalloc::Heap::derefSmallLine): Match the changes above.
|
||
|
||
* bmalloc/LineMetadata.h: We weren't using all those bits.
|
||
|
||
* bmalloc/List.h:
|
||
(bmalloc::List::remove): Put a removed Node fully back into the default
|
||
(empty) state it was in before it entered the list. This change is not
|
||
observable, but it makes things clearer when you're debugging.
|
||
|
||
* bmalloc/Object.h:
|
||
(bmalloc::Object::Object):
|
||
(bmalloc::Object::chunk):
|
||
(bmalloc::Object::offset):
|
||
(bmalloc::Object::operator+):
|
||
(bmalloc::Object::operator<=): Added some helpers for iterating by object.
|
||
|
||
* bmalloc/ObjectType.cpp:
|
||
(bmalloc::objectType): Updated for API change.
|
||
|
||
* bmalloc/Sizes.h:
|
||
(bmalloc::Sizes::maskObjectSize):
|
||
(bmalloc::Sizes::objectSize):
|
||
(bmalloc::Sizes::pageSize): Support more page sizes.
|
||
|
||
* bmalloc/SmallPage.h:
|
||
(bmalloc::SmallPage::SmallPage):
|
||
(bmalloc::SmallPage::objectType): Deleted.
|
||
(bmalloc::SmallPage::setObjectType): Deleted.
|
||
(bmalloc::SmallPage::smallPageCount): Deleted.
|
||
(bmalloc::SmallPage::setSmallPageCount): Deleted. Object type is per
|
||
Chunk now, and we can infer page count from size class.
|
||
|
||
* bmalloc/VMHeap.cpp:
|
||
(bmalloc::VMHeap::allocateChunk):
|
||
(bmalloc::VMHeap::allocateSmallChunk):
|
||
* bmalloc/VMHeap.h:
|
||
(bmalloc::VMHeap::allocateSmallPage):
|
||
(bmalloc::VMHeap::deallocateSmallPage):
|
||
(bmalloc::VMHeap::allocateLargeObject): Support our old behavior of
|
||
storing free pages in linked lists.
|
||
|
||
2016-03-29 Geoffrey Garen <ggaren@apple.com>
|
||
|
||
bmalloc: support physical page sizes that don't match the virtual page size (take 2)
|
||
https://bugs.webkit.org/show_bug.cgi?id=156003
|
||
|
||
Reviewed by Andreas Kling.
|
||
|
||
This is a memory savings on iOS devices where the virtual page size
|
||
is 16kB but the physical page size is 4kB.
|
||
|
||
Take 1 was a memory regression on 16kB virtual / 16kB physical systems
|
||
because it used a 4kB page size within a 16kB page size, allowing up to
|
||
4 different object types to mix within a physical page. Because objects
|
||
of the same type tend to deallocate at the same time, mixing objects of
|
||
different types made pages less likely to become completely empty.
|
||
|
||
(Take 1 also had a bug where it used a platform #ifdef that didn't exist.
|
||
Oops.)
|
||
|
||
Take 2 allocates units of SmallPages equal to the physical page size.
|
||
|
||
* bmalloc/Heap.cpp:
|
||
(bmalloc::Heap::Heap):
|
||
(bmalloc::Heap::initializeLineMetadata):
|
||
(bmalloc::Heap::allocateSmallBumpRanges):
|
||
(bmalloc::Heap::allocateSmallPage):
|
||
(bmalloc::Heap::allocateLarge):
|
||
(bmalloc::Heap::splitAndAllocate):
|
||
(bmalloc::Heap::tryAllocateXLarge):
|
||
(bmalloc::Heap::shrinkXLarge):
|
||
* bmalloc/Heap.h: Use the physical page size for our VM operations because
|
||
we're only concerned with returning physical pages to the OS.
|
||
|
||
* bmalloc/VMAllocate.h:
|
||
(bmalloc::vmPageSize):
|
||
(bmalloc::vmPageShift):
|
||
(bmalloc::vmSize):
|
||
(bmalloc::vmValidate):
|
||
(bmalloc::vmPageSizePhysical):
|
||
(bmalloc::vmValidatePhysical):
|
||
(bmalloc::tryVMAllocate):
|
||
(bmalloc::vmDeallocatePhysicalPages):
|
||
(bmalloc::vmAllocatePhysicalPages):
|
||
(bmalloc::vmDeallocatePhysicalPagesSloppy):
|
||
(bmalloc::vmAllocatePhysicalPagesSloppy): Use the physical page size.
|
||
|
||
2016-03-29 Geoffrey Garen <ggaren@apple.com>
|
||
|
||
bmalloc: page size should be configurable at runtime
|
||
https://bugs.webkit.org/show_bug.cgi?id=155993
|
||
|
||
Reviewed by Andreas Kling.
|
||
|
||
This is a memory win on 32bit iOS devices, since their page sizes are
|
||
4kB and not 16kB.
|
||
|
||
It's also a step toward supporting 64bit iOS devices that have a
|
||
16kB/4kB virtual/physical page size split.
|
||
|
||
* bmalloc/Chunk.h: Align to largeAlignment since 2 * smallMax isn't
|
||
required by the boundary tag allocator.
|
||
|
||
(bmalloc::Chunk::page): Account for the slide when accessing a page.
|
||
Each SmallPage hashes 4kB of memory. When we want to allocate a region
|
||
of memory larger than 4kB, we store our metadata in the first SmallPage
|
||
in the region and we assign a slide to the remaining SmallPages, so
|
||
they forward to that first SmallPage when accessed.
|
||
|
||
NOTE: We could use a less flexible technique that just hashed by
|
||
vmPageSize() instead of 4kB at runtime, with no slide, but I think we'll
|
||
be able to use this slide technique to make even more page sizes
|
||
dynamically at runtime, which should save some memory and simplify
|
||
the allocator.
|
||
|
||
(bmalloc::SmallPage::begin): It's invalid to access a SmallPage with
|
||
a slide, since such SmallPages do not contain meaningful data.
|
||
|
||
(bmalloc::SmallPage::end): Account for smallPageCount when computing
|
||
the size of a page.
|
||
|
||
(bmalloc::Chunk::pageBegin): Deleted.
|
||
(bmalloc::Chunk::pageEnd): Deleted.
|
||
(bmalloc::Object::pageBegin): Deleted.
|
||
|
||
* bmalloc/Heap.cpp:
|
||
(bmalloc::Heap::Heap): Cache vmPageSize because computing it might require
|
||
a syscall.
|
||
|
||
(bmalloc::Heap::initializeLineMetadata): Line metadata is a vector instead
|
||
of a 2D array because we don't know how much metadata we'll need until
|
||
we know the page size.
|
||
|
||
(bmalloc::Heap::scavengeSmallPage): Be sure to revert the slide when
|
||
deallocating a page. Otherwise, the next attempt to allocate the page
|
||
will slide when initializing it, sliding to nowhere.
|
||
|
||
(bmalloc::Heap::allocateSmallBumpRanges): Account for vector change to
|
||
line metadata.
|
||
|
||
(bmalloc::Heap::allocateSmallPage): Initialize slide and smallPageCount
|
||
since they aren't constant anymore.
|
||
|
||
(bmalloc::Heap::allocateLarge):
|
||
(bmalloc::Heap::splitAndAllocate):
|
||
(bmalloc::Heap::tryAllocateXLarge):
|
||
(bmalloc::Heap::shrinkXLarge): Adopt dynamic page size.
|
||
|
||
* bmalloc/Heap.h:
|
||
|
||
* bmalloc/Sizes.h: smallPageSize is no longer equal to the VM page
|
||
size -- it's just the smallest VM page size we're interested in supporting.
|
||
|
||
* bmalloc/SmallPage.h:
|
||
(bmalloc::SmallPage::slide):
|
||
(bmalloc::SmallPage::setSlide):
|
||
(bmalloc::SmallPage::smallPageCount):
|
||
(bmalloc::SmallPage::setSmallPageCount):
|
||
(bmalloc::SmallPage::ref):
|
||
(bmalloc::SmallPage::deref): Support slide and small page count as
|
||
dynamic values. This doesn't increase metadata size since sizeof(SmallPage)
|
||
rounds up to alignment anyway.
|
||
|
||
* bmalloc/VMAllocate.h:
|
||
(bmalloc::vmPageSize):
|
||
(bmalloc::vmPageShift):
|
||
(bmalloc::vmSize):
|
||
(bmalloc::vmValidate):
|
||
(bmalloc::tryVMAllocate):
|
||
(bmalloc::vmDeallocatePhysicalPagesSloppy):
|
||
(bmalloc::vmAllocatePhysicalPagesSloppy): Treat page size as a variable.
|
||
|
||
* bmalloc/Vector.h:
|
||
(bmalloc::Vector::initialCapacity):
|
||
(bmalloc::Vector<T>::insert):
|
||
(bmalloc::Vector<T>::grow):
|
||
(bmalloc::Vector<T>::shrink):
|
||
(bmalloc::Vector<T>::shrinkCapacity):
|
||
(bmalloc::Vector<T>::growCapacity): Treat page size as a variable.
|
||
|
||
2016-03-29 David Kilzer <ddkilzer@apple.com>
|
||
|
||
bmalloc: add logging for mmap() failures
|
||
<http://webkit.org/b/155409>
|
||
<rdar://problem/24568515>
|
||
|
||
Reviewed by Saam Barati.
|
||
|
||
This patch causes additional logging to be generated on internal
|
||
iOS builds when mmap() fails. We are trying to track down an
|
||
issue where the WebContent process runs out of VM address space
|
||
before it is killed by jetsam.
|
||
|
||
* CMakeLists.txt: Add Logging.cpp.
|
||
* bmalloc.xcodeproj/project.pbxproj: Add new files.
|
||
|
||
* bmalloc/BAssert.h:
|
||
(RELEASE_BASSERT_WITH_MESSAGE): Add macro.
|
||
* bmalloc/Logging.cpp: Added.
|
||
(bmalloc::logVMFailure): Implementation.
|
||
* bmalloc/Logging.h: Added.
|
||
(bmalloc::logVMFailure): Declaration.
|
||
* bmalloc/VMAllocate.h:
|
||
(bmalloc::tryVMAllocate): Call logVMFailure() on mmap() failure.
|
||
* bmalloc/darwin/BSoftLinking.h: Copied from Source/WebCore/platform/mac/SoftLinking.h.
|
||
|
||
2016-03-26 Geoffrey Garen <ggaren@apple.com>
|
||
|
||
Unreviewed, rolling out r198702, r198704.
|
||
|
||
Caused a memory regression on PLUM.
|
||
|
||
Reverted changeset:
|
||
|
||
bmalloc: fix an ASSERT on iOS
|
||
https://bugs.webkit.org/show_bug.cgi?id=155911
|
||
http://trac.webkit.org/changeset/198704
|
||
|
||
bmalloc: support physical page sizes that don't match the virtual page size
|
||
https://bugs.webkit.org/show_bug.cgi?id=155898
|
||
http://trac.webkit.org/changeset/198702
|
||
|
||
2016-03-25 Geoffrey Garen <ggaren@apple.com>
|
||
|
||
bmalloc: fix an ASSERT on iOS
|
||
https://bugs.webkit.org/show_bug.cgi?id=155911
|
||
|
||
Reviewed by Gavin Barraclough.
|
||
|
||
* bmalloc/VMAllocate.h:
|
||
(bmalloc::vmValidatePhysical): Call through to vmValidatePhysical because
|
||
the vmValidate function validates virtual sizes rather than physical
|
||
sizes.
|
||
|
||
2016-03-25 Geoffrey Garen <ggaren@apple.com>
|
||
|
||
bmalloc: support physical page sizes that don't match the virtual page size
|
||
https://bugs.webkit.org/show_bug.cgi?id=155898
|
||
|
||
Reviewed by Gavin Barraclough.
|
||
|
||
This is a memory savings on iOS devices where the virtual page size
|
||
is 16kB but the physical page size is 4kB.
|
||
|
||
* bmalloc/Chunk.h:
|
||
(bmalloc::Chunk::Chunk): smallPageSize is now unrelated to the OS's
|
||
page size -- it just reflects the optimal unit of memory to recycle
|
||
between small objects.
|
||
|
||
We only need to round up to largeAlignment because small objects allocate
|
||
as subsets of large objects now.
|
||
|
||
(bmalloc::Chunk::page):
|
||
(bmalloc::Object::pageBegin):
|
||
(bmalloc::Object::line): Adopt smallPageSize.
|
||
|
||
* bmalloc/Heap.cpp:
|
||
(bmalloc::Heap::initializeLineMetadata):
|
||
(bmalloc::Heap::allocateSmallPage):
|
||
(bmalloc::Heap::allocateLarge): Adopt smallPageSize.
|
||
|
||
(bmalloc::Heap::splitAndAllocate):
|
||
(bmalloc::Heap::tryAllocateXLarge):
|
||
(bmalloc::Heap::shrinkXLarge): Adopt vmPageSizePhysical(). We want the
|
||
physical page size because that's the unit at which the hardware MMU
|
||
will recycle memory.
|
||
|
||
* bmalloc/Sizes.h: Adopt smallPageSize.
|
||
|
||
* bmalloc/VMAllocate.h:
|
||
(bmalloc::vmPageSizePhysical):
|
||
(bmalloc::vmPageSize): Distinguish between page size, which is the virtual
|
||
memory page size advertised by the OS, and physical page size, which the
|
||
true hardware page size.
|
||
|
||
(bmalloc::vmSize):
|
||
(bmalloc::vmValidate):
|
||
(bmalloc::vmValidatePhysical):
|
||
(bmalloc::tryVMAllocate):
|
||
(bmalloc::vmDeallocatePhysicalPages):
|
||
(bmalloc::vmAllocatePhysicalPages):
|
||
(bmalloc::vmDeallocatePhysicalPagesSloppy):
|
||
(bmalloc::vmAllocatePhysicalPagesSloppy): Adopt vmPageSize() and
|
||
vmPageSizePhyiscal().
|
||
|
||
* bmalloc/Vector.h:
|
||
(bmalloc::Vector::initialCapacity):
|
||
(bmalloc::Vector<T>::shrink):
|
||
(bmalloc::Vector<T>::shrinkCapacity):
|
||
(bmalloc::Vector<T>::growCapacity): Adopt vmPageSize(). We'd prefer to
|
||
use vmPageSizePhysical() but mmap() doesn't support it.
|
||
|
||
* bmalloc/XLargeMap.cpp: #include.
|
||
|
||
2016-03-25 Geoffrey Garen <ggaren@apple.com>
|
||
|
||
Unreviewed, rolling in r198679.
|
||
|
||
r198679 was just a rename. The regression was caused by r198675 and then
|
||
fixed in r198693.
|
||
|
||
Restored changeset:
|
||
|
||
"bmalloc: Renamed LargeChunk => Chunk"
|
||
https://bugs.webkit.org/show_bug.cgi?id=155894
|
||
http://trac.webkit.org/changeset/198679
|
||
|
||
2016-03-25 Geoffrey Garen <ggaren@apple.com>
|
||
|
||
Unreviewed, try to fix a crash seen on the bots.
|
||
|
||
* bmalloc/Allocator.cpp:
|
||
(bmalloc::Allocator::reallocate): We have to take the lock even if we're
|
||
only reading our own data becuse LargeObject contains validation code
|
||
that will read our neighbors' data as well.
|
||
|
||
2016-03-25 Ryan Haddad <ryanhaddad@apple.com>
|
||
|
||
Unreviewed, rolling out r198679.
|
||
|
||
This change caused flaky LayoutTest crashes
|
||
|
||
Reverted changeset:
|
||
|
||
"bmalloc: Renamed LargeChunk => Chunk"
|
||
https://bugs.webkit.org/show_bug.cgi?id=155894
|
||
http://trac.webkit.org/changeset/198679
|
||
|
||
2016-03-25 Geoffrey Garen <ggaren@apple.com>
|
||
|
||
bmalloc: stress_aligned fails when allocating a zero-sized object with XLarge alignment
|
||
https://bugs.webkit.org/show_bug.cgi?id=155896
|
||
|
||
Reviewed by Andreas Kling.
|
||
|
||
We normally filter zero-sized allocations into small allocations, but
|
||
a zero-sized allocation can sneak through if it requires sufficiently
|
||
large alignment.
|
||
|
||
* bmalloc/Heap.cpp:
|
||
(bmalloc::Heap::tryAllocateXLarge): Set a floor on allocation size to
|
||
catch zero-sized allocations.
|
||
|
||
2016-03-25 Geoffrey Garen <ggaren@apple.com>
|
||
|
||
bmalloc: Renamed LargeChunk => Chunk
|
||
https://bugs.webkit.org/show_bug.cgi?id=155894
|
||
|
||
Reviewed by Michael Saboff.
|
||
|
||
A Chunk can contain both small and large objects now.
|
||
|
||
* bmalloc.xcodeproj/project.pbxproj:
|
||
* bmalloc/Allocator.cpp:
|
||
(bmalloc::Allocator::allocate):
|
||
* bmalloc/BoundaryTag.h:
|
||
(bmalloc::BoundaryTag::isFree):
|
||
* bmalloc/Chunk.h: Copied from Source/bmalloc/bmalloc/LargeChunk.h.
|
||
(bmalloc::Chunk::pages):
|
||
(bmalloc::Chunk::begin):
|
||
(bmalloc::Chunk::end):
|
||
(bmalloc::Chunk::Chunk):
|
||
(bmalloc::Chunk::get):
|
||
(bmalloc::Chunk::beginTag):
|
||
(bmalloc::Chunk::endTag):
|
||
(bmalloc::Chunk::offset):
|
||
(bmalloc::Chunk::object):
|
||
(bmalloc::Chunk::page):
|
||
(bmalloc::Chunk::line):
|
||
(bmalloc::SmallLine::begin):
|
||
(bmalloc::SmallPage::begin):
|
||
(bmalloc::SmallPage::end):
|
||
(bmalloc::Object::Object):
|
||
(bmalloc::Object::begin):
|
||
(bmalloc::LargeChunk::pages): Deleted.
|
||
(bmalloc::LargeChunk::begin): Deleted.
|
||
(bmalloc::LargeChunk::end): Deleted.
|
||
(bmalloc::LargeChunk::LargeChunk): Deleted.
|
||
(bmalloc::LargeChunk::get): Deleted.
|
||
(bmalloc::LargeChunk::beginTag): Deleted.
|
||
(bmalloc::LargeChunk::endTag): Deleted.
|
||
(bmalloc::LargeChunk::offset): Deleted.
|
||
(bmalloc::LargeChunk::object): Deleted.
|
||
(bmalloc::LargeChunk::page): Deleted.
|
||
(bmalloc::LargeChunk::line): Deleted.
|
||
* bmalloc/Deallocator.cpp:
|
||
* bmalloc/FreeList.cpp:
|
||
* bmalloc/Heap.cpp:
|
||
(bmalloc::Heap::allocateLarge):
|
||
* bmalloc/LargeChunk.h: Removed.
|
||
* bmalloc/LargeObject.h:
|
||
(bmalloc::LargeObject::LargeObject):
|
||
(bmalloc::LargeObject::merge):
|
||
(bmalloc::LargeObject::split):
|
||
* bmalloc/Object.h:
|
||
(bmalloc::Object::chunk):
|
||
* bmalloc/ObjectType.cpp:
|
||
* bmalloc/Sizes.h:
|
||
* bmalloc/SmallAllocator.h: Removed.
|
||
* bmalloc/VMHeap.cpp:
|
||
(bmalloc::VMHeap::VMHeap):
|
||
(bmalloc::VMHeap::allocateChunk):
|
||
(bmalloc::VMHeap::allocateLargeChunk): Deleted.
|
||
* bmalloc/VMHeap.h:
|
||
(bmalloc::VMHeap::allocateLargeObject):
|
||
(bmalloc::VMHeap::deallocateLargeObject):
|
||
* bmalloc/Zone.cpp:
|
||
(bmalloc::enumerator):
|
||
* bmalloc/Zone.h:
|
||
(bmalloc::Zone::chunks):
|
||
(bmalloc::Zone::addChunk):
|
||
(bmalloc::Zone::largeChunks): Deleted.
|
||
(bmalloc::Zone::addLargeChunk): Deleted.
|
||
|
||
2016-03-24 Geoffrey Garen <ggaren@apple.com>
|
||
|
||
bmalloc: small and large objects should share memory
|
||
https://bugs.webkit.org/show_bug.cgi?id=155866
|
||
|
||
Reviewed by Andreas Kling.
|
||
|
||
This patch cuts our VM footprint in half. (VM footprint usually doesn't
|
||
matter, but on iOS there's an artificial VM limit around 700MB, and if
|
||
you hit it you jetsam / crash.)
|
||
|
||
It's also a step toward honoring the hardware page size at runtime,
|
||
which will reduce memory usage on iOS.
|
||
|
||
This patch is a small improvement in peak memory usage because it allows
|
||
small and large objects to recycle each other's memory. The tradeoff is
|
||
that we require more metadata, which causes more memory usage after
|
||
shrinking down from peak memory usage. In the end, we have some memory
|
||
wins and some losses, and a small win in the mean on our standard memory
|
||
benchmarks.
|
||
|
||
* bmalloc.xcodeproj/project.pbxproj: Removed SuperChunk.
|
||
|
||
* bmalloc/Allocator.cpp:
|
||
(bmalloc::Allocator::reallocate): Adopt a new Heap API for shrinking
|
||
large objects because it's a little more complicated than it used to be.
|
||
|
||
Don't check for equality in the XLarge case because we don't do it in
|
||
other cases, and it's unlikely that we'll be called for no reason.
|
||
|
||
* bmalloc/BumpAllocator.h:
|
||
(bmalloc::BumpAllocator::allocate): Don't ASSERT isSmall because that's
|
||
an old concept from when small and large objects were in distinct memory
|
||
regions.
|
||
|
||
* bmalloc/Deallocator.cpp:
|
||
(bmalloc::Deallocator::deallocateSlowCase): Large objects are not
|
||
segregated anymore.
|
||
|
||
(bmalloc::Deallocator::deallocateLarge): Deleted.
|
||
|
||
* bmalloc/Deallocator.h:
|
||
(bmalloc::Deallocator::deallocateFastCase): Don't ASSERT isSmall(). See
|
||
above.
|
||
|
||
* bmalloc/Heap.cpp:
|
||
(bmalloc::Heap::scavenge):
|
||
(bmalloc::Heap::scavengeSmallPage):
|
||
(bmalloc::Heap::scavengeSmallPages): New helpers for returning cached
|
||
small pages to the large object heap.
|
||
|
||
(bmalloc::Heap::allocateSmallPage): Allocate small pages from the large
|
||
object heap. This is how we accomplish sharing.
|
||
|
||
(bmalloc::Heap::deallocateSmallLine): Handle large objects since we can
|
||
encounter them on this code path now.
|
||
|
||
(bmalloc::Heap::splitAndAllocate): Fixed a bug where we would sometimes
|
||
not split even though we could.
|
||
|
||
Allocating a large object also requires ref'ing its small line so that
|
||
we can alias memory between small and large objects.
|
||
|
||
(bmalloc::Heap::allocateLarge): Return cached small pages before
|
||
allocating a large object that would fit in a cached small page. This
|
||
allows some large allocations to reuse small object memory.
|
||
|
||
(bmalloc::Heap::shrinkLarge): New helper.
|
||
|
||
(bmalloc::Heap::deallocateLarge): Deleted.
|
||
|
||
* bmalloc/Heap.h:
|
||
|
||
* bmalloc/LargeChunk.h:
|
||
(bmalloc::LargeChunk::pageBegin):
|
||
(bmalloc::LargeChunk::pageEnd):
|
||
(bmalloc::LargeChunk::lines):
|
||
(bmalloc::LargeChunk::pages):
|
||
(bmalloc::LargeChunk::begin):
|
||
(bmalloc::LargeChunk::end):
|
||
(bmalloc::LargeChunk::LargeChunk):
|
||
(bmalloc::LargeChunk::get):
|
||
(bmalloc::LargeChunk::endTag):
|
||
(bmalloc::LargeChunk::offset):
|
||
(bmalloc::LargeChunk::object):
|
||
(bmalloc::LargeChunk::page):
|
||
(bmalloc::LargeChunk::line):
|
||
(bmalloc::SmallLine::begin):
|
||
(bmalloc::SmallLine::end):
|
||
(bmalloc::SmallPage::begin):
|
||
(bmalloc::SmallPage::end):
|
||
(bmalloc::Object::Object):
|
||
(bmalloc::Object::begin):
|
||
(bmalloc::Object::pageBegin):
|
||
(bmalloc::Object::line):
|
||
(bmalloc::Object::page): I merged all the SmallChunk metadata and code
|
||
into LargeChunk. Now we use a single class to track both small and large
|
||
metadata, so we can share memory between small and large objects.
|
||
|
||
I'm going to rename this class to Chunk in a follow-up patch.
|
||
|
||
* bmalloc/Object.h:
|
||
(bmalloc::Object::chunk): Updated for LargeChunk transition.
|
||
|
||
* bmalloc/ObjectType.cpp:
|
||
(bmalloc::objectType):
|
||
* bmalloc/ObjectType.h:
|
||
(bmalloc::isXLarge):
|
||
(bmalloc::isSmall): Deleted. The difference between small and large
|
||
objects is now stored in metadata and is not a property of their
|
||
virtual address range.
|
||
|
||
* bmalloc/SegregatedFreeList.h: One more entry because we cover all of
|
||
what used to be the super chunk in a large chunk now.
|
||
|
||
* bmalloc/Sizes.h: Removed bit masking helpers because we don't use
|
||
address masks to distinguish small vs large object type anymore.
|
||
|
||
* bmalloc/SmallChunk.h: Removed.
|
||
|
||
* bmalloc/SmallPage.h:
|
||
(bmalloc::SmallPage::SmallPage): Store object type per page because any
|
||
given page can be used for large objects or small objects.
|
||
|
||
* bmalloc/SuperChunk.h: Removed.
|
||
|
||
* bmalloc/VMHeap.cpp:
|
||
(bmalloc::VMHeap::VMHeap):
|
||
(bmalloc::VMHeap::allocateLargeChunk):
|
||
(bmalloc::VMHeap::allocateSmallChunk): Deleted.
|
||
(bmalloc::VMHeap::allocateSuperChunk): Deleted.
|
||
* bmalloc/VMHeap.h:
|
||
(bmalloc::VMHeap::allocateLargeObject):
|
||
(bmalloc::VMHeap::deallocateLargeObject):
|
||
(bmalloc::VMHeap::allocateSmallPage): Deleted.
|
||
(bmalloc::VMHeap::deallocateSmallPage): Deleted. Removed super chunk and
|
||
small chunk support.
|
||
|
||
* bmalloc/Zone.cpp:
|
||
(bmalloc::enumerator):
|
||
* bmalloc/Zone.h:
|
||
(bmalloc::Zone::largeChunks):
|
||
(bmalloc::Zone::addLargeChunk):
|
||
(bmalloc::Zone::superChunks): Deleted.
|
||
(bmalloc::Zone::addSuperChunk): Deleted. Removed super chunk and
|
||
small chunk support.
|
||
|
||
2016-03-23 Geoffrey Garen <ggaren@apple.com>
|
||
|
||
bmalloc: Added an Object helper class
|
||
https://bugs.webkit.org/show_bug.cgi?id=155818
|
||
|
||
Reviewed by Gavin Barraclough.
|
||
|
||
Object is an abstraction that breaks out a void* into its component
|
||
metadata pointers.
|
||
|
||
This is slightly faster than recomputing them, and it enables a future
|
||
patch in which Object will tell us whether it is small or large.
|
||
|
||
* bmalloc.xcodeproj/project.pbxproj: Added to the project.
|
||
|
||
* bmalloc/Allocator.cpp:
|
||
(bmalloc::Allocator::reallocate): Use Object to compute size.
|
||
|
||
* bmalloc/Deallocator.cpp:
|
||
(bmalloc::Deallocator::processObjectLog):
|
||
* bmalloc/Heap.cpp:
|
||
(bmalloc::Heap::allocateSmallPage):
|
||
(bmalloc::Heap::deallocateSmallLine):
|
||
* bmalloc/Heap.h:
|
||
(bmalloc::Heap::derefSmallLine): Use Object to deallocate.
|
||
|
||
* bmalloc/Object.h: Added.
|
||
(bmalloc::Object::Object):
|
||
(bmalloc::Object::chunk):
|
||
(bmalloc::Object::line):
|
||
(bmalloc::Object::page): Helper class to break out a void* into its
|
||
component metadata pointers.
|
||
|
||
* bmalloc/SmallChunk.h:
|
||
(bmalloc::SmallChunk::SmallChunk): SmallPage::get doesn't exist anymore
|
||
so we use our new helper functions instead.
|
||
|
||
(bmalloc::SmallChunk::offset):
|
||
(bmalloc::SmallChunk::object):
|
||
(bmalloc::SmallChunk::page):
|
||
(bmalloc::SmallChunk::line):
|
||
(bmalloc::SmallLine::begin):
|
||
(bmalloc::SmallLine::end):
|
||
(bmalloc::SmallPage::begin): New helpers that operate on the data
|
||
stored in Object.
|
||
|
||
(bmalloc::SmallLine::get): Deleted.
|
||
(bmalloc::SmallPage::get): Deleted.
|
||
|
||
* bmalloc/SmallLine.h:
|
||
(bmalloc::SmallLine::refCount): Added a default ref value for convenience.
|
||
|
||
* bmalloc/SmallPage.h:
|
||
(bmalloc::SmallPage::SmallPage):
|
||
|
||
2016-03-23 Geoffrey Garen <ggaren@apple.com>
|
||
|
||
bmalloc: process the object log before asking for new memory
|
||
https://bugs.webkit.org/show_bug.cgi?id=155801
|
||
|
||
Reviewed by Gavin Barraclough.
|
||
|
||
This is a step toward merging large and small objects: In future, if we
|
||
have large objects in the log, we need to process them right away to
|
||
avoid pushing up peak memory use.
|
||
|
||
But it also appears to be a speedup and memory use improvement now.
|
||
|
||
* bmalloc/Allocator.cpp:
|
||
(bmalloc::Allocator::allocate):
|
||
(bmalloc::Allocator::refillAllocatorSlowCase):
|
||
(bmalloc::Allocator::allocateLarge): Process the log before asking for
|
||
more memory.
|
||
|
||
* bmalloc/Deallocator.cpp:
|
||
(bmalloc::Deallocator::processObjectLog):
|
||
(bmalloc::Deallocator::deallocateSlowCase):
|
||
* bmalloc/Deallocator.h: Provide a public API for processing the object log.
|
||
|
||
* bmalloc/Heap.cpp:
|
||
(bmalloc::Heap::allocateSmallPage): Pop fragmented pages from the front
|
||
instead of from the back. This resolves a regression on tree_churn
|
||
--parallel. Popping from the front gives us the oldest pages. The oldest
|
||
pages have had the most time to accumulate free lines. They are therefore
|
||
the least fragmented on average.
|
||
|
||
* bmalloc/List.h:
|
||
(bmalloc::List::popFront):
|
||
(bmalloc::List::insertAfter): New API to pop from front.
|
||
|
||
2016-03-22 Geoffrey Garen <ggaren@apple.com>
|
||
|
||
bmalloc: use a log scale for large-ish size classes
|
||
https://bugs.webkit.org/show_bug.cgi?id=155770
|
||
|
||
Reviewed by Michael Saboff.
|
||
|
||
At larger sizes, precise allocation sizes don't save much memory -- and
|
||
they can cost memory when objects of distinct size classes can't
|
||
allocate together.
|
||
|
||
This is a small savings up to our current allocation limits, and it may
|
||
enable changing those limits in the long term.
|
||
|
||
* bmalloc/Algorithm.h:
|
||
(bmalloc::log2): We use this to compute large-ish size classes.
|
||
|
||
* bmalloc/Allocator.cpp:
|
||
(bmalloc::Allocator::Allocator): Iterate by size class instead of by
|
||
object size so we can change object size limits without breaking stuff.
|
||
|
||
(bmalloc::Allocator::scavenge): Ditto.
|
||
|
||
(bmalloc::Allocator::allocateLogSizeClass): New helper function for
|
||
allocating based on log size classes.
|
||
|
||
(bmalloc::Allocator::allocateSlowCase): Account for extra size class
|
||
possibilities.
|
||
|
||
* bmalloc/Allocator.h:
|
||
(bmalloc::Allocator::allocateFastCase): We only handle up to 512b on
|
||
the fastest fast path now.
|
||
|
||
* bmalloc/BumpAllocator.h:
|
||
(bmalloc::BumpAllocator::validate): Deleted. I noticed that this function
|
||
had been refactored not to do anything anymore.
|
||
|
||
* bmalloc/Heap.cpp:
|
||
(bmalloc::Heap::initializeLineMetadata): Iterate by size class. (See
|
||
Allocator::Allocator.)
|
||
|
||
* bmalloc/Heap.h: Use the sizeClassCount constant instead of hard coding
|
||
things.
|
||
|
||
* bmalloc/Sizes.h:
|
||
(bmalloc::Sizes::maskSizeClass):
|
||
(bmalloc::Sizes::maskObjectSize):
|
||
(bmalloc::Sizes::logSizeClass):
|
||
(bmalloc::Sizes::logObjectSize):
|
||
(bmalloc::Sizes::sizeClass):
|
||
(bmalloc::Sizes::objectSize): Separate size class calculation between
|
||
simple size classes that can be computed with a mask and are 8-byte-precise
|
||
and complex size classes that require more math and are less precise.
|
||
|
||
* bmalloc/SmallLine.h:
|
||
(bmalloc::SmallLine::ref):
|
||
* bmalloc/SmallPage.h:
|
||
(bmalloc::SmallPage::SmallPage):
|
||
(bmalloc::SmallPage::ref):
|
||
(bmalloc::SmallPage::deref): Cleaned up some ASSERTs that triggered
|
||
while working on this patch.
|
||
|
||
* bmalloc/Zone.cpp:
|
||
(bmalloc::statistics):
|
||
(bmalloc::zoneSize):
|
||
(bmalloc::Zone::Zone):
|
||
(bmalloc::size): Deleted. Renamed these symbols to work around an lldb
|
||
bug that makes it impossible to print out variables named 'size' -- which
|
||
can be a problem when working on malloc.
|
||
|
||
2016-03-22 Geoffrey Garen <ggaren@apple.com>
|
||
|
||
bmalloc: shrink largeMax
|
||
https://bugs.webkit.org/show_bug.cgi?id=155759
|
||
|
||
Reviewed by Michael Saboff.
|
||
|
||
If a largeChunk contains N bytes and we allocate objects of size
|
||
N / 2 + 8 bytes, then we waste 50% of physical memory at peak.
|
||
|
||
This patch sets largeMax to N / 2, reducing maximum waste to 25%.
|
||
|
||
* bmalloc/BoundaryTag.h:
|
||
* bmalloc/LargeChunk.h:
|
||
(bmalloc::LargeChunk::LargeChunk):
|
||
* bmalloc/SegregatedFreeList.cpp:
|
||
(bmalloc::SegregatedFreeList::SegregatedFreeList):
|
||
(bmalloc::SegregatedFreeList::insert): Honor largeMax vs largeObjectMax.
|
||
|
||
* bmalloc/Sizes.h: Distinguish between the largest thing we can store
|
||
in a free list (largeObjectMax) and the largest thing we're willing to
|
||
allocate (largeMax).
|
||
|
||
2016-03-20 Dan Bernstein <mitz@apple.com>
|
||
|
||
[Mac] Determine TARGET_MAC_OS_X_VERSION_MAJOR from MACOSX_DEPLOYMENT_TARGET rather than from MAC_OS_X_VERSION_MAJOR
|
||
https://bugs.webkit.org/show_bug.cgi?id=155707
|
||
<rdar://problem/24980691>
|
||
|
||
Reviewed by Darin Adler.
|
||
|
||
* Configurations/Base.xcconfig: Set TARGET_MAC_OS_X_VERSION_MAJOR based on the last
|
||
component of MACOSX_DEPLOYMENT_TARGET.
|
||
* Configurations/DebugRelease.xcconfig: For engineering builds, preserve the behavior of
|
||
TARGET_MAC_OS_X_VERSION_MAJOR being the host’s OS version.
|
||
|
||
2016-03-20 Dan Bernstein <mitz@apple.com>
|
||
|
||
Update build settings
|
||
|
||
Rubber-stamped by Andy Estes.
|
||
|
||
* Configurations/DebugRelease.xcconfig:
|
||
|
||
2016-03-14 Geoffrey Garen <ggaren@apple.com>
|
||
|
||
Unreviewed, rolling out r197955.
|
||
|
||
I decided to go in another direction
|
||
|
||
Reverted changeset:
|
||
|
||
"bmalloc: Rename SmallPage to SmallRun"
|
||
https://bugs.webkit.org/show_bug.cgi?id=155320
|
||
http://trac.webkit.org/changeset/197955
|
||
|
||
2016-03-10 Geoffrey Garen <ggaren@apple.com>
|
||
|
||
bmalloc: Rename SmallPage to SmallRun
|
||
https://bugs.webkit.org/show_bug.cgi?id=155320
|
||
|
||
Reviewed by Alex Christensen.
|
||
|
||
A page is a fixed-size set of lines.
|
||
|
||
A run is an variable-sized set of lines.
|
||
|
||
We want to start using runs because:
|
||
|
||
(a) we want to support varying the hardware page size by OS;
|
||
|
||
(b) we want to support allocations larger than our current page size.
|
||
|
||
* bmalloc.xcodeproj/project.pbxproj:
|
||
* bmalloc/Allocator.cpp:
|
||
(bmalloc::Allocator::reallocate):
|
||
* bmalloc/Heap.cpp:
|
||
(bmalloc::Heap::Heap):
|
||
(bmalloc::Heap::initializeSmallRunMetadata):
|
||
(bmalloc::Heap::scavenge):
|
||
(bmalloc::Heap::scavengeSmallRuns):
|
||
(bmalloc::Heap::allocateSmallBumpRanges):
|
||
(bmalloc::Heap::allocateSmallRun):
|
||
(bmalloc::Heap::deallocateSmallLine):
|
||
(bmalloc::Heap::initializeLineMetadata): Deleted.
|
||
(bmalloc::Heap::scavengeSmallPages): Deleted.
|
||
(bmalloc::Heap::allocateSmallPage): Deleted.
|
||
* bmalloc/Heap.h:
|
||
* bmalloc/LineMetadata.h:
|
||
* bmalloc/SmallChunk.h:
|
||
(bmalloc::SmallChunk::begin):
|
||
(bmalloc::SmallChunk::end):
|
||
(bmalloc::SmallChunk::lines):
|
||
(bmalloc::SmallChunk::runs):
|
||
(bmalloc::SmallChunk::SmallChunk):
|
||
(bmalloc::SmallLine::end):
|
||
(bmalloc::SmallRun::get):
|
||
(bmalloc::SmallRun::begin):
|
||
(bmalloc::SmallRun::end):
|
||
(bmalloc::SmallChunk::pages): Deleted.
|
||
(bmalloc::SmallPage::get): Deleted.
|
||
(bmalloc::SmallPage::begin): Deleted.
|
||
(bmalloc::SmallPage::end): Deleted.
|
||
* bmalloc/SmallPage.h: Removed.
|
||
* bmalloc/SmallRun.h: Copied from Source/bmalloc/bmalloc/SmallPage.h.
|
||
(bmalloc::SmallRun::SmallRun):
|
||
(bmalloc::SmallRun::ref):
|
||
(bmalloc::SmallRun::deref):
|
||
(bmalloc::SmallPage::SmallPage): Deleted.
|
||
(bmalloc::SmallPage::ref): Deleted.
|
||
(bmalloc::SmallPage::deref): Deleted.
|
||
* bmalloc/VMHeap.cpp:
|
||
(bmalloc::VMHeap::allocateSmallChunk):
|
||
(bmalloc::VMHeap::allocateLargeChunk):
|
||
* bmalloc/VMHeap.h:
|
||
(bmalloc::VMHeap::allocateSmallRun):
|
||
(bmalloc::VMHeap::allocateLargeObject):
|
||
(bmalloc::VMHeap::deallocateSmallRun):
|
||
(bmalloc::VMHeap::deallocateLargeObject):
|
||
(bmalloc::VMHeap::allocateSmallPage): Deleted.
|
||
(bmalloc::VMHeap::deallocateSmallPage): Deleted.
|
||
|
||
2016-03-08 Geoffrey Garen <ggaren@apple.com>
|
||
|
||
Unreviewed, rolling in r197722.
|
||
https://bugs.webkit.org/show_bug.cgi?id=155171
|
||
|
||
The right calculation for our static_assert is actually:
|
||
|
||
sizeof(SmallChunk) % vmPageSize + 2 * smallMax <= vmPageSize
|
||
|
||
instead of:
|
||
|
||
sizeof(SmallChunk) % vmPageSize + smallMax <= vmPageSize
|
||
|
||
smallMax is not enough because line metadata might require us to begin
|
||
allocation at an offset as large as smallMax, so we need 2 * smallMax.
|
||
|
||
Once correct, this static_assert fires, and we fix it by increasing
|
||
the alignment of SmallChunk.
|
||
|
||
Restored changeset:
|
||
|
||
"bmalloc: Use List<T> instead of Vector<T> in some places"
|
||
https://bugs.webkit.org/show_bug.cgi?id=155150
|
||
http://trac.webkit.org/changeset/197722
|
||
|
||
2016-03-08 Commit Queue <commit-queue@webkit.org>
|
||
|
||
Unreviewed, rolling out r197722.
|
||
https://bugs.webkit.org/show_bug.cgi?id=155171
|
||
|
||
This change caused 800+ JSC test failures (Requested by
|
||
ryanhaddad on #webkit).
|
||
|
||
Reverted changeset:
|
||
|
||
"bmalloc: Use List<T> instead of Vector<T> in some places"
|
||
https://bugs.webkit.org/show_bug.cgi?id=155150
|
||
http://trac.webkit.org/changeset/197722
|
||
|
||
2016-03-07 Geoffrey Garen <ggaren@apple.com>
|
||
|
||
bmalloc: Use List<T> instead of Vector<T> in some places
|
||
https://bugs.webkit.org/show_bug.cgi?id=155150
|
||
|
||
Reviewed by Andreas Kling.
|
||
|
||
Vector<T> is expensive when you want a lot of them because our minimum
|
||
allocation size is the system page size.
|
||
|
||
* bmalloc.xcodeproj/project.pbxproj: Added a List<T> class.
|
||
|
||
* bmalloc/Heap.cpp:
|
||
(bmalloc::Heap::scavengeSmallPages):
|
||
(bmalloc::Heap::allocateSmallPage): Use the List<T> API. No need to check
|
||
for stale entries anymore because List<T> supports O(1) eager removal
|
||
and we remove eagerly now.
|
||
|
||
(bmalloc::Heap::deallocateSmallLine): Remove eagerly. This simplifies
|
||
the allocation code and it is also required for correctness since we
|
||
only have enough metadata to be in one list at a time.
|
||
|
||
* bmalloc/Heap.h: List!
|
||
|
||
* bmalloc/SmallChunk.h: Made this assert a little more precise since this
|
||
patch triggered the old version in a benign way.
|
||
|
||
(bmalloc::SmallChunk::SmallChunk): This code moved to the SmallPage
|
||
constructor.
|
||
|
||
* bmalloc/SmallPage.h:
|
||
(bmalloc::SmallPage::SmallPage): Accomodate the List<T> data structure.
|
||
This is a net memory savings on Mac for heaps smaller than ~128MB and on
|
||
iOS for heaps smaller than ~512MB. The maximum memory saved is 512kB on
|
||
Mac and 2MB on iOS. For larger heaps, there's a memory cost of 0.4% on
|
||
Mac and 0.1% on iOS.
|
||
|
||
* bmalloc/VMHeap.h:
|
||
(bmalloc::VMHeap::allocateSmallPage): Use List<T> API.
|
||
|
||
2016-03-03 Geoffrey Garen <ggaren@apple.com>
|
||
|
||
Unreviewed, rolling in r197174.
|
||
https://bugs.webkit.org/show_bug.cgi?id=154762
|
||
|
||
The right calculation for alignment is actually:
|
||
|
||
vmAlignment - getpagesize() + vmSize
|
||
|
||
instead of:
|
||
|
||
vmAlignment - vmPageSize + vmSize
|
||
|
||
The vmPageSize might be larger than getpagesize().
|
||
|
||
Restored changeset:
|
||
|
||
"bmalloc: Added a fast XLarge allocator"
|
||
https://bugs.webkit.org/show_bug.cgi?id=154720
|
||
http://trac.webkit.org/changeset/197174
|
||
|
||
2016-02-26 Commit Queue <commit-queue@webkit.org>
|
||
|
||
Unreviewed, rolling out r197174.
|
||
https://bugs.webkit.org/show_bug.cgi?id=154762
|
||
|
||
This change caused LayoutTests to crash on iOS simulator
|
||
(Requested by ryanhaddad on #webkit).
|
||
|
||
Reverted changeset:
|
||
|
||
"bmalloc: Added a fast XLarge allocator"
|
||
https://bugs.webkit.org/show_bug.cgi?id=154720
|
||
http://trac.webkit.org/changeset/197174
|
||
|
||
2016-02-25 Geoffrey Garen <ggaren@apple.com>
|
||
|
||
bmalloc: Added a fast XLarge allocator
|
||
https://bugs.webkit.org/show_bug.cgi?id=154720
|
||
|
||
Reviewed by Andreas Kling.
|
||
|
||
This is a big speedup for XLarge allocations because it avoids mmap
|
||
and page fault churn. It also enables future design changes to handle
|
||
a smaller size range on the fast path.
|
||
|
||
* bmalloc.xcodeproj/project.pbxproj:
|
||
|
||
* bmalloc/Algorithm.h:
|
||
(bmalloc::roundUpToMultipleOf):
|
||
(bmalloc::roundDownToMultipleOf): Added a non-constant round down.
|
||
|
||
* bmalloc/Allocator.cpp:
|
||
(bmalloc::Allocator::tryAllocate): XLarge no longer requires the caller
|
||
to align things.
|
||
|
||
(bmalloc::Allocator::allocate): Tweaked the alignment calculation for
|
||
clarity. When alignment and largeAlignment are equal, no adjustment
|
||
is necessary since all allocations guarantee largeAlignment.
|
||
|
||
(bmalloc::Allocator::reallocate): Updated for interface change.
|
||
|
||
Note that the new interface fixes some concurrency bugs. The old code
|
||
kept an iterator into the XLarge allocator across lock drop and acquisition,
|
||
which is not cool.
|
||
|
||
(bmalloc::Allocator::allocateXLarge): XLarge no longer requires the caller
|
||
to align things.
|
||
|
||
* bmalloc/Heap.cpp:
|
||
(bmalloc::Heap::scavengeXLargeObjects): Added scavenging for XLarge.
|
||
|
||
(bmalloc::Heap::allocateXLarge):
|
||
|
||
(bmalloc::Heap::splitAndAllocate): Split XLarge objects to xLargeAlignment.
|
||
|
||
(bmalloc::Heap::tryAllocateXLarge):
|
||
(bmalloc::Heap::xLargeSize):
|
||
(bmalloc::Heap::shrinkXLarge):
|
||
(bmalloc::Heap::deallocateXLarge): Allocate from our map before going
|
||
to the OS.
|
||
|
||
(bmalloc::Heap::findXLarge): Deleted.
|
||
|
||
* bmalloc/Heap.h:
|
||
|
||
* bmalloc/LargeObject.h:
|
||
(bmalloc::LargeObject::split):
|
||
|
||
* bmalloc/ObjectType.h:
|
||
(bmalloc::isXLarge): Give XLarge objects an explicit alignment for clarity.
|
||
|
||
* bmalloc/Range.h:
|
||
(bmalloc::Range::size):
|
||
(bmalloc::Range::operator!):
|
||
(bmalloc::Range::operator bool):
|
||
(bmalloc::Range::operator<):
|
||
(bmalloc::canMerge):
|
||
(bmalloc::merge): Some helpers that were useful in writing this patch.
|
||
|
||
* bmalloc/Sizes.h:
|
||
|
||
* bmalloc/SortedVector.h: Added.
|
||
(bmalloc::SortedVector::Bucket::Bucket):
|
||
(bmalloc::SortedVector::Bucket::operator<):
|
||
(bmalloc::SortedVector::iterator::iterator):
|
||
(bmalloc::SortedVector::iterator::operator++):
|
||
(bmalloc::SortedVector::iterator::operator!=):
|
||
(bmalloc::SortedVector::iterator::operator*):
|
||
(bmalloc::SortedVector::iterator::operator->):
|
||
(bmalloc::SortedVector::iterator::skipDeletedBuckets):
|
||
(bmalloc::SortedVector::begin):
|
||
(bmalloc::SortedVector::end):
|
||
(bmalloc::SortedVector<T>::insert):
|
||
(bmalloc::SortedVector<T>::find):
|
||
(bmalloc::SortedVector<T>::get):
|
||
(bmalloc::SortedVector<T>::take):
|
||
(bmalloc::SortedVector<T>::shrinkToFit): A simple abstraction for keeping
|
||
a sorted vector. Insertion is average amortized log(n) because we keep
|
||
deleted buckets that we can reuse.
|
||
|
||
This is better than a tree because we get better locality, less memory
|
||
use, and simpler code. Also, trees require a node memory allocator, and
|
||
implementing a memory allocator in a memory allocator is no fun.
|
||
|
||
Arguably we should use a hash table instead. But that's more code, and
|
||
sorted vector has other nice properties that we might want to take
|
||
adavantage of in the future.
|
||
|
||
* bmalloc/VMAllocate.h:
|
||
(bmalloc::tryVMAllocate): Fixed an inaccuracy in the alignment calculation
|
||
here. This code was sort of trying to enforce the alignment that the
|
||
XLarge allocator enforces -- but it's better to enforce that alignment
|
||
there.
|
||
|
||
The right calculation is:
|
||
|
||
vmAlignment - vmPageSize + vmSize
|
||
|
||
because the worst case is when you are aligned to 0 + vmPageSize, and
|
||
you must walk forward vmAlignment - vmPageSize to reach the next
|
||
vmAlignment.
|
||
|
||
(bmalloc::tryVMExtend): Deleted. No need to go back to the OS for VM
|
||
since we manage our own.
|
||
|
||
* bmalloc/VMHeap.cpp:
|
||
(bmalloc::VMHeap::allocateLargeChunk): Updated for clarity. When we
|
||
grow the large heap we know that grown region is where the next allocation
|
||
will take place, so we return it directly instead of pushing it to the
|
||
free list.
|
||
|
||
This fixes a subtle bug where an overly conservative aligned allocation
|
||
algorithm can fail to allocate at all when it grows the heap.
|
||
|
||
* bmalloc/VMHeap.h:
|
||
(bmalloc::VMHeap::allocateLargeObject): Ditto.
|
||
(bmalloc::VMHeap::allocateLargeObject): Ditto.
|
||
|
||
* bmalloc/VMState.h:
|
||
(bmalloc::merge): Added a helper.
|
||
|
||
* bmalloc/Vector.h:
|
||
(bmalloc::Vector::begin):
|
||
(bmalloc::Vector::end):
|
||
(bmalloc::Vector::size):
|
||
(bmalloc::Vector::capacity):
|
||
(bmalloc::Vector::last):
|
||
(bmalloc::Vector::pop):
|
||
(bmalloc::Vector<T>::push):
|
||
(bmalloc::Vector<T>::pop):
|
||
(bmalloc::Vector<T>::shrink): Use a proper iterator API to play nice
|
||
with std algorithms.
|
||
|
||
(bmalloc::Vector<T>::insert): New function required by SortedVector.
|
||
|
||
(bmalloc::Vector<T>::reallocateBuffer):
|
||
(bmalloc::Vector<T>::shrinkCapacity): Allow for shrinking back all the way
|
||
to 0 because that's what shrinkToFit wants.
|
||
(bmalloc::Vector<T>::growCapacity):
|
||
(bmalloc::Vector<T>::shrinkToFit):
|
||
|
||
* bmalloc/XLargeMap.cpp: Added. Helper data structure for managing XLarge
|
||
objects. We have enough granularity in our metadata to represent any
|
||
kind of address range.
|
||
|
||
We store free ranges in a flat vector because most programs have very
|
||
few individual free XLarge ranges. (They usually merge.)
|
||
|
||
We store allocated ranges in a sorted vector because programs might
|
||
allocate lots of XLarge ranges. For example, if the XLarge minimum is
|
||
128kB, and you have a 1GB process, that's 8192 ranges. Linear scan would
|
||
examine 8192 items but binary search only 13.
|
||
|
||
Empirically, this is 1.5X faster than our current large allocator if you
|
||
modify MallocBench/big to allocate XLarge objects and not to initialize
|
||
objects and you allocate 128kB-256kB objects in a 1GB address space.
|
||
|
||
(bmalloc::XLargeMap::takeFree): Be careful about overflow in this function
|
||
because we support super huge pointers, alignments, and sizes.
|
||
|
||
(bmalloc::XLargeMap::addFree): Merge eagerly on free because the cost
|
||
of missing an XLarge opportunity is catastrophic. Also, I discovered
|
||
by experiment that any allocator that doesn't merge eagerly can create
|
||
lots of subtle opportunities for snowballing fragmentation, as
|
||
fragmentation in range A forces you to chop up range B, and so on.
|
||
|
||
We allocate "first fit" (allocating the lowest address) because someone
|
||
wrote a paper once that said that it's the best algorithm to combat
|
||
fragmentation (even though worst case fragmentation is unavoidable
|
||
regardless of algorithm).
|
||
|
||
(bmalloc::XLargeMap::addAllocated):
|
||
(bmalloc::XLargeMap::getAllocated):
|
||
(bmalloc::XLargeMap::takeAllocated):
|
||
(bmalloc::XLargeMap::shrinkToFit):
|
||
(bmalloc::XLargeMap::takePhysical):
|
||
(bmalloc::XLargeMap::addVirtual):
|
||
* bmalloc/XLargeMap.h: Added.
|
||
(bmalloc::XLargeMap::Allocation::operator<):
|
||
|
||
* bmalloc/XLargeRange.h: Added.
|
||
(bmalloc::XLargeRange::XLargeRange):
|
||
(bmalloc::XLargeRange::vmState):
|
||
(bmalloc::XLargeRange::setVMState):
|
||
(bmalloc::canMerge):
|
||
(bmalloc::merge):
|
||
(bmalloc::XLargeRange::split): Helper for tracking VMState in a range.
|
||
|
||
2016-02-23 Dan Bernstein <mitz@apple.com>
|
||
|
||
[Xcode] Linker errors display mangled names, but no longer should
|
||
https://bugs.webkit.org/show_bug.cgi?id=154632
|
||
|
||
Reviewed by Sam Weinig.
|
||
|
||
* Configurations/Base.xcconfig: Stop setting LINKER_DISPLAYS_MANGLED_NAMES to YES.
|
||
|
||
2016-02-22 Konstantin Tokarev <annulen@yandex.ru>
|
||
|
||
Fixed compilation of bmalloc with GCC 4.8 after r196873.
|
||
https://bugs.webkit.org/show_bug.cgi?id=154534
|
||
|
||
Reviewed by Mark Lam.
|
||
|
||
See https://gcc.gnu.org/bugzilla/show_bug.cgi?id=55382.
|
||
|
||
* bmalloc/LargeChunk.h:
|
||
* bmalloc/SmallChunk.h:
|
||
|
||
2016-02-21 Geoffrey Garen <ggaren@apple.com>
|
||
|
||
bmalloc: Don't use a whole page for metadata
|
||
https://bugs.webkit.org/show_bug.cgi?id=154510
|
||
|
||
Reviewed by Andreas Kling.
|
||
|
||
(1) Don't round up metadata to a page boundary. This saves 1.5% dirty
|
||
memory on iOS and 0.2% on Mac. It also enables a future patch to allocate
|
||
smaller chunks without wasting memory.
|
||
|
||
(2) Initialize metadata lazily. This saves dirty memory when the program
|
||
allocates primarily small or large objects (but not both), leaving some
|
||
metadata uninitialized.
|
||
|
||
* bmalloc.xcodeproj/project.pbxproj: Medium objects are gone now.
|
||
|
||
* bmalloc/BumpAllocator.h:
|
||
(bmalloc::BumpAllocator::refill): Added an ASSERT to help debug a bug
|
||
I cause while working on this patch.
|
||
|
||
* bmalloc/Heap.cpp:
|
||
(bmalloc::Heap::allocateSmallBumpRanges): Ditto.
|
||
|
||
(bmalloc::Heap::splitAndAllocate):
|
||
(bmalloc::Heap::allocateLarge): Updated for interface change.
|
||
|
||
* bmalloc/LargeChunk.h: Changed the boundaryTagCount calculation to
|
||
a static_assert.
|
||
|
||
Don't round up to page boundary. (See above.)
|
||
|
||
(bmalloc::LargeChunk::LargeChunk): Moved code here from LargeChunk::init.
|
||
A constructor is a more natural / automatic way to do this initialization.
|
||
|
||
* bmalloc/LargeObject.h:
|
||
(bmalloc::LargeObject::init): Deleted. Moved to LargeChunk.
|
||
|
||
* bmalloc/Sizes.h: Chagned largeChunkMetadataSize to a simpler constant
|
||
because metadata size no longer varies by page size.
|
||
|
||
* bmalloc/SmallChunk.h:
|
||
(bmalloc::SmallChunk::begin):
|
||
(bmalloc::SmallChunk::end):
|
||
(bmalloc::SmallChunk::lines):
|
||
(bmalloc::SmallChunk::pages): Use std::array to make begin/end
|
||
calculations easier.
|
||
|
||
(bmalloc::SmallChunk::SmallChunk): Treat our metadata like a series
|
||
of allocated objects. We used to avoid trampling our metadata by
|
||
starting object memory at the next page. Now we share the first page
|
||
between metadata and objects, and we account for metadata explicitly.
|
||
|
||
* bmalloc/SuperChunk.h:
|
||
(bmalloc::SuperChunk::SuperChunk):
|
||
(bmalloc::SuperChunk::smallChunk):
|
||
(bmalloc::SuperChunk::largeChunk):
|
||
(bmalloc::SuperChunk::create): Deleted. Don't eagerly run the SmallChunk
|
||
and LargeChunk constructors. We'll run them lazily as needed.
|
||
|
||
* bmalloc/VMHeap.cpp:
|
||
(bmalloc::VMHeap::VMHeap):
|
||
(bmalloc::VMHeap::allocateSmallChunk):
|
||
(bmalloc::VMHeap::allocateLargeChunk):
|
||
(bmalloc::VMHeap::allocateSuperChunk):
|
||
(bmalloc::VMHeap::grow): Deleted. Track small and large chunks explicitly
|
||
so we can initialize them lazily.
|
||
|
||
* bmalloc/VMHeap.h:
|
||
(bmalloc::VMHeap::allocateSmallPage):
|
||
(bmalloc::VMHeap::allocateLargeObject): Specify whether we're allocating
|
||
a small or large chunk since we don't allocate both at once anymore.
|
||
|
||
2016-02-20 Mark Lam <mark.lam@apple.com>
|
||
|
||
Use of inlined asm statements causes problems for -std=c99 builds.
|
||
https://bugs.webkit.org/show_bug.cgi?id=154507
|
||
|
||
Reviewed by Dan Bernstein.
|
||
|
||
* bmalloc/BAssert.h:
|
||
|
||
2016-02-19 Joonghun Park <jh718.park@samsung.com>
|
||
|
||
Unreviewed. Fix debug build error since r196847
|
||
|
||
Fix gcc build warning appeared as below
|
||
by removing BASSERT(refCount <= maxRefCount).
|
||
error: comparison is always true due to limited range of data type
|
||
[-Werror=type-limits]
|
||
|
||
* bmalloc/SmallLine.h:
|
||
(bmalloc::SmallLine::ref): Deleted.
|
||
|
||
2016-02-19 Geoffrey Garen <ggaren@apple.com>
|
||
|
||
bmalloc: Chunk, Page, and Line don't need to be class templates
|
||
https://bugs.webkit.org/show_bug.cgi?id=154480
|
||
|
||
Reviewed by Gavin Barraclough.
|
||
|
||
We needed class templates to distinguish between small and medium,
|
||
but medium is gone now.
|
||
|
||
* bmalloc.xcodeproj/project.pbxproj:
|
||
* bmalloc/Chunk.h: Removed.
|
||
* bmalloc/Heap.cpp:
|
||
(bmalloc::Heap::initializeLineMetadata):
|
||
(bmalloc::Heap::allocateSmallBumpRanges):
|
||
* bmalloc/Heap.h:
|
||
* bmalloc/Line.h: Removed.
|
||
* bmalloc/Page.h: Removed.
|
||
* bmalloc/Sizes.h:
|
||
* bmalloc/SmallChunk.h: Replaced with Source/bmalloc/bmalloc/Chunk.h.
|
||
(bmalloc::SmallChunk::begin):
|
||
(bmalloc::SmallChunk::end):
|
||
(bmalloc::SmallChunk::lines):
|
||
(bmalloc::SmallChunk::pages):
|
||
(bmalloc::SmallChunk::get):
|
||
(bmalloc::SmallLine::get):
|
||
(bmalloc::SmallLine::begin):
|
||
(bmalloc::SmallLine::end):
|
||
(bmalloc::SmallPage::get):
|
||
(bmalloc::SmallPage::begin):
|
||
(bmalloc::SmallPage::end):
|
||
(bmalloc::Chunk::begin): Deleted.
|
||
(bmalloc::Chunk::end): Deleted.
|
||
(bmalloc::Chunk::lines): Deleted.
|
||
(bmalloc::Chunk::pages): Deleted.
|
||
* bmalloc/SmallLine.h: Replaced with Source/bmalloc/bmalloc/Line.h.
|
||
(bmalloc::SmallLine::ref):
|
||
(bmalloc::SmallLine::deref):
|
||
(bmalloc::Line<Traits>::begin): Deleted.
|
||
(bmalloc::Line<Traits>::end): Deleted.
|
||
(bmalloc::Line<Traits>::ref): Deleted.
|
||
(bmalloc::Line<Traits>::deref): Deleted.
|
||
* bmalloc/SmallPage.h: Replaced with Source/bmalloc/bmalloc/Page.h.
|
||
(bmalloc::SmallPage::hasFreeLines):
|
||
(bmalloc::SmallPage::setHasFreeLines):
|
||
(bmalloc::SmallPage::ref):
|
||
(bmalloc::SmallPage::deref):
|
||
(bmalloc::Page::hasFreeLines): Deleted.
|
||
(bmalloc::Page::setHasFreeLines): Deleted.
|
||
(bmalloc::Page<Traits>::ref): Deleted.
|
||
(bmalloc::Page<Traits>::deref): Deleted.
|
||
* bmalloc/SmallTraits.h: Removed.
|
||
|
||
2016-02-18 Geoffrey Garen <ggaren@apple.com>
|
||
|
||
bmalloc: Remove the concept of medium objects
|
||
https://bugs.webkit.org/show_bug.cgi?id=154436
|
||
|
||
Reviewed by Sam Weinig.
|
||
|
||
There's no need to distinguish medium objects from small: Small object
|
||
metadata works naturally for both as long as we allow an object to
|
||
span more than two small lines. (We already allow an object to span
|
||
more than one small line.)
|
||
|
||
This change reduces memory use because it eliminates the 1kB line size,
|
||
so we don't have to hold down 1kB lines for individual 264+ byte objects.
|
||
|
||
1kB lines were always a bit of a compromise. The main point of bump
|
||
allocation is to take advantage of cache lines. Cache lines are usually
|
||
64 bytes, so line sizes above 256 bytes are a bit of a stretch.
|
||
|
||
This change speeds up small object benchmarks because it eliminates the
|
||
branch to detect medium objects in deallocation log processing.
|
||
|
||
This change reduces virtual memory use from worst cast 4X to worst case
|
||
2X because the medium chunk is gone. iOS cares about virtual memory use
|
||
and terminates apps above ~1GB, so this change gives us more breathing room.
|
||
|
||
This change slows down medium benchmarks a bit because we end up doing
|
||
more work to recycle fragmented medium objects. Overall, the tradeoff
|
||
seems justified, since we have a net speedup and a memory use savings.
|
||
|
||
* bmalloc.xcodeproj/project.pbxproj: Removed all the medium files. We
|
||
can simplify even further in a follow-up patch, removing the base class
|
||
templates for Chunk, Page, and Line as well.
|
||
|
||
* bmalloc/Allocator.cpp:
|
||
(bmalloc::Allocator::Allocator):
|
||
(bmalloc::Allocator::allocate):
|
||
(bmalloc::Allocator::reallocate):
|
||
(bmalloc::Allocator::scavenge):
|
||
(bmalloc::Allocator::refillAllocatorSlowCase):
|
||
(bmalloc::Allocator::refillAllocator):
|
||
(bmalloc::Allocator::allocateSlowCase): Medium is gone. Small max is the
|
||
new medium max.
|
||
|
||
* bmalloc/Allocator.h:
|
||
(bmalloc::Allocator::allocateFastCase): Ditto.
|
||
|
||
* bmalloc/BumpAllocator.h:
|
||
(bmalloc::BumpAllocator::validate):
|
||
(bmalloc::BumpAllocator::allocate): No more medium.
|
||
|
||
* bmalloc/Chunk.h: No more medium.
|
||
|
||
* bmalloc/Deallocator.cpp:
|
||
(bmalloc::Deallocator::processObjectLog): No check for medium. This is
|
||
a speedup.
|
||
|
||
(bmalloc::Deallocator::deallocateSlowCase): No more medium.
|
||
|
||
* bmalloc/Deallocator.h:
|
||
(bmalloc::Deallocator::deallocateFastCase): Ditto.
|
||
|
||
* bmalloc/Heap.cpp:
|
||
(bmalloc::Heap::initializeLineMetadata): The algorithm here changed from
|
||
iterating each line to iterating each object. This helps us accomodate
|
||
objects that might span more than two lines -- i.e., all objects between
|
||
(512 bytes, 1024 bytes].
|
||
|
||
(bmalloc::Heap::scavenge):
|
||
(bmalloc::Heap::scavengeSmallPages):
|
||
(bmalloc::Heap::scavengeLargeObjects): Medium is gone.
|
||
|
||
(bmalloc::Heap::allocateSmallBumpRanges): Allow for lines that allocate
|
||
zero objects. This happens when an object spans more than two lines --
|
||
the middle lines allocate zero objects.
|
||
|
||
Also set the "has free lines" bit to false if we consume the last free
|
||
line. This needs to be a bit now because not all pages agree on their
|
||
maximum refcount anymore, so we need an explicit signal for the transition
|
||
from maximum to maximum - 1.
|
||
|
||
(bmalloc::Heap::allocateSmallPage): This code didn't change; I just removed
|
||
the medium code.
|
||
|
||
(bmalloc::Heap::deallocateSmallLine): Changed the algorithm to check
|
||
hasFreeLines. See allocateSmallBumpRanges.
|
||
|
||
(bmalloc::Heap::scavengeMediumPages): Deleted.
|
||
(bmalloc::Heap::allocateMediumBumpRanges): Deleted.
|
||
(bmalloc::Heap::allocateMediumPage): Deleted.
|
||
(bmalloc::Heap::deallocateMediumLine): Deleted.
|
||
* bmalloc/Heap.h:
|
||
(bmalloc::Heap::derefMediumLine): Deleted.
|
||
|
||
* bmalloc/LargeChunk.h:
|
||
(bmalloc::LargeChunk::get):
|
||
(bmalloc::LargeChunk::endTag):
|
||
* bmalloc/Line.h: No more medium.
|
||
|
||
* bmalloc/MediumChunk.h: Removed.
|
||
* bmalloc/MediumLine.h: Removed.
|
||
* bmalloc/MediumPage.h: Removed.
|
||
* bmalloc/MediumTraits.h: Removed.
|
||
|
||
* bmalloc/ObjectType.cpp:
|
||
(bmalloc::objectType):
|
||
* bmalloc/ObjectType.h:
|
||
(bmalloc::isSmall):
|
||
(bmalloc::isXLarge):
|
||
(bmalloc::isSmallOrMedium): Deleted.
|
||
(bmalloc::isMedium): Deleted. No more medium.
|
||
|
||
* bmalloc/Page.h:
|
||
(bmalloc::Page::sizeClass):
|
||
(bmalloc::Page::setSizeClass):
|
||
(bmalloc::Page::hasFreeLines):
|
||
(bmalloc::Page::setHasFreeLines): Add the free lines bit. You get better
|
||
codegen if you make it the low bit, since ref / deref can then add / sub
|
||
2. So do that.
|
||
|
||
* bmalloc/Sizes.h:
|
||
(bmalloc::Sizes::sizeClass): Expand the small size class to include the
|
||
medium size class.
|
||
|
||
* bmalloc/SuperChunk.h:
|
||
(bmalloc::SuperChunk::SuperChunk):
|
||
(bmalloc::SuperChunk::smallChunk):
|
||
(bmalloc::SuperChunk::largeChunk):
|
||
(bmalloc::SuperChunk::mediumChunk): Deleted. No more medium.
|
||
|
||
* bmalloc/VMHeap.cpp:
|
||
(bmalloc::VMHeap::grow):
|
||
* bmalloc/VMHeap.h:
|
||
(bmalloc::VMHeap::allocateSmallPage): Set the has free lines bit before
|
||
returning a Page to the Heap since this is the correct default state
|
||
when we first allocate a page.
|
||
|
||
(bmalloc::VMHeap::allocateMediumPage): Deleted.
|
||
(bmalloc::VMHeap::deallocateMediumPage): Deleted.
|
||
|
||
2016-02-19 Michael Saboff <msaboff@apple.com>
|
||
|
||
bmalloc: Unify VMHeap and Heap LargeObjects free lists to reduce fragmentation
|
||
https://bugs.webkit.org/show_bug.cgi?id=154192
|
||
|
||
Reviewed by Geoffrey Garen.
|
||
|
||
Change the operation of Heap and VMHeap LargeObject free lists.
|
||
Renamed Owner to VMState to track the state of each LargeObject.
|
||
Physical - The pages have been allocated.
|
||
Virtual - The pages have not been allocated.
|
||
Mixed - The object contains a mixture of Physical and Virtual pages.
|
||
VMState uses one bit each for Physical and Virtual to simplify merging states
|
||
when merging two adjacent blocks. This change enforces the rule that objects in
|
||
the Heap free list must have have the Physical bit set in their VMState while objects
|
||
in the VMHeap free list must have the Physical bit clear. Thie means that the Heap
|
||
can have LargeObjects in Physical or Mixed VMState, but the VMHeap's free list can
|
||
only contain Virtual LargeObjects.
|
||
|
||
In both Heap::allocateLarge(), we now allocate physical pages if the LargeObject we
|
||
pull from the free list has any Virtual pages before we possilby split the
|
||
object. When we merge objects, the result might be made up of Mixed page allocations.
|
||
When allocating a Mixed LargeObject, we need to allocate memory for them as well.
|
||
The scavenger deallocates both Physical and Mixed LargeObjects, placing them back into
|
||
the VMHeap's free list.
|
||
|
||
When we allocate or deallocate Mixed LargeObjects, there are pages that within these
|
||
objects that will be redundantly modified. It would require additional metadata to
|
||
eliminate this redundancy.
|
||
|
||
* bmalloc.xcodeproj/project.pbxproj:
|
||
* bmalloc/BoundaryTag.h:
|
||
(bmalloc::BoundaryTag::vmState): New helper.
|
||
(bmalloc::BoundaryTag::setVMState): New helper.
|
||
(bmalloc::BoundaryTag::owner): Deleted.
|
||
(bmalloc::BoundaryTag::setOwner): Deleted.
|
||
* bmalloc/Heap.h:
|
||
(bmalloc::Heap::splitAndAllocate): New helpers.
|
||
* bmalloc/LargeObject.h:
|
||
(bmalloc::LargeObject::vmState): New helper.
|
||
(bmalloc::LargeObject::setVMState): New helper.
|
||
|
||
* bmalloc/Heap.cpp:
|
||
(bmalloc::Heap::splitAndAllocate): New helpers.
|
||
(bmalloc::Heap::allocateLarge):
|
||
(bmalloc::Heap::deallocatePhysicalPages): Refactored from VMHeap::deallocateLargeObjectMemory.
|
||
|
||
* bmalloc/FreeList.cpp:
|
||
(bmalloc::FreeList::takeGreedy):
|
||
(bmalloc::FreeList::take):
|
||
(bmalloc::FreeList::removeInvalidAndDuplicateEntries):
|
||
* bmalloc/FreeList.h:
|
||
(bmalloc::FreeList::FreeList):
|
||
(bmalloc::FreeList::push):
|
||
* bmalloc/Heap.cpp:
|
||
(bmalloc::Heap::Heap):
|
||
(bmalloc::Heap::scavengeLargeObjects):
|
||
* bmalloc/LargeObject.h:
|
||
(bmalloc::LargeObject::isValidAndFree):
|
||
(bmalloc::LargeObject::validateSelf):
|
||
* bmalloc/SegregatedFreeList.cpp:
|
||
(bmalloc::SegregatedFreeList::SegregatedFreeList): Changed to initialize our required Physical state.
|
||
* bmalloc/SegregatedFreeList.h:
|
||
(bmalloc::SegregatedFreeList::SegregatedFreeList):
|
||
(bmalloc::SegregatedFreeList::insert):
|
||
(bmalloc::SegregatedFreeList::takeGreedy):
|
||
(bmalloc::SegregatedFreeList::take):
|
||
Replaced Owner parameters and checks with VMState::HasPhysical.
|
||
|
||
* bmalloc/LargeObject.h:
|
||
(bmalloc::LargeObject::prevCanMerge): Removed owner from tests.
|
||
(bmalloc::LargeObject::nextCanMerge): Removed owner from tests.
|
||
(bmalloc::LargeObject::merge): Removed owner from tests. Updated to merge VMStates andset the
|
||
VMState after the merge.
|
||
|
||
* bmalloc/LargeObject.h:
|
||
(bmalloc::LargeObject::owner): Deleted.
|
||
(bmalloc::LargeObject::setOwner): Deleted.
|
||
|
||
* bmalloc/Owner.h: Removed.
|
||
|
||
* bmalloc/VMAllocate.h:
|
||
(bmalloc::vmAllocatePhysicalPagesSloppy): Changed to round begin down to eliminate the left to right
|
||
allocation constraint.
|
||
|
||
* bmalloc/VMHeap.cpp:
|
||
(bmalloc::VMHeap::grow): Large space managed like small or medium as a vector of LargeChunks.
|
||
(bmalloc::VMHeap::VMHeap): Changed to initialize our required Physical state.
|
||
|
||
* bmalloc/VMHeap.h:
|
||
(bmalloc::VMHeap::allocateLargeObject): These no longer allocate memory.
|
||
(bmalloc::VMHeap::deallocateLargeObject): Removed setOwner. Now we set the VMState after any merges.
|
||
|
||
* bmalloc/VMState.h: Copied from Source/bmalloc/bmalloc/Owner.h.
|
||
(bmalloc::VMState::VMState):
|
||
(bmalloc::VMState::hasPhysical):
|
||
(bmalloc::VMState::hasVirtual):
|
||
(bmalloc::VMState::merge):
|
||
(bmalloc::VMState::operator ==):
|
||
(bmalloc::VMState::operator unsigned):
|
||
New class with various helpers.
|
||
|
||
2016-02-12 Michael Saboff <msaboff@apple.com>
|
||
|
||
BASSERTs added in r196421 are causing debug test failures
|
||
https://bugs.webkit.org/show_bug.cgi?id=154113
|
||
|
||
Reviewed by Geoffrey Garen.
|
||
|
||
In VMHeap::deallocateLargeObject(), we drop the lock to deallocate the physical pages.
|
||
If the scavenger thread is running at the same time a synchronous call to scavenge()
|
||
comes in, we could call VMHeap::deallocateLargeObject() for an adjacent object while the
|
||
lock in the other thread is dropped. We fix this by checking for adjacent objects we
|
||
can merge with and loop if we have one.
|
||
|
||
* bmalloc/FreeList.h:
|
||
(bmalloc::FreeList::push): Added BASSERT to catch adding unmerged free objects
|
||
* bmalloc/Heap.cpp:
|
||
(bmalloc::Heap::allocateLarge): Changed to use nextCanMerge().
|
||
* bmalloc/LargeObject.h:
|
||
(bmalloc::LargeObject::prevCanMerge): Repurposed prevIsAllocated.
|
||
(bmalloc::LargeObject::nextCanMerge): Repurposed nextIsAllocated.
|
||
(bmalloc::LargeObject::prevIsAllocated): Deleted.
|
||
(bmalloc::LargeObject::nextIsAllocated): Deleted.
|
||
* bmalloc/VMHeap.h:
|
||
(bmalloc::VMHeap::allocateLargeObject): Moved adding the extra object back to the free list
|
||
to after we set the object we'll return as being allocated.
|
||
(bmalloc::VMHeap::deallocateLargeObject):
|
||
|
||
2016-02-12 Mark Lam <mark.lam@apple.com>
|
||
|
||
Make BCRASH() use breakpoint traps too for non-debug OS(DARWIN).
|
||
https://bugs.webkit.org/show_bug.cgi?id=154184
|
||
|
||
Reviewed by Saam Barati.
|
||
|
||
This makes it behave consistently with WTFCrash().
|
||
|
||
* bmalloc/BAssert.h:
|
||
* bmalloc/BPlatform.h:
|
||
|
||
2016-02-11 Michael Saboff <msaboff@apple.com>
|
||
|
||
Unreviewed build fix after r196421.
|
||
|
||
Removed BASSERTs that are firing to eliminate Debug build crashes. I'll debug locally and
|
||
enable or alter after the issue is understood.
|
||
|
||
* bmalloc/LargeObject.h:
|
||
(bmalloc::LargeObject::merge): Removed BASSERTs that are firing.
|
||
|
||
2016-02-11 Michael Saboff <msaboff@apple.com>
|
||
|
||
bmalloc: large aligned allocations will put 1 or 2 free object on free list without merging with free neighbors
|
||
https://bugs.webkit.org/show_bug.cgi?id=154091
|
||
|
||
Reviewed by Geoffrey Garen.
|
||
|
||
If we split off any unused free object in the aligned version of Heap::allocateLarge(), we merge them with
|
||
free neighbors before putting them back on the free list. Added helpers to verify that when we
|
||
add LargeObjects to the free list their neighbors are allocated.
|
||
|
||
* bmalloc/Heap.cpp:
|
||
(bmalloc::Heap::allocateLarge): Deleted private helper version and rolled it into the two the
|
||
two public versions of allocateLarge().
|
||
* bmalloc/Heap.h:
|
||
* bmalloc/LargeObject.h:
|
||
(bmalloc::LargeObject::prevIsAllocated): New helper.
|
||
(bmalloc::LargeObject::nextIsAllocated): New helper.
|
||
(bmalloc::LargeObject::merge): Check that the merge object has allocated neighbors.
|
||
|
||
2016-02-05 Saam barati <sbarati@apple.com>
|
||
|
||
bmalloc: largeMax calculation is wrong on iOS
|
||
https://bugs.webkit.org/show_bug.cgi?id=153923
|
||
|
||
Reviewed by Mark Lam.
|
||
|
||
Our number for largeMax was larger than what we had
|
||
space to actually allocate inside the LargeChunk. This made
|
||
it so that we would allocate a large object for something
|
||
that really should be extra large. Previously:
|
||
largeMax + sizeof(LargeChunk) > 1MB
|
||
which meant that when we would grow() to accommodate an allocation
|
||
of a particular size inside a LargeObject despite the fact that
|
||
the allocation size would be too large to actually fit in the LargeObject.
|
||
This would manifest when we had an allocation size in the range:
|
||
1MB - sizeof(LargeChunk) < allocation size < largeMax
|
||
|
||
We fix this bug by being precise in our calculation of largeMax
|
||
instead of just assuming largeChunkSize * 99/100 is enough
|
||
space for the metadata.
|
||
|
||
* bmalloc/LargeChunk.h:
|
||
(bmalloc::LargeChunk::get):
|
||
* bmalloc/Sizes.h:
|
||
|
||
2016-01-31 Dan Bernstein <mitz@apple.com>
|
||
|
||
[Cocoa] Remove unused definition of HAVE_HEADER_DETECTION_H
|
||
https://bugs.webkit.org/show_bug.cgi?id=153729
|
||
|
||
Reviewed by Sam Weinig.
|
||
|
||
After r141700, HAVE_HEADER_DETECTION_H is no longer used.
|
||
|
||
* Configurations/Base.xcconfig:
|
||
|
||
2015-12-19 Dan Bernstein <mitz@apple.com>
|
||
|
||
[Mac] WebKit contains dead source code for OS X Mavericks and earlier
|
||
https://bugs.webkit.org/show_bug.cgi?id=152462
|
||
|
||
Reviewed by Alexey Proskuryakov.
|
||
|
||
* Configurations/DebugRelease.xcconfig: Removed definition of MACOSX_DEPLOYMENT_TARGET for
|
||
OS X 10.9.
|
||
|
||
2015-12-03 Anders Carlsson <andersca@apple.com>
|
||
|
||
Remove Objective-C GC support
|
||
https://bugs.webkit.org/show_bug.cgi?id=151819
|
||
rdar://problem/23746991
|
||
|
||
Reviewed by Dan Bernstein.
|
||
|
||
* Configurations/Base.xcconfig:
|
||
|
||
2015-12-03 Michael Saboff <msaboff@apple.com>
|
||
|
||
bmalloc: extra large allocations could be more efficient
|
||
https://bugs.webkit.org/show_bug.cgi?id=151817
|
||
|
||
Reviewed by Geoffrey Garen.
|
||
|
||
Reduced the super chunk size from 4MB to 2MB.
|
||
|
||
Added path to reallocate() of an extra large object to see if we can extend the allocation.
|
||
|
||
* bmalloc/Allocator.cpp:
|
||
(bmalloc::Allocator::reallocate):
|
||
* bmalloc/SegregatedFreeList.h:
|
||
* bmalloc/Sizes.h:
|
||
* bmalloc/VMAllocate.h:
|
||
(bmalloc::tryVMAllocate):
|
||
(bmalloc::tryVMExtend):
|
||
(bmalloc::vmAllocate):
|
||
|
||
2015-11-11 Akos Kiss <akiss@inf.u-szeged.hu>
|
||
|
||
bmalloc: Add libdl dependency
|
||
https://bugs.webkit.org/show_bug.cgi?id=151140
|
||
|
||
Reviewed by Csaba Osztrogonác.
|
||
|
||
Make sure that the linker links libdl and finds the references to
|
||
dlopen, dlsym and dlclose in Environment.cpp.
|
||
|
||
* CMakeLists.txt:
|
||
|
||
2015-11-02 Andy Estes <aestes@apple.com>
|
||
|
||
[Cocoa] Add tvOS and watchOS to SUPPORTED_PLATFORMS
|
||
https://bugs.webkit.org/show_bug.cgi?id=150819
|
||
|
||
Reviewed by Dan Bernstein.
|
||
|
||
This tells Xcode to include these platforms in its Devices dropdown, making it possible to build in the IDE.
|
||
|
||
* Configurations/Base.xcconfig:
|
||
|
||
2015-11-01 Philip Chimento <philip.chimento@gmail.com>
|
||
|
||
[GTK] Fix combinations of PLATFORM(GTK) and OS(DARWIN)
|
||
https://bugs.webkit.org/show_bug.cgi?id=144560
|
||
|
||
Reviewed by Darin Adler.
|
||
|
||
* PlatformGTK.cmake: Added. This adds Zone.cpp to the PlatformGTK
|
||
build, on Darwin only. Since there was previously nothing for the
|
||
build system to do that was specific to the GTK platform in
|
||
bmalloc, we need to create this file.
|
||
|
||
2015-10-29 Geoffrey Garen <ggaren@apple.com>
|
||
|
||
bmalloc: AsyncTask should handle destruction
|
||
https://bugs.webkit.org/show_bug.cgi?id=150648
|
||
|
||
Reviewed by Mark Lam.
|
||
|
||
So we can use it in more places.
|
||
|
||
* bmalloc/AsyncTask.h: Use std::thread instead of pthread because it
|
||
should be more portable.
|
||
|
||
(bmalloc::Function>::AsyncTask): Renamed Signaled to RunRequested for
|
||
clarity. Added an ExitRequested state.
|
||
|
||
(bmalloc::Function>::~AsyncTask): Wait for our child thread to exit
|
||
before destroying ourselves because our child thread will modify our
|
||
data (and might modify our client's data). Note that we only need to
|
||
wait for the last child thread since any prior child thread, having
|
||
reached the Exited condition, is guaranteed not to read or write any
|
||
data.
|
||
|
||
(bmalloc::Function>::run):
|
||
(bmalloc::Function>::runSlowCase): Updated for interface changes. Also
|
||
changed to use our WebKit style for condition signal: Hold the lock
|
||
during the signal and always notify all. Technically, neither is necessary,
|
||
but it is easier to understand the code this way, and harder to make
|
||
mistakes.
|
||
|
||
(bmalloc::Function>::threadEntryPoint):
|
||
(bmalloc::Function>::threadRunLoop): Handle the new ExitRequested state.
|
||
Technically, this state has no meaningful difference from the Exited
|
||
state, but it is nice to be explicit.
|
||
|
||
(bmalloc::Function>::join): Deleted.
|
||
(bmalloc::Function>::pthreadEntryPoint): Deleted.
|
||
(bmalloc::Function>::entryPoint): Deleted.
|
||
|
||
2015-10-15 Geoffrey Garen <ggaren@apple.com>
|
||
|
||
bmalloc: per-thread cache data structure should be smaller
|
||
https://bugs.webkit.org/show_bug.cgi?id=150218
|
||
|
||
Reviewed by Andreas Kling.
|
||
|
||
Reduce the number of entries in the range cache because it's really
|
||
big, and the bigness only helps in cases of serious fragmentation, and
|
||
it only saves us a little bit of lock acquisition time.
|
||
|
||
* bmalloc/Allocator.cpp:
|
||
(bmalloc::Allocator::scavenge):
|
||
(bmalloc::Allocator::refillAllocatorSlowCase):
|
||
(bmalloc::Allocator::refillAllocator):
|
||
(bmalloc::Allocator::allocateLarge):
|
||
(bmalloc::Allocator::allocateSlowCase):
|
||
(bmalloc::Allocator::allocateBumpRangeSlowCase): Deleted.
|
||
(bmalloc::Allocator::allocateBumpRange): Deleted.
|
||
* bmalloc/Allocator.h: Pass through the empty allocator and the range
|
||
cache when refilling, and refill both. Otherwise, we always immediately
|
||
pop the last item in the range cache, wasting that slot of capacity.
|
||
|
||
* bmalloc/Heap.cpp:
|
||
(bmalloc::Heap::allocateSmallBumpRanges):
|
||
(bmalloc::Heap::allocateMediumBumpRanges): Account for the fact that
|
||
the range cache is no longer big enough to guarantee that it can hold
|
||
all the ranges in a page.
|
||
|
||
(bmalloc::Heap::refillSmallBumpRangeCache): Deleted.
|
||
(bmalloc::Heap::refillMediumBumpRangeCache): Deleted.
|
||
|
||
* bmalloc/Heap.h: Move VMHeap to the end of the object because it
|
||
contains a lot of unused / wasted space, and we want to pack our data
|
||
together in memory.
|
||
|
||
* bmalloc/Sizes.h: Make the range cache smaller.
|
||
|
||
2015-10-13 Chris Dumez <cdumez@apple.com>
|
||
|
||
Avoid useless copies in range-loops that are using 'auto'
|
||
https://bugs.webkit.org/show_bug.cgi?id=150091
|
||
|
||
Reviewed by Sam Weinig.
|
||
|
||
Avoid useless copies in range-loops that are using 'auto'. Also use
|
||
'auto*' instead of 'auto' when range values are pointers for clarity.
|
||
|
||
* bmalloc/Deallocator.cpp:
|
||
(bmalloc::Deallocator::processObjectLog):
|
||
|
||
2015-10-12 Geoffrey Garen <ggaren@apple.com>
|
||
|
||
bmalloc: Don't ASSERT that all syscalls succeed
|
||
https://bugs.webkit.org/show_bug.cgi?id=150047
|
||
<rdar://problem/22649531>
|
||
|
||
Reviewed by Mark Lam.
|
||
|
||
madvise can fail due to VM behaviors outside of our control:
|
||
copy-on-write, fork, mprotect, and other stuff.
|
||
|
||
Older darwin kernels sometimes return this error value, and new kernels
|
||
might again in future.
|
||
|
||
We haven't gained much from this ASSERT so far, so let's remove it.
|
||
|
||
Perhaps in future we can come up with a scheme that makes madvise
|
||
never fail, or that responds to failure.
|
||
|
||
* bmalloc/Syscall.h:
|
||
|
||
2015-10-10 Dan Bernstein <mitz@apple.com>
|
||
|
||
[iOS] Remove project support for iOS 8
|
||
https://bugs.webkit.org/show_bug.cgi?id=149993
|
||
|
||
Reviewed by Alexey Proskuryakov.
|
||
|
||
* Configurations/Base.xcconfig:
|
||
* Configurations/bmalloc.xcconfig:
|
||
* Configurations/mbmalloc.xcconfig:
|
||
|
||
2015-08-31 Michael Catanzaro <mcatanzaro@igalia.com>
|
||
|
||
Implement bmalloc::isASanEnabled for generic Unix
|
||
https://bugs.webkit.org/show_bug.cgi?id=148623
|
||
|
||
Reviewed by Geoffrey Garen.
|
||
|
||
* bmalloc/BPlatform.h: Add BOS_UNIX to detect whether the OS is a Unix.
|
||
* bmalloc/Environment.cpp:
|
||
(bmalloc::isASanEnabled): Implement a runtime check that should work on any Unix.
|
||
|
||
2015-08-19 Geoffrey Garen <ggaren@apple.com>
|
||
|
||
Crash @ bmalloc::Environment::computeIsBmallocEnabled
|
||
https://bugs.webkit.org/show_bug.cgi?id=148183
|
||
|
||
Reviewed by NOBODY Michael Saboff.
|
||
|
||
CrashTracer says we have some crashes beneath computeIsBmallocEnabled
|
||
dereferencing null in strstr. We null check getenv but not
|
||
_dyld_get_image_name, so deduction indicates that _dyld_get_image_name
|
||
must be returning null. _dyld_get_image_name isn't really documented,
|
||
so let's assume it can return null.
|
||
|
||
* bmalloc/Environment.cpp:
|
||
(bmalloc::isASanEnabled): Check _dyld_get_image_name's return value for
|
||
null because we can't prove it won't be null.
|
||
|
||
2015-07-24 Geoffrey Garen <ggaren@apple.com>
|
||
|
||
vmmap crash at JavaScriptCore: 0x31cd12f6 (the JavaScript malloc zone enumerator)
|
||
https://bugs.webkit.org/show_bug.cgi?id=147274
|
||
|
||
Reviewed by Anders Carlsson.
|
||
|
||
It's not really clear why vmmap sometimes fails to read the target
|
||
process, but we can avoid a crash when it does. This is useful because
|
||
you'll still get all the non-bmalloc data out of the target process,
|
||
and bmalloc might not even be relevant to your investigation.
|
||
|
||
* bmalloc/Zone.cpp:
|
||
(bmalloc::remoteRead): Check for failure.
|
||
|
||
2015-07-24 Geoffrey Garen <ggaren@apple.com>
|
||
|
||
JavaScriptCore bmalloc should not register its malloc zone more than once
|
||
https://bugs.webkit.org/show_bug.cgi?id=147273
|
||
|
||
Reviewed by Andreas Kling.
|
||
|
||
This was a goof: The Zone constructor, by virtue of running automatically,
|
||
was registering a Zone inside the analysis process.
|
||
|
||
* bmalloc/Zone.cpp:
|
||
(bmalloc::remoteRead): Clarify that the pointer is remote.
|
||
|
||
(bmalloc::enumerator):
|
||
(bmalloc::Zone::Zone):
|
||
* bmalloc/Zone.h: Separate the normal constructor and the remote constructor.
|
||
The remote constructor skips zone registration since its goal is not
|
||
to register a zone in the current process or do any allocation but rather
|
||
to mirror the bytes of the zone from the target process.
|
||
|
||
2015-07-23 Geoffrey Garen <ggaren@apple.com>
|
||
|
||
bmalloc: Shrink the super chunk size (again)
|
||
https://bugs.webkit.org/show_bug.cgi?id=147240
|
||
|
||
Reviewed by Andreas Kling.
|
||
|
||
Shrinking to 8MB reduced VM exhaustion crashes but did not eliminate them.
|
||
Let's try 4MB.
|
||
|
||
(My previous comment was that the maximum fast object was 2MB. But it
|
||
was 4MB! Now it's 2MB for realsies.)
|
||
|
||
* bmalloc/Sizes.h:
|
||
|
||
2015-07-03 Dan Bernstein <mitz@apple.com>
|
||
|
||
[Xcode] Update some build settings as recommended by Xcode 7
|
||
https://bugs.webkit.org/show_bug.cgi?id=146597
|
||
|
||
Reviewed by Sam Weinig.
|
||
|
||
* Configurations/Base.xcconfig: Enabled CLANG_WARN_UNREACHABLE_CODE, GCC_NO_COMMON_BLOCKS,
|
||
and ENABLE_STRICT_OBJC_MSGSEND. Removed GCC_MODEL_TUNING.
|
||
|
||
* bmalloc.xcodeproj/project.pbxproj: Updated LastUpgradeCheck.
|
||
|
||
2015-07-02 Geoffrey Garen <ggaren@apple.com>
|
||
|
||
bmalloc: Shrink the super chunk size
|
||
https://bugs.webkit.org/show_bug.cgi?id=146519
|
||
|
||
Reviewed by Andreas Kling.
|
||
|
||
We have lots of reports of crashing due to failed VM allocation on iOS.
|
||
(This VM limit on iOS is usually 1GB-2GB, and has been as low as 256MB.)
|
||
|
||
Shrink the super chunk size in case fragmentation is the reason for
|
||
VM allocation failure.
|
||
|
||
This has the downside that >= 2MB allocations will now be super slow,
|
||
but they are also super rare (as in never on most websites), so this
|
||
is probably an OK tradeoff.
|
||
|
||
* bmalloc/Sizes.h:
|
||
|
||
2015-07-01 Geoffrey Garen <ggaren@apple.com>
|
||
|
||
bmalloc: realloc of an XLarge range can unmap adjacent VM ranges
|
||
https://bugs.webkit.org/show_bug.cgi?id=146535
|
||
|
||
Reviewed by Anders Carlsson.
|
||
|
||
This bug causes a crash when running fast/css/large-list-of-rules-crash.html
|
||
with the fix applied for https://bugs.webkit.org/show_bug.cgi?id=146519.
|
||
|
||
* bmalloc/Allocator.cpp:
|
||
(bmalloc::Allocator::reallocate): Start at object + newSize since starting
|
||
at object + oldSize means deleting the adjacent VM range.
|
||
|
||
2015-05-26 Geoffrey Garen <ggaren@apple.com>
|
||
|
||
Integer overflow in XLarge allocation (due to unchecked roundUpToMultipleOf)
|
||
https://bugs.webkit.org/show_bug.cgi?id=145385
|
||
|
||
Reviewed by Andreas Kling.
|
||
|
||
Added some checking to verify that round-up operations will not overflow
|
||
a size_t.
|
||
|
||
The simplest way to do this was to introduce a notion of xLargeMax, like
|
||
we have for smallMax, mediumMax, and largeMax. It's a bit surprising at
|
||
first to think that there is an xLargeMax, since xLarge is what we use
|
||
to handle the biggest things. But computers have limits, so it makes sense.
|
||
|
||
FWIW, TCMalloc used to have an xLargeMax too, which it called kMaxValidPages.
|
||
|
||
No test because this bug was found by code inspection and I don't know
|
||
of a practical way to convince WebKit to make an allocation this large.
|
||
|
||
* bmalloc/Allocator.cpp:
|
||
(bmalloc::Allocator::tryAllocate):
|
||
(bmalloc::Allocator::allocate):
|
||
(bmalloc::Allocator::reallocate):
|
||
(bmalloc::Allocator::allocateSlowCase): Check against xLargeMax to avoid
|
||
overflow when rounding up.
|
||
|
||
* bmalloc/BAssert.h: Added support for explicit crashing.
|
||
|
||
* bmalloc/Sizes.h:
|
||
|
||
2015-05-26 Dan Bernstein <mitz@apple.com>
|
||
|
||
<rdar://problem/21104551> Update build settings
|
||
|
||
Reviewed by Anders Carlsson.
|
||
|
||
* Configurations/DebugRelease.xcconfig:
|
||
|
||
2015-05-23 Dan Bernstein <mitz@apple.com>
|
||
|
||
Remove unused definitions of WEBKIT_VERSION_MIN_REQUIRED
|
||
https://bugs.webkit.org/show_bug.cgi?id=145345
|
||
|
||
Reviewed by Sam Weinig.
|
||
|
||
* Configurations/Base.xcconfig: Also changed to use $(inherited).
|
||
|
||
2015-05-07 Geoffrey Garen <ggaren@apple.com>
|
||
|
||
Release assert in com.apple.WebKit.WebContent under JavaScriptCore: JSC::JSONProtoFuncStringify
|
||
https://bugs.webkit.org/show_bug.cgi?id=144758
|
||
|
||
Reviewed by Andreas Kling.
|
||
|
||
This was an out-of-memory error when trying to shrink a string builder.
|
||
bmalloc was missing the optimization that allowed realloc() to shrink
|
||
without copying. So, let's add it.
|
||
|
||
* bmalloc/Allocator.cpp:
|
||
(bmalloc::Allocator::reallocate): Added Large and XLarge cases for
|
||
shrinking without copying. This isn't possible for small and medium
|
||
objects, and probably not very profitable, either.
|
||
|
||
* bmalloc/Heap.cpp:
|
||
(bmalloc::Heap::findXLarge):
|
||
(bmalloc::Heap::deallocateXLarge):
|
||
* bmalloc/Heap.h: Refactored this code to return a reference to an
|
||
XLarge range. This makes the code reusable, and also makes it easier
|
||
for realloc() to update metadata.
|
||
|
||
* bmalloc/LargeObject.h:
|
||
(bmalloc::LargeObject::split): Allow allocated objects to split because
|
||
that's what realloc() wants to do, and there's nothing intrinsically
|
||
wrong with it.
|
||
|
||
2015-05-07 Dan Bernstein <mitz@apple.com>
|
||
|
||
<rdar://problem/19317140> [Xcode] Remove usage of AspenFamily.xcconfig in Source/
|
||
https://bugs.webkit.org/show_bug.cgi?id=144727
|
||
|
||
Reviewed by Darin Adler.
|
||
|
||
* Configurations/Base.xcconfig: Dont’s include AspenFamily.xcconfig, and define
|
||
INSTALL_PATH_PREFIX for the iOS 8.x Simulator.
|
||
|
||
2015-04-01 Alex Christensen <achristensen@webkit.org>
|
||
|
||
Progress towards CMake on Windows and Mac.
|
||
https://bugs.webkit.org/show_bug.cgi?id=143293
|
||
|
||
Reviewed by Filip Pizlo.
|
||
|
||
* bmalloc/BAssert.h:
|
||
Removed ellipses from macros to appease Visual Studio.
|
||
|
||
2015-03-13 Alex Christensen <achristensen@webkit.org>
|
||
|
||
Progress towards CMake on Mac.
|
||
https://bugs.webkit.org/show_bug.cgi?id=142680
|
||
|
||
Reviewed by Gyuyoung Kim.
|
||
|
||
* CMakeLists.txt:
|
||
* PlatformMac.cmake:
|
||
Added Zone.cpp to Mac CMake builds.
|
||
|
||
2015-03-12 Geoffrey Garen <ggaren@apple.com>
|
||
|
||
Assertion failure in bmalloc::LargeObject::validateSelf on Mavericks Debug layout test bot
|
||
https://bugs.webkit.org/show_bug.cgi?id=142642
|
||
|
||
Reviewed by Michael Saboff.
|
||
|
||
The typical backtrace to this crash shows the main thread trying to
|
||
realloc a large string while a DFG compiler thread tries to
|
||
free a large vector buffer.
|
||
|
||
I believe that this is a race condition -- at least in debug builds --
|
||
since the main thread will try to validate its object's neighbors
|
||
without holding a lock, even though those neighbors might be in the
|
||
midst of changing.
|
||
|
||
In general, there may be sneaky times when it is valid to look at an
|
||
object's metadata without holding the heap lock, but it is best not to
|
||
do so unless we have a really really good reason to.
|
||
|
||
* bmalloc/Allocator.cpp:
|
||
(bmalloc::Allocator::reallocate): Take a lock before reading the metadata
|
||
for this object, since we generally require any access to shared heap
|
||
metadata to take a lock.
|
||
|
||
2015-03-10 Geoffrey Garen <ggaren@apple.com>
|
||
|
||
bmalloc: tryFastMalloc shouldn't crash
|
||
https://bugs.webkit.org/show_bug.cgi?id=142443
|
||
|
||
Reviewed by Sam Weinig.
|
||
|
||
Rolling back in r181307 with a check for whether bmalloc is enabled, to
|
||
avoid crashes when running with ASan and GuardMalloc.
|
||
|
||
* bmalloc/Allocator.cpp:
|
||
(bmalloc::Allocator::tryAllocate):
|
||
* bmalloc/Allocator.h:
|
||
* bmalloc/Cache.cpp:
|
||
(bmalloc::Cache::tryAllocateSlowCaseNullCache):
|
||
* bmalloc/Cache.h:
|
||
(bmalloc::Cache::tryAllocate):
|
||
* bmalloc/Heap.cpp:
|
||
(bmalloc::Heap::allocateXLarge):
|
||
(bmalloc::Heap::tryAllocateXLarge):
|
||
* bmalloc/Heap.h:
|
||
* bmalloc/VMAllocate.h:
|
||
(bmalloc::tryVMAllocate):
|
||
(bmalloc::vmAllocate):
|
||
* bmalloc/bmalloc.h:
|
||
(bmalloc::api::tryMalloc):
|
||
(bmalloc::api::realloc):
|
||
(bmalloc::api::free):
|
||
|
||
2015-03-09 Commit Queue <commit-queue@webkit.org>
|
||
|
||
Unreviewed, rolling out r181307.
|
||
https://bugs.webkit.org/show_bug.cgi?id=142525
|
||
|
||
Broke ASan tests (Requested by ap on #webkit).
|
||
|
||
Reverted changeset:
|
||
|
||
"bmalloc: tryFastMalloc shouldn't crash"
|
||
https://bugs.webkit.org/show_bug.cgi?id=142443
|
||
http://trac.webkit.org/changeset/181307
|
||
|
||
2015-03-09 Geoffrey Garen <ggaren@apple.com>
|
||
|
||
bmalloc: tryFastMalloc shouldn't crash
|
||
https://bugs.webkit.org/show_bug.cgi?id=142443
|
||
|
||
Reviewed by Darin Adler.
|
||
|
||
Added support for tryMalloc.
|
||
|
||
We assume that non-x-large allocations always succeed, and we crash
|
||
otherwise, since normal allocation failure will just cause the next
|
||
non-try allocation or internal metadata allocation to fail, and it's
|
||
hard and not really useful to keep limping along after that. But
|
||
extra-large allocations can meaningfully fail, and we can recover.
|
||
|
||
* bmalloc/Heap.cpp:
|
||
(bmalloc::Heap::allocateXLarge):
|
||
(bmalloc::Heap::tryAllocateXLarge):
|
||
* bmalloc/Heap.h: Added support for non-crashy x-large allocation.
|
||
|
||
* bmalloc/VMAllocate.h:
|
||
(bmalloc::tryVMAllocate):
|
||
(bmalloc::vmAllocate): Added support for non-crashy VM allocation.
|
||
|
||
* bmalloc/bmalloc.h:
|
||
(bmalloc::api::tryMalloc):
|
||
(bmalloc::api::realloc):
|
||
(bmalloc::api::free): Tried to clarify our behavior with some comments.
|
||
Unfortunately, calling what we do "malloc" is still not quite right, since
|
||
malloc returns null on failure and we don't.
|
||
|
||
2015-03-03 Geoffrey Garen <ggaren@apple.com>
|
||
|
||
bmalloc: Don't branch when setting the owner of a large object
|
||
https://bugs.webkit.org/show_bug.cgi?id=142241
|
||
|
||
Reviewed by Andreas Kling.
|
||
|
||
* bmalloc/BoundaryTag.h:
|
||
(bmalloc::BoundaryTag::owner):
|
||
(bmalloc::BoundaryTag::setOwner):
|
||
|
||
2015-03-03 Geoffrey Garen <ggaren@apple.com>
|
||
|
||
bmalloc should implement malloc introspection (to stop false-positive leaks when MallocStackLogging is off)
|
||
https://bugs.webkit.org/show_bug.cgi?id=141802
|
||
|
||
Reviewed by Andreas Kling.
|
||
|
||
Re-enabled this feature on iOS, now that the iOS crash should be fixed.
|
||
|
||
* bmalloc/VMHeap.cpp:
|
||
(bmalloc::VMHeap::grow):
|
||
* bmalloc/VMHeap.h:
|
||
|
||
2015-03-03 Geoffrey Garen <ggaren@apple.com>
|
||
|
||
bmalloc: Added missing features to the malloc zone introspection API
|
||
https://bugs.webkit.org/show_bug.cgi?id=142235
|
||
|
||
Reviewed by Andreas Kling.
|
||
|
||
This should fix the crash we saw on the iOS PLT bot
|
||
(c.f. http://trac.webkit.org/changeset/180604).
|
||
|
||
* bmalloc/Zone.cpp:
|
||
(bmalloc::good_size):
|
||
(bmalloc::check):
|
||
(bmalloc::print):
|
||
(bmalloc::log):
|
||
(bmalloc::force_lock):
|
||
(bmalloc::force_unlock):
|
||
(bmalloc::statistics):
|
||
(bmalloc::size):
|
||
(bmalloc::enumerator): Provide all of these functions since they are called
|
||
indiscriminately on all zones.
|
||
|
||
(bmalloc::Zone::Zone):
|
||
(bmalloc::Zone::size): Deleted.
|
||
(bmalloc::Zone::enumerator): Deleted. Moved these functions out of the
|
||
Zone class since they can stand alone.
|
||
|
||
* bmalloc/Zone.h:
|
||
|
||
2015-03-03 Geoffrey Garen <ggaren@apple.com>
|
||
|
||
bmalloc should implement malloc introspection (to stop false-positive leaks when MallocStackLogging is off)
|
||
https://bugs.webkit.org/show_bug.cgi?id=141802
|
||
|
||
Reviewed by Andreas Kling.
|
||
|
||
Rolling back in but disabled on iOS until I can debug why the iOS PLT crashes.
|
||
|
||
* bmalloc/VMHeap.cpp:
|
||
(bmalloc::VMHeap::grow):
|
||
* bmalloc/VMHeap.h:
|
||
* bmalloc/Zone.cpp:
|
||
(bmalloc::Zone::size):
|
||
(bmalloc::Zone::Zone):
|
||
* bmalloc/Zone.h:
|
||
|
||
2015-03-03 Geoffrey Garen <ggaren@apple.com>
|
||
|
||
bmalloc: Miscellaneous cleanup
|
||
https://bugs.webkit.org/show_bug.cgi?id=142231
|
||
|
||
Reviewed by Andreas Kling.
|
||
|
||
No performance change -- maybe a tiny reduction in memory use.
|
||
|
||
* bmalloc/Heap.cpp: Moved the sleep function into StaticMutex, since
|
||
it's a helper for working with mutexes.
|
||
|
||
(bmalloc::Heap::scavenge): Make sure to wait before we start any
|
||
scavenging, since individual scavenging functions now always scavenge
|
||
at least one page before waiting themselves.
|
||
|
||
(bmalloc::Heap::scavengeSmallPages):
|
||
(bmalloc::Heap::scavengeMediumPages):
|
||
(bmalloc::Heap::scavengeLargeObjects): Use the new wait helper to
|
||
simplify this code. Also, we now require our caller to wait until at
|
||
least one deallocation is desirable. This simplifies our loop.
|
||
|
||
(bmalloc::Heap::allocateSmallPage):
|
||
(bmalloc::Heap::allocateMediumPage):
|
||
(bmalloc::Heap::allocateXLarge):
|
||
(bmalloc::Heap::allocateLarge): Don't freak out any time the heap does
|
||
an allocation. Only consider the heap to be growing if it actually needs
|
||
to allocate new VM. This allows us to shrink the heap back down from a
|
||
high water mark more reliably even if heap activity continues.
|
||
|
||
(bmalloc::sleep): Deleted.
|
||
(bmalloc::Heap::scavengeLargeRanges): Renamed to match our use of
|
||
"LargeObject".
|
||
|
||
* bmalloc/Heap.h:
|
||
|
||
* bmalloc/LargeObject.h:
|
||
(bmalloc::LargeObject::operator bool): Added to simplify a while loop.
|
||
|
||
* bmalloc/StaticMutex.h:
|
||
(bmalloc::sleep):
|
||
(bmalloc::waitUntilFalse): New helper for waiting until a condition
|
||
becomes reliably false.
|
||
|
||
* bmalloc/Vector.h:
|
||
(bmalloc::Vector<T>::~Vector): Oops! Don't deallocate the null pointer.
|
||
We don't actually run any Vector destructors, but an iteration of this
|
||
patch did, and then crashed. So, let's fix that.
|
||
|
||
2015-03-02 Geoffrey Garen <ggaren@apple.com>
|
||
|
||
bmalloc: Eagerly remove allocated objects from the free list
|
||
https://bugs.webkit.org/show_bug.cgi?id=142194
|
||
|
||
Reviewed by Andreas Kling.
|
||
|
||
This reduces the pressure to garbage collect the free list.
|
||
|
||
Might be a 1% speedup on MallocBench.
|
||
|
||
* bmalloc/FreeList.cpp: Put this comment at the top of the file instead
|
||
of repeating it inside of each function. Tried to clarify the details.
|
||
|
||
(bmalloc::FreeList::takeGreedy): Matched the other iteration code in this
|
||
file for consistency -- even though either direction works fine in this
|
||
function.
|
||
|
||
(bmalloc::FreeList::take): Change to iterate from low to high so that we
|
||
can maintain an index into the vector that is not disturbed even if we
|
||
pop from the middle (which invalidates the last index in the vector).
|
||
|
||
Decrement i when popping from the middle to make sure that we don't
|
||
skip the next item after popping.
|
||
|
||
(bmalloc::FreeList::removeInvalidAndDuplicateEntries): Ditto.
|
||
|
||
2015-02-27 Ryosuke Niwa <rniwa@webkit.org>
|
||
|
||
Fixed a typo in the previous commit.
|
||
|
||
* bmalloc/BoundaryTag.h:
|
||
(bmalloc::BoundaryTag::setOwner):
|
||
|
||
2015-02-27 Ryosuke Niwa <rniwa@webkit.org>
|
||
|
||
EFL build fix after r180797.
|
||
|
||
* bmalloc/BoundaryTag.h:
|
||
(bmalloc::BoundaryTag::owner):
|
||
(bmalloc::BoundaryTag::setOwner):
|
||
|
||
2015-02-27 Geoffrey Garen <ggaren@apple.com>
|
||
|
||
bmalloc: Pathological madvise churn on the free(malloc(x)) benchmark
|
||
https://bugs.webkit.org/show_bug.cgi?id=142058
|
||
|
||
Reviewed by Andreas Kling.
|
||
|
||
The churn was caused by repeatedly splitting an object with physical
|
||
pages from an object without, and then merging them back together again.
|
||
The merge would conservatively forget that we had physical pages, forcing
|
||
a new call to madvise on the next allocation.
|
||
|
||
This patch more strictly segregates objects in the heap from objects in
|
||
the VM heap, with these changes:
|
||
|
||
(1) Objects in the heap are not allowed to merge with objects in the VM
|
||
heap, and vice versa -- since that would erase our precise knowledge of
|
||
which physical pages had been allocated.
|
||
|
||
(2) The VM heap is exclusively responsible for allocating and deallocating
|
||
physical pages.
|
||
|
||
(3) The heap free list must consider entries for objects that are in the
|
||
VM heap to be invalid, and vice versa. (This condition can arise
|
||
because the free list does not eagerly remove items.)
|
||
|
||
With these changes, we can know that any valid object in the heap's free
|
||
list already has physical pages, and does not need to call madvise.
|
||
|
||
Note that the VM heap -- as before -- might sometimes contain ranges
|
||
or pieces of ranges that have physical pages, since we allow splitting
|
||
of ranges at granularities smaller than the VM page size. These ranges
|
||
can eventually merge with ranges in the heap during scavenging.
|
||
|
||
* bmalloc.xcodeproj/project.pbxproj:
|
||
|
||
* bmalloc/BoundaryTag.h:
|
||
(bmalloc::BoundaryTag::owner):
|
||
(bmalloc::BoundaryTag::setOwner):
|
||
(bmalloc::BoundaryTag::initSentinel):
|
||
(bmalloc::BoundaryTag::hasPhysicalPages): Deleted.
|
||
(bmalloc::BoundaryTag::setHasPhysicalPages): Deleted. Replaced the concept
|
||
of "has physical pages" with a bit indicating which heap owns the large
|
||
object. This is a more precise concept, since the old bit was really a
|
||
Yes / Maybe bit.
|
||
|
||
* bmalloc/Deallocator.cpp:
|
||
|
||
* bmalloc/FreeList.cpp: Adopt
|
||
(bmalloc::FreeList::takeGreedy):
|
||
(bmalloc::FreeList::take):
|
||
(bmalloc::FreeList::removeInvalidAndDuplicateEntries):
|
||
* bmalloc/FreeList.h:
|
||
(bmalloc::FreeList::push): Added API for considering the owner when
|
||
deciding if a free list entry is valid.
|
||
|
||
* bmalloc/Heap.cpp:
|
||
(bmalloc::Heap::Heap): Adopt new API.
|
||
|
||
(bmalloc::Heap::scavengeLargeRanges): Scavenge all ranges with no minimum,
|
||
since some ranges might be able to merge with ranges in the VM heap, and
|
||
they won't be allowed to until we scavenge them.
|
||
|
||
(bmalloc::Heap::allocateSmallPage):
|
||
(bmalloc::Heap::allocateMediumPage):
|
||
(bmalloc::Heap::allocateLarge): New VM heap API makes this function
|
||
simpler, since we always get back physical pages now.
|
||
|
||
* bmalloc/Heap.h:
|
||
* bmalloc/LargeObject.h:
|
||
(bmalloc::LargeObject::end):
|
||
(bmalloc::LargeObject::owner):
|
||
(bmalloc::LargeObject::setOwner):
|
||
(bmalloc::LargeObject::isValidAndFree):
|
||
(bmalloc::LargeObject::merge): Do not merge objects across heaps since
|
||
that causes madvise churn.
|
||
(bmalloc::LargeObject::validateSelf):
|
||
(bmalloc::LargeObject::init):
|
||
(bmalloc::LargeObject::hasPhysicalPages): Deleted.
|
||
(bmalloc::LargeObject::setHasPhysicalPages): Deleted. Propogate the Owner API.
|
||
|
||
* bmalloc/Owner.h: Added.
|
||
|
||
* bmalloc/SegregatedFreeList.cpp:
|
||
(bmalloc::SegregatedFreeList::SegregatedFreeList):
|
||
(bmalloc::SegregatedFreeList::insert):
|
||
(bmalloc::SegregatedFreeList::takeGreedy):
|
||
(bmalloc::SegregatedFreeList::take):
|
||
* bmalloc/SegregatedFreeList.h: Propogate the owner API.
|
||
|
||
* bmalloc/VMAllocate.h:
|
||
(bmalloc::vmDeallocatePhysicalPagesSloppy):
|
||
(bmalloc::vmAllocatePhysicalPagesSloppy): Clarified these functions and
|
||
removed an edge case.
|
||
|
||
* bmalloc/VMHeap.cpp:
|
||
(bmalloc::VMHeap::VMHeap):
|
||
* bmalloc/VMHeap.h:
|
||
(bmalloc::VMHeap::allocateSmallPage):
|
||
(bmalloc::VMHeap::allocateMediumPage):
|
||
(bmalloc::VMHeap::allocateLargeObject):
|
||
(bmalloc::VMHeap::deallocateLargeObject): Be sure to give each object
|
||
a new chance to merge, since it might have been prohibited from merging
|
||
before by virtue of not being in the VM heap.
|
||
|
||
(bmalloc::VMHeap::allocateLargeRange): Deleted.
|
||
(bmalloc::VMHeap::deallocateLargeRange): Deleted.
|
||
|
||
2015-02-26 Geoffrey Garen <ggaren@apple.com>
|
||
|
||
bmalloc: Large object free list can grow infinitely
|
||
https://bugs.webkit.org/show_bug.cgi?id=142055
|
||
|
||
Reviewed by Andreas Kling.
|
||
|
||
By design, we don't eagerly remove large objects from the free list.
|
||
This creates two simple pathologies:
|
||
|
||
(1) If you free and then allocate the same object repeatedly, it will
|
||
duplicate itself in the free list repeatedly. Since it is never
|
||
invalid at the time of allocation, it will never be removed.
|
||
|
||
(2) If you split and then merge the same object repeatedly, it will
|
||
duplicate its split sibling in the free list repeatedly. If its
|
||
sibling is in a separate free list size class, it will never be
|
||
consulted at the time of allocation, so it will never be removed.
|
||
|
||
So, a simple "while (1) { free(malloc(x)); }" causes infinite memory
|
||
use in the free list.
|
||
|
||
The solution in this patch is a simple helper to remove garbage from the
|
||
free list if it grows too large. This pathology is not common, so the
|
||
cost is OK.
|
||
|
||
Long-term, perhaps we should rethink the laziness of these free lists.
|
||
|
||
* bmalloc/BoundaryTag.h:
|
||
(bmalloc::BoundaryTag::isMarked):
|
||
(bmalloc::BoundaryTag::setMarked): New bit, used by free list GC.
|
||
|
||
* bmalloc/FreeList.cpp:
|
||
(bmalloc::FreeList::removeInvalidAndDuplicateEntries): The GC algorithm.
|
||
|
||
* bmalloc/FreeList.h:
|
||
(bmalloc::FreeList::FreeList):
|
||
(bmalloc::FreeList::push): Invoke the GC if we're getting huge.
|
||
|
||
* bmalloc/LargeObject.h:
|
||
(bmalloc::LargeObject::isMarked):
|
||
(bmalloc::LargeObject::setMarked):
|
||
(bmalloc::LargeObject::validateSelf): Expose the new bit.
|
||
|
||
* bmalloc/Sizes.h: New constant to control GC frequency.
|
||
|
||
2015-02-26 Csaba Osztrogonác <ossy@webkit.org>
|
||
|
||
URTBF after r180693.
|
||
|
||
* CMakeLists.txt:
|
||
|
||
2015-02-26 Geoffrey Garen <ggaren@apple.com>
|
||
|
||
Try to fix the Mac build.
|
||
|
||
Unreviewed.
|
||
|
||
* bmalloc.xcodeproj/project.pbxproj: Make FreeList.h available.
|
||
|
||
2015-02-26 Geoffrey Garen <ggaren@apple.com>
|
||
|
||
bmalloc: Refactored SegregatedFreeList and BoundaryTag::init
|
||
https://bugs.webkit.org/show_bug.cgi?id=142049
|
||
|
||
Reviewed by Anders Carlsson.
|
||
|
||
Split out a FreeList class from SegregatedFreeList. This will make it
|
||
easier to add behaviors on free list insertion and removal -- and it's
|
||
probably how I should have designed things at the start.
|
||
|
||
Moved BoundaryTag::init into LargeObject, since all the related logic
|
||
lives in LargeObject now too, and this allows us to remove BoundaryTagInlines.h.
|
||
|
||
* bmalloc.xcodeproj/project.pbxproj:
|
||
* bmalloc/BoundaryTagInlines.h: Removed.
|
||
* bmalloc/FreeList.cpp: Copied from Source/bmalloc/bmalloc/SegregatedFreeList.cpp.
|
||
(bmalloc::FreeList::takeGreedy):
|
||
(bmalloc::FreeList::take):
|
||
(bmalloc::SegregatedFreeList::SegregatedFreeList): Deleted.
|
||
(bmalloc::SegregatedFreeList::insert): Deleted.
|
||
(bmalloc::SegregatedFreeList::takeGreedy): Deleted.
|
||
(bmalloc::SegregatedFreeList::take): Deleted.
|
||
* bmalloc/FreeList.h: Copied from Source/bmalloc/bmalloc/SegregatedFreeList.h.
|
||
(bmalloc::FreeList::push):
|
||
* bmalloc/LargeObject.h:
|
||
(bmalloc::LargeObject::init):
|
||
* bmalloc/SegregatedFreeList.cpp:
|
||
(bmalloc::SegregatedFreeList::SegregatedFreeList):
|
||
(bmalloc::SegregatedFreeList::insert):
|
||
(bmalloc::SegregatedFreeList::takeGreedy):
|
||
(bmalloc::SegregatedFreeList::take):
|
||
* bmalloc/SegregatedFreeList.h:
|
||
* bmalloc/Sizes.h:
|
||
* bmalloc/VMHeap.cpp:
|
||
(bmalloc::VMHeap::grow):
|
||
|
||
2015-02-26 Geoffrey Garen <ggaren@apple.com>
|
||
|
||
bmalloc: free up a bit in BoundaryTag
|
||
https://bugs.webkit.org/show_bug.cgi?id=142048
|
||
|
||
Reviewed by Brady Eidson.
|
||
|
||
We were wasting a bit by accident, and I need one now.
|
||
|
||
* bmalloc/Algorithm.h:
|
||
(bmalloc::rightShift): Deleted. Not needed, now that I've simplified
|
||
the math.
|
||
|
||
* bmalloc/BoundaryTag.h: Since each boundary tag bucket is 1024 bytes
|
||
long, the maximum offset into a bucket is 1023.
|
||
|
||
You need 5 bits to count up to 1024, but only 4 to count up to 1023.
|
||
|
||
Math is hard.
|
||
|
||
(bmalloc::BoundaryTag::compactBegin): Switched to division because it
|
||
is simpler, and easier to match up with our ASSERT. The compiler will
|
||
turn division by constant power of two into a shift for us.
|
||
|
||
(bmalloc::BoundaryTag::setRange): Added an ASSERT for compactBegin
|
||
because we do encode it, so we should ASSERT that encoding did not
|
||
lose information.
|
||
|
||
* bmalloc/Sizes.h: Shifting is no longer used since we use division
|
||
instead.
|
||
|
||
2015-02-24 Stephanie Lewis <slewis@apple.com>
|
||
|
||
Rolling out http://trac.webkit.org/changeset/180430 as it causes the PLT to crash.
|
||
<rdar://problem/19948015>
|
||
|
||
Unreviewed.
|
||
|
||
* bmalloc/VMHeap.cpp:
|
||
(bmalloc::VMHeap::grow):
|
||
* bmalloc/VMHeap.h:
|
||
* bmalloc/Zone.cpp:
|
||
(bmalloc::Zone::Zone):
|
||
(bmalloc::Zone::size): Deleted.
|
||
* bmalloc/Zone.h:
|
||
|
||
2015-02-24 Geoffrey Garen <ggaren@apple.com>
|
||
|
||
bmalloc: Added a little more abstraction for large objects
|
||
https://bugs.webkit.org/show_bug.cgi?id=141978
|
||
|
||
Reviewed by Sam Weinig.
|
||
|
||
Previously, each client needed to manage the boundary tags of
|
||
a large object using free functions. This patch introduces a LargeObject
|
||
class that does things a little more automatically.
|
||
|
||
* bmalloc.xcodeproj/project.pbxproj:
|
||
|
||
* bmalloc/Allocator.cpp:
|
||
(bmalloc::Allocator::reallocate): Use the new LargeObject class.
|
||
|
||
* bmalloc/BeginTag.h:
|
||
(bmalloc::BeginTag::isInFreeList): Deleted. Moved this logic into the
|
||
LargeObject class.
|
||
|
||
* bmalloc/BoundaryTag.h:
|
||
(bmalloc::BoundaryTag::isSentinel):
|
||
(bmalloc::BoundaryTag::compactBegin):
|
||
(bmalloc::BoundaryTag::setRange):
|
||
(bmalloc::BoundaryTag::initSentinel): Added an explicit API for sentinels,
|
||
which we used to create and test for implicitly.
|
||
|
||
* bmalloc/BoundaryTagInlines.h:
|
||
(bmalloc::BoundaryTag::init):
|
||
(bmalloc::validate): Deleted.
|
||
(bmalloc::validatePrev): Deleted.
|
||
(bmalloc::validateNext): Deleted.
|
||
(bmalloc::BoundaryTag::mergeLeft): Deleted.
|
||
(bmalloc::BoundaryTag::mergeRight): Deleted.
|
||
(bmalloc::BoundaryTag::merge): Deleted.
|
||
(bmalloc::BoundaryTag::deallocate): Deleted.
|
||
(bmalloc::BoundaryTag::split): Deleted.
|
||
(bmalloc::BoundaryTag::allocate): Deleted. Moved this logic into the
|
||
LargeObject class.
|
||
|
||
* bmalloc/EndTag.h:
|
||
(bmalloc::EndTag::init):
|
||
(bmalloc::EndTag::operator=): Deleted. Re-reading this code, I found
|
||
special behavior in the assignment operator to be a surprising API.
|
||
So, I replaced the assignment operation with an explicit initializing
|
||
function.
|
||
|
||
* bmalloc/Heap.cpp:
|
||
(bmalloc::Heap::scavengeLargeRanges):
|
||
(bmalloc::Heap::allocateXLarge):
|
||
(bmalloc::Heap::findXLarge):
|
||
(bmalloc::Heap::deallocateXLarge):
|
||
(bmalloc::Heap::allocateLarge):
|
||
(bmalloc::Heap::deallocateLarge):
|
||
* bmalloc/Heap.h: No behavior changes here -- just adopting the
|
||
LargeObject interface.
|
||
|
||
* bmalloc/LargeObject.h: Added.
|
||
(bmalloc::LargeObject::operator!):
|
||
(bmalloc::LargeObject::begin):
|
||
(bmalloc::LargeObject::size):
|
||
(bmalloc::LargeObject::range):
|
||
(bmalloc::LargeObject::LargeObject):
|
||
(bmalloc::LargeObject::setFree):
|
||
(bmalloc::LargeObject::isFree):
|
||
(bmalloc::LargeObject::hasPhysicalPages):
|
||
(bmalloc::LargeObject::setHasPhysicalPages):
|
||
(bmalloc::LargeObject::isValidAndFree):
|
||
(bmalloc::LargeObject::merge):
|
||
(bmalloc::LargeObject::split):
|
||
(bmalloc::LargeObject::validateSelf):
|
||
(bmalloc::LargeObject::validate): Moved this code into a class, out of
|
||
BoundaryTag free functions.
|
||
|
||
New to the class are these features:
|
||
|
||
(1) Every reference to an object is validated upon creation and use.
|
||
|
||
(2) There's an explicit API for "This is a reference to an object
|
||
that might be stale (the DoNotValidate API)".
|
||
|
||
(3) The begin and end tags are kept in sync automatically.
|
||
|
||
* bmalloc/SegregatedFreeList.cpp:
|
||
(bmalloc::SegregatedFreeList::insert):
|
||
(bmalloc::SegregatedFreeList::takeGreedy):
|
||
(bmalloc::SegregatedFreeList::take):
|
||
* bmalloc/SegregatedFreeList.h: Adopt the LargeObject interface.
|
||
|
||
* bmalloc/VMHeap.cpp:
|
||
(bmalloc::VMHeap::grow):
|
||
* bmalloc/VMHeap.h:
|
||
(bmalloc::VMHeap::allocateLargeRange):
|
||
(bmalloc::VMHeap::deallocateLargeRange): Adopt the LargeObject interface.
|
||
|
||
2015-02-20 Geoffrey Garen <ggaren@apple.com>
|
||
|
||
bmalloc should implement malloc introspection (to stop false-positive leaks when MallocStackLogging is off)
|
||
https://bugs.webkit.org/show_bug.cgi?id=141802
|
||
|
||
Reviewed by Andreas Kling.
|
||
|
||
Rolling back in with a fix for a crash seen while using GuardMalloc.
|
||
|
||
* bmalloc/VMHeap.cpp:
|
||
(bmalloc::VMHeap::grow):
|
||
* bmalloc/VMHeap.h:
|
||
* bmalloc/Zone.cpp: Re-land the old patch.
|
||
|
||
(bmalloc::Zone::size): Be sure to implement the size() function since
|
||
it's accessible indirectly via the malloc_zone_from_ptr public API --
|
||
and GuardMalloc calls it all the time.
|
||
|
||
(bmalloc::Zone::Zone):
|
||
* bmalloc/Zone.h: Re-land the old patch.
|
||
|
||
2015-02-19 Commit Queue <commit-queue@webkit.org>
|
||
|
||
Unreviewed, rolling out r180363.
|
||
https://bugs.webkit.org/show_bug.cgi?id=141814
|
||
|
||
Caused >50 crashes when running LayoutTests in GuardMalloc or
|
||
ASAN modes. (Requested by jernoble on #webkit).
|
||
|
||
Reverted changeset:
|
||
|
||
"bmalloc should implement malloc introspection (to stop false-
|
||
positive leaks when MallocStackLogging is off)"
|
||
https://bugs.webkit.org/show_bug.cgi?id=141802
|
||
http://trac.webkit.org/changeset/180363
|
||
|
||
2015-02-19 Geoffrey Garen <ggaren@apple.com>
|
||
|
||
bmalloc should implement malloc introspection (to stop false-positive leaks when MallocStackLogging is off)
|
||
https://bugs.webkit.org/show_bug.cgi?id=141802
|
||
|
||
Reviewed by Andreas Kling.
|
||
|
||
Fixed a last-minute type.
|
||
|
||
The macro is OS, not PLATFORM.
|
||
|
||
* bmalloc/VMHeap.cpp:
|
||
(bmalloc::VMHeap::grow):
|
||
* bmalloc/VMHeap.h:
|
||
* bmalloc/Zone.h:
|
||
|
||
2015-02-19 Geoffrey Garen <ggaren@apple.com>
|
||
|
||
bmalloc should implement malloc introspection (to stop false-positive leaks when MallocStackLogging is off)
|
||
https://bugs.webkit.org/show_bug.cgi?id=141802
|
||
|
||
Reviewed by Andreas Kling.
|
||
|
||
This patch does the bare minimum to stop false positive leaks from
|
||
being reported by the Darwin leaks tool. We register each super chunk
|
||
as a single object, and then request that the leaks tool scan it.
|
||
|
||
* bmalloc.xcodeproj/project.pbxproj: Added an abstraction for the malloc
|
||
zone introspection API.
|
||
|
||
* bmalloc/Algorithm.h: Missing #include.
|
||
|
||
* bmalloc/VMHeap.cpp:
|
||
(bmalloc::VMHeap::grow):
|
||
* bmalloc/VMHeap.h: Adopt the new abstraction.
|
||
|
||
* bmalloc/Zone.cpp: Added.
|
||
(bmalloc::remoteRead): Helper for reading an object out of another process.
|
||
(bmalloc::Zone::enumerator):
|
||
(bmalloc::Zone::Zone): Register a malloc zone so that we will participate
|
||
in introspection.
|
||
|
||
* bmalloc/Zone.h: Added.
|
||
(bmalloc::Zone::superChunks):
|
||
(bmalloc::Zone::addSuperChunk): Use a non-dynamically-allocated vector
|
||
since our dynamic allocations will not be scanned by leaks since they
|
||
will have the malloc VM tag.
|
||
|
||
2015-02-18 Geoffrey Garen <ggaren@apple.com>
|
||
|
||
bmalloc: VMHeap should keep a record of all of its VM ranges (for malloc introspection)
|
||
https://bugs.webkit.org/show_bug.cgi?id=141759
|
||
|
||
Reviewed by Andreas Kling.
|
||
|
||
* bmalloc.xcodeproj/project.pbxproj:
|
||
* bmalloc/SuperChunk.h: Added.
|
||
(bmalloc::SuperChunk::create):
|
||
(bmalloc::SuperChunk::SuperChunk):
|
||
(bmalloc::SuperChunk::smallChunk):
|
||
(bmalloc::SuperChunk::mediumChunk):
|
||
(bmalloc::SuperChunk::largeChunk): Factored out super chunk creation
|
||
into a separate class, for clarity and type safety.
|
||
|
||
* bmalloc/VMHeap.cpp:
|
||
(bmalloc::VMHeap::grow):
|
||
(bmalloc::VMHeap::allocateSuperChunk): Renamed "allocateSuperChunk" to
|
||
"grow" because Andreas found "allocateSuperChunk" to be unclear.
|
||
|
||
* bmalloc/VMHeap.h: Track all our VM ranges. We will use this information
|
||
for malloc introspection.
|
||
|
||
(bmalloc::VMHeap::allocateSmallPage):
|
||
(bmalloc::VMHeap::allocateMediumPage):
|
||
(bmalloc::VMHeap::allocateLargeRange): Updated for renames.
|
||
|
||
2015-02-18 Zan Dobersek <zdobersek@igalia.com>
|
||
|
||
Build bmalloc through CMake as a static library. It's then linked either
|
||
into the WTF library (if built as a shared library) or into the JSC and
|
||
WebKit2 libraries. There's no need to build it as a standalone shared library.
|
||
|
||
Rubber-stamped by Carlos Garcia Campos.
|
||
|
||
* CMakeLists.txt:
|
||
|
||
2015-02-13 Gyuyoung Kim <gyuyoung.kim@samsung.com>
|
||
|
||
[BMalloc] Add a FIXME comment for memory alignas
|
||
https://bugs.webkit.org/show_bug.cgi?id=141556
|
||
|
||
Reviewed by Csaba Osztrogonác.
|
||
|
||
* bmalloc/Chunk.h: Add a FIXME comment.
|
||
* bmalloc/LargeChunk.h: ditto.
|
||
|
||
2015-02-11 Csaba Osztrogonác <ossy@webkit.org>
|
||
|
||
bmalloc buildfix on 32 bit Linux (x86/ARM)
|
||
https://bugs.webkit.org/show_bug.cgi?id=141472
|
||
|
||
Reviewed by Gyuyoung Kim.
|
||
|
||
* bmalloc/Algorithm.h:
|
||
(bmalloc::roundUpToMultipleOf):
|
||
* bmalloc/FixedVector.h:
|
||
(bmalloc::FixedVector::clear):
|
||
* bmalloc/Sizes.h:
|
||
(bmalloc::Sizes::sizeClass):
|
||
|
||
2015-02-11 Gyuyoung Kim <gyuyoung.kim@samsung.com>
|
||
|
||
[EFL][GTK] Use bmalloc instead of tcmalloc
|
||
https://bugs.webkit.org/show_bug.cgi?id=140162
|
||
|
||
Reviewed by Carlos Garcia Campos.
|
||
|
||
Support to use bmalloc on EFL and GTK ports.
|
||
|
||
* CMakeLists.txt: Added.
|
||
* bmalloc/Allocator.cpp:
|
||
(bmalloc::Allocator::allocate):
|
||
Fix unused return value caused by posix_memalign().
|
||
* bmalloc/AsyncTask.h:
|
||
* bmalloc/BoundaryTag.h:
|
||
(bmalloc::BoundaryTag::clear):
|
||
* bmalloc/Chunk.h:
|
||
Change Traits::Page with Traits::PageType in order to fix
|
||
-fpermitive build error on EFL and GTK port.
|
||
* bmalloc/EndTag.h:
|
||
(bmalloc::EndTag::operator=):
|
||
* bmalloc/Line.h: ditto.
|
||
* bmalloc/MediumTraits.h:
|
||
* bmalloc/Page.h: ditto.
|
||
* bmalloc/PerThread.h:
|
||
EFL port doesn't support __has_include definition yet.
|
||
Define HAVE_PTHREAD_MACHDEP_H according to check if __has_include is supported.
|
||
* bmalloc/SmallTraits.h: ditto.
|
||
* bmalloc/VMAllocate.h:
|
||
(bmalloc::vmDeallocatePhysicalPages):
|
||
(bmalloc::vmAllocatePhysicalPages):
|
||
* bmalloc/Vector.h:
|
||
(bmalloc::Vector<T>::push):
|
||
(bmalloc::Vector<T>::reallocateBuffer):
|
||
|
||
2015-01-31 Sam Weinig <sam@webkit.org>
|
||
|
||
Remove even more Mountain Lion support
|
||
https://bugs.webkit.org/show_bug.cgi?id=141124
|
||
|
||
Reviewed by Alexey Proskuryakov.
|
||
|
||
* Configurations/Base.xcconfig:
|
||
* Configurations/DebugRelease.xcconfig:
|
||
|
||
2015-01-30 Geoffrey Garen <ggaren@apple.com>
|
||
|
||
GC marking threads should clear malloc caches
|
||
https://bugs.webkit.org/show_bug.cgi?id=141097
|
||
|
||
Reviewed by Andreas Kling.
|
||
|
||
Split the scavenging API into per-thread vs global, so that you can
|
||
request to scavenge your own thread without scavenging the whole heap.
|
||
|
||
* bmalloc/Cache.cpp:
|
||
(bmalloc::Cache::scavenge):
|
||
* bmalloc/bmalloc.h:
|
||
(bmalloc::api::scavengeThisThread):
|
||
(bmalloc::api::scavenge):
|
||
|
||
2015-01-28 Dana Burkart <dburkart@apple.com>
|
||
|
||
Move ASan flag settings from DebugRelease.xcconfig to Base.xcconfig
|
||
https://bugs.webkit.org/show_bug.cgi?id=136765
|
||
|
||
Reviewed by Alexey Proskuryakov.
|
||
|
||
* Configurations/Base.xcconfig:
|
||
* Configurations/DebugRelease.xcconfig:
|
||
|
||
2015-01-21 Geoffrey Garen <ggaren@apple.com>
|
||
|
||
bmalloc: support aligned allocation
|
||
https://bugs.webkit.org/show_bug.cgi?id=140732
|
||
|
||
Reviewed by Andreas Kling.
|
||
|
||
* bmalloc/Allocator.cpp:
|
||
(bmalloc::Allocator::allocate): New function for aligned allocation.
|
||
|
||
Small and medium requests just allocate and free until they find an
|
||
aligned pointer. This is slightly inefficient in the worst case, but
|
||
still constant-time with little-to-no space overhead.
|
||
|
||
Large requests use a new API that requires the client to specify both
|
||
its ideal size and alignment, and the worst-case size you would have to
|
||
allocate in order to produce some interior pointer of the requested size
|
||
and alignment. We put the burden of this calculation on the client
|
||
because it simplifies things if we guarantee that allocation won't fail.
|
||
|
||
XLarge requests are easy: we just forward them to vmAllocate, which
|
||
already supported aligned requests.
|
||
|
||
* bmalloc/BoundaryTag.h:
|
||
* bmalloc/BoundaryTagInlines.h:
|
||
(bmalloc::BoundaryTag::mergeLeft):
|
||
(bmalloc::BoundaryTag::mergeRight):
|
||
(bmalloc::BoundaryTag::merge):
|
||
(bmalloc::BoundaryTag::deallocate):
|
||
(bmalloc::BoundaryTag::split):
|
||
(bmalloc::BoundaryTag::allocate): No behavior change here. I just
|
||
refactored the interface to remove some reference out parameters in
|
||
order to clarify what changes and what doesn't.
|
||
|
||
* bmalloc/Heap.cpp:
|
||
(bmalloc::Heap::allocateXLarge): Added an alignment API.
|
||
|
||
(bmalloc::Heap::allocateLarge):
|
||
* bmalloc/Heap.h: Added an alignment API. I split out allocateLarge into
|
||
a few variants, so aligned and unaligned allocation could share some code.
|
||
|
||
* bmalloc/SegregatedFreeList.cpp:
|
||
(bmalloc::SegregatedFreeList::take):
|
||
* bmalloc/SegregatedFreeList.h: Changed to use a separate, explicit API
|
||
for aligned allocation. It turns out that the aligned path is pretty
|
||
different, since it ends up searching for two potential ways to satisfy
|
||
an allocation: either large enough and aligned, or large enough to split
|
||
into something not aligned and something large enough and aligned.
|
||
|
||
* bmalloc/VMAllocate.h:
|
||
(bmalloc::vmAllocate): Switched alignment to come before size because
|
||
that's how the memalign API specifies it.
|
||
|
||
* bmalloc/VMHeap.h:
|
||
(bmalloc::VMHeap::allocateLargeRange): Added an alignment API.
|
||
|
||
2015-01-20 Geoffrey Garen <ggaren@apple.com>
|
||
|
||
bmalloc: a little bit of cleanup
|
||
https://bugs.webkit.org/show_bug.cgi?id=140687
|
||
|
||
Reviewed by Anders Carlsson.
|
||
|
||
* bmalloc/Algorithm.h:
|
||
(bmalloc::isPowerOfTwo): Added a check for 0, since 0 would break a lot
|
||
of code.
|
||
|
||
* bmalloc/BoundaryTag.h:
|
||
* bmalloc/BoundaryTagInlines.h:
|
||
(bmalloc::BoundaryTag::mergeLeft):
|
||
(bmalloc::BoundaryTag::mergeRight):
|
||
(bmalloc::BoundaryTag::merge):
|
||
(bmalloc::BoundaryTag::deallocate):
|
||
(bmalloc::BoundaryTag::split):
|
||
(bmalloc::BoundaryTag::allocate):
|
||
(bmalloc::BoundaryTag::mergeLargeLeft): Deleted.
|
||
(bmalloc::BoundaryTag::mergeLargeRight): Deleted.
|
||
(bmalloc::BoundaryTag::mergeLarge): Deleted.
|
||
(bmalloc::BoundaryTag::splitLarge): Deleted. Removed the word "Large"
|
||
from all these functions, since boundary tags always pertain to large
|
||
objects, and putting the word "Large" everywhere wasn't helping to
|
||
explain that.
|
||
|
||
* bmalloc/Heap.cpp:
|
||
(bmalloc::Heap::allocateXLarge):
|
||
(bmalloc::Heap::findXLarge):
|
||
(bmalloc::Heap::deallocateXLarge):
|
||
* bmalloc/Heap.h:
|
||
* bmalloc/VMHeap.h:
|
||
(bmalloc::VMHeap::allocateXLarge): Deleted.
|
||
(bmalloc::VMHeap::findXLarge): Deleted.
|
||
(bmalloc::VMHeap::deallocateXLarge): Deleted. Moved XLarge allocation
|
||
from VMHeap to Heap. Since the purpose of the VMHeap is to cache VM
|
||
ranges, and the VMHeap never caches any XLarge ranges, it doesn't
|
||
really make sense for the VMHeap to be involved.
|
||
|
||
2015-01-16 Geoffrey Garen <ggaren@apple.com>
|
||
|
||
bmalloc: refactored XLarge allocation for better alignment
|
||
https://bugs.webkit.org/show_bug.cgi?id=140582
|
||
|
||
Reviewed by Andreas Kling.
|
||
|
||
XLarge objects used to be Large objects with an extra bit of metadata
|
||
that said "actually, I'm not large -- I'm extra large".
|
||
|
||
The metadata header in an XLarge allocation made it impossible for the
|
||
XLarge object to honor a very large alignment request.
|
||
|
||
The solution is to stop using a metadata header for XLarge objects, and
|
||
instead to store explicit metadata on the side.
|
||
|
||
This is a bit less astonishing, which is also nice.
|
||
|
||
Finding XLarge metadata is now a linear search. That's probably OK, since
|
||
it was always so in TCMalloc, and the usual number of XLarge allocations
|
||
in a process is 0.
|
||
|
||
This design makes it possible for the heap to cache XLarge allocations
|
||
with and/or without physical pages. I haven't actually done that yet
|
||
because the tradeoffs are subtle, so I don't want to do anything without
|
||
a motivating test case.
|
||
|
||
* bmalloc.xcodeproj/project.pbxproj:
|
||
* bmalloc/Allocator.cpp:
|
||
(bmalloc::Allocator::reallocate): Removed the concept of an XLargeChunk,
|
||
since an XLarge allocation is now just a naked buffer without a header.
|
||
|
||
(bmalloc::Allocator::allocateXLarge): Added an explicit qualifier for
|
||
XLarge alignment, since XLargeChunk won't give this to us implicitly
|
||
anymore.
|
||
|
||
* bmalloc/BoundaryTag.h:
|
||
(bmalloc::BoundaryTag::setRange):
|
||
(bmalloc::BoundaryTag::isXLarge): Deleted.
|
||
(bmalloc::BoundaryTag::setXLarge): Deleted.
|
||
* bmalloc/BoundaryTagInlines.h:
|
||
(bmalloc::validate):
|
||
(bmalloc::BoundaryTag::deallocate): Removed the XLarge hacks from Large allocations.
|
||
|
||
* bmalloc/Deallocator.cpp:
|
||
(bmalloc::Deallocator::deallocateXLarge):
|
||
(bmalloc::Deallocator::deallocateSlowCase):
|
||
* bmalloc/Heap.cpp:
|
||
(bmalloc::Heap::findXLarge):
|
||
(bmalloc::Heap::allocateXLarge):
|
||
(bmalloc::Heap::deallocateXLarge):
|
||
* bmalloc/Heap.h: Updated for interface changes.
|
||
|
||
* bmalloc/ObjectType.cpp:
|
||
(bmalloc::objectType):
|
||
* bmalloc/ObjectType.h:
|
||
(bmalloc::isXLarge): We can now tell if a pointer is XLarge just by
|
||
examining its bit pattern -- just like we do for other kinds of
|
||
allocations -- which is nice.
|
||
|
||
* bmalloc/Sizes.h:
|
||
* bmalloc/VMHeap.h:
|
||
(bmalloc::VMHeap::allocateXLarge):
|
||
(bmalloc::VMHeap::findXLarge):
|
||
(bmalloc::VMHeap::deallocateXLarge): Keep an explicit vector of metadata
|
||
for XLarge allocations.
|
||
|
||
* bmalloc/XLargeChunk.h: Removed.
|
||
|
||
2015-01-16 Geoffrey Garen <ggaren@apple.com>
|
||
|
||
bmalloc: added some infrastructure for aligned allocation
|
||
https://bugs.webkit.org/show_bug.cgi?id=140572
|
||
|
||
Reviewed by Andreas Kling.
|
||
|
||
* bmalloc/Algorithm.h:
|
||
(bmalloc::isPowerOfTwo):
|
||
(bmalloc::roundUpToMultipleOf):
|
||
(bmalloc::roundDownToMultipleOf): Refactored some duplicate code to use our
|
||
isPowerOfTwo helper function.
|
||
|
||
* bmalloc/Allocator.cpp:
|
||
(bmalloc::Allocator::allocate):
|
||
* bmalloc/Allocator.h: Stubbed out an implementation of aligned allocation.
|
||
Doesn't do anything yet, but does correctly forward to system malloc
|
||
when bmalloc is disabled.
|
||
|
||
* bmalloc/Cache.cpp:
|
||
(bmalloc::Cache::allocateSlowCaseNullCache):
|
||
* bmalloc/Cache.h:
|
||
(bmalloc::Cache::allocate):
|
||
* bmalloc/bmalloc.h:
|
||
(bmalloc::api::memalign):
|
||
* bmalloc/mbmalloc.cpp: Stubbed out an API for aligned allocation.
|
||
|
||
2015-01-13 Geoffrey Garen <ggaren@apple.com>
|
||
|
||
Consider alignment when allocating from a SegregatedFreeList
|
||
https://bugs.webkit.org/show_bug.cgi?id=140408
|
||
|
||
Reviewed by Sam Weinig.
|
||
|
||
In preparation for supporting aligned allocation.
|
||
|
||
No performance change.
|
||
|
||
Since this is just one extra branch in an already expensive function,
|
||
I decided not to duplicate the function just to avoid the branch in
|
||
the un-aligned case.
|
||
|
||
* bmalloc/SegregatedFreeList.cpp:
|
||
(bmalloc::SegregatedFreeList::take):
|
||
* bmalloc/SegregatedFreeList.h:
|
||
|
||
2015-01-13 Geoffrey Garen <ggaren@apple.com>
|
||
|
||
Renamed minimum to size in SegregatedFreeList
|
||
https://bugs.webkit.org/show_bug.cgi?id=140406
|
||
|
||
Reviewed by Sam Weinig.
|
||
|
||
In preparation for supporting aligned allocation.
|
||
|
||
* bmalloc/SegregatedFreeList.cpp:
|
||
(bmalloc::SegregatedFreeList::takeGreedy):
|
||
(bmalloc::SegregatedFreeList::take): Every size passed to malloc is
|
||
really just a minimum. Let's not imply that this value is special.
|
||
|
||
2015-01-11 Dan Bernstein <mitz@apple.com>
|
||
|
||
Geoff is organized, but he is not an organization.
|
||
|
||
Rubber-stamped by Anders Carlsson.
|
||
|
||
* bmalloc.xcodeproj/project.pbxproj: Removed the ORGANIZATIONNAME project attribute.
|
||
|
||
2015-01-07 Geoffrey Garen <ggaren@apple.com>
|
||
|
||
Make bmalloc work with ASan
|
||
https://bugs.webkit.org/show_bug.cgi?id=140194
|
||
|
||
Reviewed by Mark Lam.
|
||
|
||
* bmalloc/BPlatform.h: Added a way to detect Darwin OSes, since we need
|
||
an OS-specific API to test for loaded runtime libraries.
|
||
|
||
* bmalloc/Environment.cpp:
|
||
(bmalloc::isASanEnabled):
|
||
(bmalloc::Environment::computeIsBmallocEnabled): Disabled bmalloc if
|
||
ASan is enabled, since system malloc has the Asan hooks we need.
|
||
|
||
You could check for the ASan compile-time flag instead, but doing this
|
||
check at runtime prepares bmalloc for a world where it is a dynamic
|
||
library that might be loaded into projects it did not compile with.
|
||
|
||
2015-01-05 Geoffrey Garen <ggaren@apple.com>
|
||
|
||
Fix up bmalloc's PerThread for use on Linux
|
||
https://bugs.webkit.org/show_bug.cgi?id=139804
|
||
|
||
Reviewed by Anders Carlsson.
|
||
|
||
The previous implementation was a bit slow.
|
||
|
||
* bmalloc/PerThread.h:
|
||
(bmalloc::PerThreadStorage<Cache>::get):
|
||
(bmalloc::PerThreadStorage::get):
|
||
(bmalloc::PerThreadStorage::init): Added a catch-all cross-platform Unix
|
||
way to do fast per-thread access without taking a lock every time. This
|
||
probably works on all the platforms we care about, and it matches other
|
||
techniques we use elsewhere in WebKit.
|
||
|
||
(bmalloc::PerThread<T>::getFastCase): Removed the conditional from
|
||
this class because PerThreadStorage now encapsulates everything that
|
||
needs to be conditional.
|
||
|
||
(bmalloc::PerThreadStorage::initSharedKeyIfNeeded): Deleted.
|
||
|
||
2014-12-26 Dan Bernstein <mitz@apple.com>
|
||
|
||
<rdar://problem/19348208> REGRESSION (r177027): iOS builds use the wrong toolchain
|
||
https://bugs.webkit.org/show_bug.cgi?id=139950
|
||
|
||
Reviewed by David Kilzer.
|
||
|
||
* Configurations/Base.xcconfig: Only define TOOLCHAINS when building for OS X, doing so
|
||
in a manner that works with Xcode 5.1.1.
|
||
|
||
2014-12-15 Geoffrey Garen <ggaren@apple.com>
|
||
|
||
Safari crashes when you set Malloc environment variables
|
||
https://bugs.webkit.org/show_bug.cgi?id=139656
|
||
|
||
Reviewed by Michael Saboff.
|
||
|
||
I forgot to cover the realloc() case. Whoops. (OoPS?)
|
||
|
||
This time around, I ran the full MallocBench test suite in Malloc=1
|
||
mode, and it passed.
|
||
|
||
* bmalloc/Allocator.cpp:
|
||
(bmalloc::Allocator::reallocate):
|
||
* bmalloc/Allocator.h: Pushed realloc() logic down into the allocator.
|
||
It needs to be down there so that we can do the short-circuiting check
|
||
for whether bmalloc is enabled first.
|
||
|
||
Also added the check.
|
||
|
||
* bmalloc/Cache.cpp:
|
||
(bmalloc::Cache::scavenge):
|
||
(bmalloc::Cache::Cache):
|
||
(bmalloc::Cache::reallocateSlowCaseNullCache):
|
||
* bmalloc/Cache.h:
|
||
(bmalloc::Cache::deallocator):
|
||
(bmalloc::Cache::reallocate): Ditto.
|
||
|
||
* bmalloc/bmalloc.h:
|
||
(bmalloc::api::free):
|
||
(bmalloc::api::realloc): Ditto.
|
||
|
||
(bmalloc::api::scavenge): Pushed this down into Cache to match the
|
||
surrounding functions.
|
||
|
||
2014-12-11 Geoffrey Garen <ggaren@apple.com>
|
||
|
||
bmalloc should support system memory analysis tools (part 2)
|
||
https://bugs.webkit.org/show_bug.cgi?id=139565
|
||
|
||
Reviewed by Mark Lam.
|
||
|
||
This patch actually queries the environment to see if memory analysis
|
||
tools have been enabled.
|
||
|
||
* bmalloc/Deallocator.cpp:
|
||
(bmalloc::Deallocator::scavenge): Don't process the object log if
|
||
we've disabled bmalloc because it will be full of invalid nullptrs.
|
||
|
||
* bmalloc/Environment.cpp:
|
||
(bmalloc::isMallocEnvironmentVariableSet): Test for the list of known
|
||
Malloc debugging flags. I also added a plain "Malloc" catch-all for
|
||
when you want to disable bmalloc without enabling any kind of funny
|
||
business.
|
||
|
||
It would be slightly nicer just to iterate the list of environment
|
||
variables and strstr them, but getenv is the more portable option,
|
||
and performance here doesn't really matter.
|
||
|
||
(bmalloc::isLibgmallocEnabled): Test for the libgmalloc insertion
|
||
environment variable.
|
||
|
||
(bmalloc::Environment::computeIsBmallocEnabled):
|
||
|
||
2014-12-11 Geoffrey Garen <ggaren@apple.com>
|
||
|
||
Try to fix the iOS simulator build.
|
||
|
||
#include the declaration of malloc / free.
|
||
|
||
* bmalloc/Allocator.cpp:
|
||
* bmalloc/Deallocator.cpp:
|
||
|
||
2014-12-11 Geoffrey Garen <ggaren@apple.com>
|
||
|
||
Try to fix the build.
|
||
|
||
* bmalloc.xcodeproj/project.pbxproj: Marked a header exported.
|
||
|
||
2014-12-11 Geoffrey Garen <ggaren@apple.com>
|
||
|
||
bmalloc should support system memory analysis tools (part 1)
|
||
https://bugs.webkit.org/show_bug.cgi?id=139559
|
||
|
||
Reviewed by Mark Lam.
|
||
|
||
This patch adds the hooks to disable bmalloc at runtime if certain
|
||
environment variables are set, but doesn't actually read from the
|
||
environment yet.
|
||
|
||
No performance change.
|
||
|
||
* bmalloc.xcodeproj/project.pbxproj: Added the Environment class, which
|
||
we'll use to read environment variables and see if memory analysis tools
|
||
have been enabled.
|
||
|
||
* bmalloc/Allocator.cpp:
|
||
(bmalloc::Allocator::Allocator):
|
||
(bmalloc::Allocator::allocateSlowCase): Added a hook to disable bmalloc
|
||
on the allocation path. We cache the setting to make the check fast.
|
||
|
||
* bmalloc/Allocator.h: Interface changes.
|
||
|
||
* bmalloc/Cache.cpp:
|
||
(bmalloc::Cache::Cache): Pass a heap pointer through to our allocator
|
||
and deallocator. This main purpose is to enable them to query the
|
||
environment for whether bmalloc is enabled; but this is also a slightly
|
||
cleaner way to guarantee to them that the Heap has been pre-initialized.
|
||
|
||
* bmalloc/Deallocator.cpp:
|
||
(bmalloc::Deallocator::Deallocator): If bmalloc is disable, artificially
|
||
fill the object log to force us to take the slow path on all deallocations.
|
||
|
||
(bmalloc::Deallocator::deallocateSlowCase): Do the disabled check.
|
||
|
||
* bmalloc/Deallocator.h: Interface changes.
|
||
|
||
* bmalloc/Environment.cpp: Added.
|
||
(bmalloc::Environment::Environment):
|
||
(bmalloc::Environment::computeIsBmallocEnabled):
|
||
* bmalloc/Environment.h: Added.
|
||
(bmalloc::Environment::isBmallocEnabled): This is the class that will
|
||
encapsulate looking for environment variables that turn on heap
|
||
analysis tools.
|
||
|
||
* bmalloc/Heap.h:
|
||
(bmalloc::Heap::environment):
|
||
|
||
* bmalloc/Mutex.h:
|
||
(bmalloc::Mutex::Mutex):
|
||
* bmalloc/StaticMutex.h: A little refactoring to clarify these comments,
|
||
since I got super confused about them while writing this patch.
|
||
|
||
* bmalloc/VMHeap.cpp: Fixed an #include.
|
||
|
||
2014-12-09 David Kilzer <ddkilzer@apple.com>
|
||
|
||
Switch from using PLATFORM_NAME to SDK selectors in ANGLE, bmalloc, gtest, JavaScriptCore, WTF
|
||
<http://webkit.org/b/139212>
|
||
|
||
Reviewed by Joseph Pecoraro.
|
||
|
||
* Configurations/Base.xcconfig:
|
||
- Only set GCC_ENABLE_OBJC_GC, GCC_MODEL_TUNING and TOOLCHAINS
|
||
on OS X.
|
||
* Configurations/DebugRelease.xcconfig:
|
||
- Only set MACOSX_DEPLOYMENT_TARGET and SDKROOT on OS X.
|
||
|
||
2014-11-07 Geoffrey Garen <ggaren@apple.com>
|
||
|
||
bmalloc uses 8X more virtual memory than necessary
|
||
https://bugs.webkit.org/show_bug.cgi?id=138495
|
||
|
||
Reviewed by Mark Lam.
|
||
|
||
iOS has a per-process virtual memory cap around 1GB, so there's some
|
||
value to not going totally ham with virtual memory.
|
||
|
||
We currently use about 8X the necessary amount:
|
||
- 2X to align our VM allocation
|
||
- 4X to reserve small / medium / (2) large chunk VM ranges per superchunk
|
||
|
||
We can cut that down:
|
||
- Return the unaligned portion of our VM allocation (-2X)
|
||
- Use all the chunks in a superchunk, instead of allocating one
|
||
chunk per superchunk (-4X)
|
||
|
||
* bmalloc/Algorithm.h:
|
||
(bmalloc::roundUpToMultipleOf): Added a non-constant version of this
|
||
function so we can call it with getpagesize() at runtime.
|
||
|
||
* bmalloc/Chunk.h:
|
||
* bmalloc/LargeChunk.h:
|
||
(bmalloc::LargeChunk::create): Deleted. Instead of each chunk allocating
|
||
its own VM, VMHeap allocates the superchunk and all the chunks in it at a time.
|
||
|
||
* bmalloc/VMAllocate.h:
|
||
(bmalloc::vmValidate):
|
||
(bmalloc::vmAllocate): ASSERT that mmap succeeds to make crashes clearer
|
||
if it does not succeed. Allocate precisely, and give back the extra.
|
||
|
||
* bmalloc/VMHeap.cpp:
|
||
(bmalloc::VMHeap::allocateSuperChunk):
|
||
(bmalloc::VMHeap::allocateSmallChunk): Deleted.
|
||
(bmalloc::VMHeap::allocateMediumChunk): Deleted.
|
||
(bmalloc::VMHeap::allocateLargeChunk): Deleted. Use all the chunks
|
||
in a superchunk, instead of just one.
|
||
|
||
* bmalloc/VMHeap.h:
|
||
(bmalloc::VMHeap::allocateSmallPage):
|
||
(bmalloc::VMHeap::allocateMediumPage):
|
||
(bmalloc::VMHeap::allocateLargeRange):
|
||
* bmalloc/XLargeChunk.h:
|
||
(bmalloc::XLargeChunk::create): Updated to match changes above.
|
||
|
||
2014-11-01 David Kilzer <ddkilzer@apple.com>
|
||
|
||
JavaScriptCore is missing debug info for bmalloc because libbmalloc.a is stripped
|
||
<https://webkit.org/b/138286>
|
||
<rdar://problem/18847087>
|
||
|
||
Reviewed by Dan Bernstein.
|
||
|
||
* Configurations/bmalloc.xcconfig: Set STRIP_INSTALLED_PRODUCT
|
||
to NO for the target that produces libbmalloc.a so that the
|
||
debug symbols will be linked into JavaScriptCore and end up in
|
||
its dSYM file.
|
||
|
||
2014-10-30 Dana Burkart <dburkart@apple.com>
|
||
|
||
<rdar://problem/18821260> Prepare for the mysterious future
|
||
|
||
Reviewed by Lucas Forschler.
|
||
|
||
* Configurations/Base.xcconfig:
|
||
* Configurations/DebugRelease.xcconfig:
|
||
|
||
2014-09-24 Geoffrey Garen <ggaren@apple.com>
|
||
|
||
bmalloc: cleaned up fast path vs slow path
|
||
https://bugs.webkit.org/show_bug.cgi?id=137081
|
||
|
||
Reviewed by Sam Weinig.
|
||
|
||
Might be a 1% speedup on MallocBench. Also cleans up the code a bit.
|
||
|
||
* bmalloc/Allocator.cpp:
|
||
(bmalloc::Allocator::Allocator): Merged the small and medium range
|
||
caches, just like the small and medium allocators. Ranges are abstract
|
||
objects that don't really care whether they hold small or medium objects,
|
||
so they don't need to be segregated.
|
||
|
||
(bmalloc::Allocator::scavenge): Ditto.
|
||
|
||
(bmalloc::Allocator::allocateBumpRangeSlowCase):
|
||
(bmalloc::Allocator::allocateBumpRange): Same thing here, except that
|
||
we do care a tiny bit, because we need to specify small vs medium when
|
||
allocating new ranges from the heap, to ensure that the heap allocates
|
||
from the right segment of VM.
|
||
|
||
(bmalloc::Allocator::allocateLarge):
|
||
(bmalloc::Allocator::allocateXLarge): NO_INLINE because this was clouding
|
||
up the fast path. Large allocation performance is dominated by allocation
|
||
logic and initialization, so inlining it doesn't help.
|
||
|
||
(bmalloc::Allocator::allocateSlowCase): Slow path got a bit cleaner since
|
||
it doesn't need to distinguish small vs medium objects.
|
||
|
||
(bmalloc::Allocator::allocateSmallBumpRange): Deleted.
|
||
(bmalloc::Allocator::allocateMediumBumpRange): Deleted.
|
||
|
||
* bmalloc/Allocator.h:
|
||
* bmalloc/BumpRange.h:
|
||
|
||
* bmalloc/Cache.cpp:
|
||
(bmalloc::Cache::allocateSlowCase): Deleted.
|
||
(bmalloc::Cache::deallocateSlowCase): Deleted.
|
||
* bmalloc/Cache.h:
|
||
(bmalloc::Cache::allocate):
|
||
(bmalloc::Cache::deallocate):
|
||
(bmalloc::Cache::allocateFastCase): Deleted.
|
||
(bmalloc::Cache::deallocateFastCase): Deleted. Removed the Cache slow
|
||
paths. The downside to this change is that the fast path branches to two
|
||
distinct failure cases instead of one. The upside is that the slow path
|
||
doesn't need to re-read the segment register, which is not as cheap as a
|
||
normal register, and it doesn't need to do an extra level of function
|
||
call. Seems to be worth it.
|
||
|
||
* bmalloc/Deallocator.h:
|
||
* bmalloc/Heap.cpp:
|
||
(bmalloc::Heap::refillSmallBumpRangeCache):
|
||
(bmalloc::Heap::refillMediumBumpRangeCache):
|
||
* bmalloc/Heap.h: Updated for interface changes.
|
||
|
||
* bmalloc/Sizes.h: The most ranges a cache will hold is the number of
|
||
small lines in a page / 2, since any other free lines will coalesce
|
||
with their neighbors.
|
||
|
||
2014-09-23 Geoffrey Garen <ggaren@apple.com>
|
||
|
||
Rolled out r173346.
|
||
|
||
bmalloc should honor the FastMalloc statistics API
|
||
https://bugs.webkit.org/show_bug.cgi?id=136592
|
||
|
||
This didn't really work. Because we allow ranges with and without
|
||
physical pages to merge, and we allow double-committing and
|
||
double-decommitting, we can't rely on commit actions to track memory
|
||
footprint.
|
||
|
||
* bmalloc/Heap.cpp:
|
||
(bmalloc::Heap::size): Deleted.
|
||
(bmalloc::Heap::capacity): Deleted.
|
||
* bmalloc/Heap.h:
|
||
* bmalloc/VMHeap.cpp:
|
||
(bmalloc::VMHeap::VMHeap):
|
||
(bmalloc::VMHeap::allocateSmallChunk):
|
||
(bmalloc::VMHeap::allocateMediumChunk):
|
||
(bmalloc::VMHeap::allocateLargeChunk):
|
||
* bmalloc/VMHeap.h:
|
||
(bmalloc::VMHeap::allocateSmallPage):
|
||
(bmalloc::VMHeap::allocateMediumPage):
|
||
(bmalloc::VMHeap::allocateLargeRange):
|
||
(bmalloc::VMHeap::deallocateSmallPage):
|
||
(bmalloc::VMHeap::deallocateMediumPage):
|
||
(bmalloc::VMHeap::deallocateLargeRange):
|
||
(bmalloc::VMHeap::size): Deleted.
|
||
(bmalloc::VMHeap::capacity): Deleted.
|
||
* bmalloc/bmalloc.h:
|
||
(bmalloc::api::heapSize): Deleted.
|
||
(bmalloc::api::heapCapacity): Deleted.
|
||
|
||
2014-09-23 Geoffrey Garen <ggaren@apple.com>
|
||
|
||
bmalloc: Allocation should be more precise
|
||
https://bugs.webkit.org/show_bug.cgi?id=136993
|
||
|
||
Reviewed by Gavin Barraclough.
|
||
|
||
13% reduction in heap size on the MallocBench *_memory_warning benchmarks.
|
||
|
||
This patch teaches the allocator to merge adjacent free lines into a
|
||
single allocatable range. This allows us to shrink the size of an
|
||
individual line without increasing fragmentation or the rate of allocator
|
||
slow paths.
|
||
|
||
We'll only take more slow paths when available memory is sparse, which
|
||
is exactly when it's worth it. When available memory is dense, we'll
|
||
take fewer slow paths.
|
||
|
||
* bmalloc.xcodeproj/project.pbxproj:
|
||
* bmalloc/Algorithm.h:
|
||
(bmalloc::divideRoundingUp):
|
||
|
||
* bmalloc/Allocator.cpp:
|
||
(bmalloc::Allocator::Allocator): Updated for interface changes.
|
||
|
||
(bmalloc::Allocator::scavenge): Scavenge by object instead of by line.
|
||
Now that we merge lines, it's not convenient to scavenge by line.
|
||
|
||
(bmalloc::Allocator::allocateSmallBumpRange):
|
||
(bmalloc::Allocator::allocateMediumBumpRange): Allocate whole ranges
|
||
instead of individual lines.
|
||
|
||
(bmalloc::Allocator::allocateSlowCase):
|
||
(bmalloc::Allocator::allocateSmallLine): Deleted.
|
||
(bmalloc::Allocator::allocateMediumLine): Deleted.
|
||
(bmalloc::Allocator::allocateMedium): Deleted.
|
||
* bmalloc/Allocator.h:
|
||
(bmalloc::Allocator::allocateFastCase): Folded medium allocations
|
||
into the standard fast path with small allocations. Since a BumpAllocator
|
||
just allocates out of an arbitrary range, it doesn't need to distinguish
|
||
between small and medium lines.
|
||
|
||
* bmalloc/BumpAllocator.h:
|
||
(bmalloc::BumpAllocator::size):
|
||
(bmalloc::BumpAllocator::BumpAllocator):
|
||
(bmalloc::BumpAllocator::init):
|
||
(bmalloc::BumpAllocator::refill):
|
||
(bmalloc::BumpAllocator::line): Deleted. No need to track line information
|
||
anymore: the heap just gives us a pointer and a pre-computed number of
|
||
objects, and we allocate them.
|
||
|
||
* bmalloc/Deallocator.cpp:
|
||
(bmalloc::Deallocator::processObjectLog): Updated for interface changes.
|
||
|
||
* bmalloc/Heap.cpp:
|
||
(bmalloc::Heap::Heap):
|
||
(bmalloc::Heap::initializeLineMetadata): Pre-compute precise metadata
|
||
detailing where all objects will lie in memory. After we merge two lines,
|
||
we might allocate an object that spans from one line to the next. This
|
||
metadata details which bits of memory overlap in that way, and how they
|
||
overlap.
|
||
|
||
(bmalloc::Heap::refillSmallBumpRangeCache):
|
||
(bmalloc::Heap::refillMediumBumpRangeCache): Scan a whole page at a time,
|
||
and merge adjacent free lines into BumpRanges.
|
||
|
||
(bmalloc::Heap::allocateSmallPage):
|
||
(bmalloc::Heap::allocateMediumPage):
|
||
(bmalloc::Heap::deallocateSmallLine):
|
||
(bmalloc::Heap::deallocateMediumLine): Track pages rather than lines,
|
||
since we scan for free memory a page at a time.
|
||
|
||
(bmalloc::Heap::allocateSmallLineSlowCase): Deleted.
|
||
(bmalloc::Heap::allocateMediumLineSlowCase): Deleted. Folded into the
|
||
fast path.
|
||
|
||
* bmalloc/Heap.h:
|
||
(bmalloc::Heap::derefSmallLine):
|
||
(bmalloc::Heap::derefMediumLine):
|
||
(bmalloc::Heap::deallocateSmallLine): Deleted.
|
||
(bmalloc::Heap::allocateSmallLine): Deleted.
|
||
(bmalloc::Heap::deallocateMediumLine): Deleted.
|
||
(bmalloc::Heap::allocateMediumLine): Deleted. Updated for interface changes.
|
||
|
||
* bmalloc/Line.h:
|
||
(bmalloc::Line<Traits>::ref):
|
||
(bmalloc::Line<Traits>::deref):
|
||
(bmalloc::Line<Traits>::concurrentRef): Deleted. We don't pass a derefCount
|
||
anymore, since we only ever deref by 1 now.
|
||
|
||
* bmalloc/MediumAllocator.h:
|
||
(bmalloc::MediumAllocator::isNull): Deleted.
|
||
(bmalloc::MediumAllocator::MediumAllocator): Deleted.
|
||
(bmalloc::MediumAllocator::line): Deleted.
|
||
(bmalloc::MediumAllocator::allocate): Deleted.
|
||
(bmalloc::MediumAllocator::derefCount): Deleted.
|
||
(bmalloc::MediumAllocator::refill): Deleted.
|
||
(bmalloc::MediumAllocator::clear): Deleted. Deleted some code that's
|
||
been dead for a while, since it doesn't build anymore with this patch.
|
||
|
||
* bmalloc/Page.h:
|
||
(bmalloc::Page::sizeClass):
|
||
(bmalloc::Page::setSizeClass):
|
||
(bmalloc::Page::smallSizeClass): Deleted.
|
||
(bmalloc::Page::setSmallSizeClass): Deleted. Renamed setSmallSizeClass
|
||
to sizeClass, since we use it for medium sizes too.
|
||
|
||
* bmalloc/Sizes.h:
|
||
(bmalloc::Sizes::sizeClass):
|
||
(bmalloc::Sizes::objectSize): Shrank line sizes to save memory.
|
||
|
||
(bmalloc::Sizes::smallSizeClassFor): Deleted.
|
||
(bmalloc::Sizes::mediumSizeClassFor): Deleted.
|
||
|
||
* bmalloc/bmalloc.h:
|
||
(bmalloc::api::realloc): Now that we have precise objects sizes, realloc
|
||
can be a bit more precise. It also has to be, since we can't guarantee
|
||
that an object ends at the end of a line anymore.
|
||
|
||
2014-09-19 Daniel Bates <dabates@apple.com>
|
||
|
||
Always assume internal SDK when building configuration Production
|
||
https://bugs.webkit.org/show_bug.cgi?id=136925
|
||
<rdar://problem/18362399>
|
||
|
||
Reviewed by Dan Bernstein.
|
||
|
||
* Configurations/Base.xcconfig:
|
||
|
||
2014-09-16 Geoffrey Garen <ggaren@apple.com>
|
||
|
||
bmalloc: moved line caches from the deallocator to the allocator
|
||
https://bugs.webkit.org/show_bug.cgi?id=136868
|
||
|
||
Reviewed by Gavin Barraclough.
|
||
|
||
I did this mostly as a simplification, to make it easier to change the
|
||
allocation strategy.
|
||
|
||
No throughput change on MallocBench. Saves about 50kB.
|
||
|
||
Since the deallocator needs to lock the heap when freeing lines anyway,
|
||
there isn't much benefit to giving the deallocator a local cache of
|
||
deallocated lines.
|
||
|
||
We still give the allocator a local cache of lines because that does
|
||
reduce the frequency at which it needs to lock the heap in order to
|
||
acquire more lines.
|
||
|
||
* bmalloc/Allocator.cpp:
|
||
(bmalloc::Allocator::scavenge):
|
||
(bmalloc::Allocator::allocateSmallLine):
|
||
(bmalloc::Allocator::allocateMediumLine):
|
||
(bmalloc::Allocator::allocateMedium):
|
||
(bmalloc::Allocator::allocateSlowCase):
|
||
* bmalloc/Allocator.h:
|
||
* bmalloc/Deallocator.cpp:
|
||
(bmalloc::Deallocator::Deallocator):
|
||
(bmalloc::Deallocator::scavenge):
|
||
(bmalloc::Deallocator::processObjectLog):
|
||
(bmalloc::Deallocator::deallocateSmallLine): Deleted.
|
||
(bmalloc::Deallocator::allocateSmallLine): Deleted.
|
||
(bmalloc::Deallocator::deallocateMediumLine): Deleted.
|
||
(bmalloc::Deallocator::allocateMediumLine): Deleted.
|
||
* bmalloc/Deallocator.h:
|
||
|
||
* bmalloc/Sizes.h:
|
||
* bmalloc/VMAllocate.h: Took the opportunity to make the line cache size
|
||
exactly one page in size. That's about what we were shooting for anyway,
|
||
and it may make it easier to switch to per-page allocation in future.
|
||
|
||
2014-09-15 Geoffrey Garen <ggaren@apple.com>
|
||
|
||
bmalloc: allocate small and medium objects using the same bump pointer class
|
||
https://bugs.webkit.org/show_bug.cgi?id=136843
|
||
|
||
Reviewed by Gavin Barraclough.
|
||
|
||
4% speedup on MallocBench.
|
||
|
||
Now that medium-sized objects have dedicated per-size allocators, they
|
||
don't need to use an arbitrary bump pointer allocator. This means that
|
||
every allocator knows how many objects it will allocate from the start,
|
||
and we don't need a post-processing step to adjust refcounts based on
|
||
real allocation count.
|
||
|
||
* bmalloc.xcodeproj/project.pbxproj: Renamed SmallAllocator to BumpAllocator
|
||
since it's used for small and medium objects now.
|
||
|
||
* bmalloc/Allocator.cpp:
|
||
(bmalloc::Allocator::Allocator): Updated to use new interface.
|
||
(bmalloc::Allocator::scavenge): To "retire" an allocator, we just need
|
||
to make sure that we finish allocating all the objects in it.
|
||
|
||
(bmalloc::Allocator::allocateMedium):
|
||
(bmalloc::Allocator::allocateSlowCase):
|
||
(bmalloc::Allocator::retire): Deleted.
|
||
(bmalloc::Allocator::processSmallAllocatorLog): Deleted.
|
||
(bmalloc::Allocator::processMediumAllocatorLog): Deleted.
|
||
* bmalloc/Allocator.h:
|
||
(bmalloc::Allocator::allocateFastCase): Removed abstractions and data
|
||
used to post-process an allocator based on how many objects it allocated.
|
||
|
||
* bmalloc/BumpAllocator.h: Copied from Source/bmalloc/bmalloc/SmallAllocator.h.
|
||
(bmalloc::BumpAllocator::BumpAllocator):
|
||
(bmalloc::BumpAllocator::init):
|
||
(bmalloc::BumpAllocator::line):
|
||
(bmalloc::BumpAllocator::validate):
|
||
(bmalloc::BumpAllocator::allocate):
|
||
(bmalloc::BumpAllocator::refill):
|
||
(bmalloc::BumpAllocator::clear): Updated these functions to be agnostic
|
||
about the kinds of lines they allocate into. In some cases, the line
|
||
type must be provided as a template parameter by the caller.
|
||
|
||
(bmalloc::SmallAllocator::SmallAllocator): Deleted.
|
||
(bmalloc::SmallAllocator::line): Deleted.
|
||
(bmalloc::SmallAllocator::allocate): Deleted.
|
||
(bmalloc::SmallAllocator::objectCount): Deleted.
|
||
(bmalloc::SmallAllocator::derefCount): Deleted.
|
||
(bmalloc::SmallAllocator::refill): Deleted.
|
||
(bmalloc::SmallAllocator::clear): Deleted.
|
||
|
||
* bmalloc/ObjectType.h:
|
||
(bmalloc::isMedium):
|
||
|
||
* bmalloc/SmallAllocator.h:
|
||
(bmalloc::SmallAllocator::isNull): Deleted.
|
||
(bmalloc::SmallAllocator::canAllocate): Deleted.
|
||
(bmalloc::SmallAllocator::SmallAllocator): Deleted.
|
||
(bmalloc::SmallAllocator::line): Deleted.
|
||
(bmalloc::SmallAllocator::allocate): Deleted.
|
||
(bmalloc::SmallAllocator::objectCount): Deleted.
|
||
(bmalloc::SmallAllocator::derefCount): Deleted.
|
||
(bmalloc::SmallAllocator::refill): Deleted.
|
||
(bmalloc::SmallAllocator::clear): Deleted.
|
||
|
||
2014-09-12 Geoffrey Garen <ggaren@apple.com>
|
||
|
||
Fixed a goof in bmalloc Vector sizing
|
||
https://bugs.webkit.org/show_bug.cgi?id=136795
|
||
|
||
Reviewed by Gavin Barraclough and Sam Weinig.
|
||
|
||
We want our minimum vector to be page-sized since the OS will give us
|
||
a page no matter what -- but we want that many bytes, and not enough
|
||
bytes to store that many elements.
|
||
|
||
* bmalloc/Vector.h: Math is hard.
|
||
|
||
2014-09-11 Geoffrey Garen <ggaren@apple.com>
|
||
|
||
bmalloc should segregate medium-sized objects by line like it does for small-sized objects
|
||
https://bugs.webkit.org/show_bug.cgi?id=136693
|
||
|
||
Reviewed by Gavin Barraclough.
|
||
|
||
4% reduction in heap size on the MallocBench *_memory_warning benchmarks.
|
||
|
||
No throughput change.
|
||
|
||
We keep an array of medium allocators, just like our array of small
|
||
allocators.
|
||
|
||
In future, we can simplify the allocation fast path by merging the small
|
||
and medium allocator arrays. For now, this is the simplest change that
|
||
gets the win.
|
||
|
||
* bmalloc/Allocator.cpp:
|
||
(bmalloc::Allocator::Allocator):
|
||
(bmalloc::Allocator::scavenge):
|
||
(bmalloc::Allocator::allocateMedium):
|
||
* bmalloc/Allocator.h:
|
||
* bmalloc/Sizes.h:
|
||
(bmalloc::Sizes::mediumSizeClassFor):
|
||
|
||
2014-09-11 Geoffrey Garen <ggaren@apple.com>
|
||
|
||
Reviewed by Sam Weinig.
|
||
|
||
Renamed log => retire for clarity.
|
||
|
||
* bmalloc/Allocator.cpp:
|
||
(bmalloc::Allocator::scavenge):
|
||
(bmalloc::Allocator::retire):
|
||
(bmalloc::Allocator::allocateMedium):
|
||
(bmalloc::Allocator::allocateSlowCase):
|
||
(bmalloc::Allocator::log): Deleted.
|
||
* bmalloc/Allocator.h:
|
||
|
||
2014-09-11 Geoffrey Garen <ggaren@apple.com>
|
||
|
||
bmalloc: eager scavenge leaves behind a bogus allocator
|
||
https://bugs.webkit.org/show_bug.cgi?id=136743
|
||
|
||
Reviewed by Sam Weinig.
|
||
|
||
Be sure to clear the allocator after logging it in the eager scavenge
|
||
case, so that we don't later try to allocate out of the lines that we
|
||
have thrown away.
|
||
|
||
We didn't need to do this previously because scavenge would only happen
|
||
at thread exit time, after which no further allocation from the per-thread
|
||
cache would take place.
|
||
|
||
* bmalloc/Allocator.cpp:
|
||
(bmalloc::Allocator::scavenge):
|
||
* bmalloc/MediumAllocator.h:
|
||
(bmalloc::MediumAllocator::clear):
|
||
* bmalloc/SmallAllocator.h:
|
||
(bmalloc::SmallAllocator::clear):
|
||
|
||
2014-09-05 Geoffrey Garen <ggaren@apple.com>
|
||
|
||
bmalloc should honor the FastMalloc statistics API
|
||
https://bugs.webkit.org/show_bug.cgi?id=136592
|
||
|
||
Reviewed by Gavin Barraclough.
|
||
|
||
We do this by tracking "size" and "capacity" in the VM heap.
|
||
|
||
The VM heap's "capacity" is all the VM we ever allocated.
|
||
|
||
The VM heap's "size" the subset of VM currently held onto by the
|
||
VM heap (and therefore not in use by the regular heap).
|
||
|
||
Somewhat ironically, reducing the process's memory footprint, increases
|
||
the size of the VM heap, since the VM heap holds the pages that are
|
||
purely virtual and not physical.
|
||
|
||
* bmalloc/Heap.cpp:
|
||
(bmalloc::Heap::size):
|
||
(bmalloc::Heap::capacity):
|
||
* bmalloc/Heap.h:
|
||
* bmalloc/VMHeap.cpp:
|
||
(bmalloc::VMHeap::VMHeap):
|
||
(bmalloc::VMHeap::allocateSmallChunk):
|
||
(bmalloc::VMHeap::allocateMediumChunk):
|
||
(bmalloc::VMHeap::allocateLargeChunk):
|
||
* bmalloc/VMHeap.h:
|
||
(bmalloc::VMHeap::size):
|
||
(bmalloc::VMHeap::capacity):
|
||
(bmalloc::VMHeap::allocateSmallPage):
|
||
(bmalloc::VMHeap::allocateMediumPage):
|
||
(bmalloc::VMHeap::allocateLargeRange):
|
||
(bmalloc::VMHeap::deallocateSmallPage):
|
||
(bmalloc::VMHeap::deallocateMediumPage):
|
||
(bmalloc::VMHeap::deallocateLargeRange):
|
||
* bmalloc/bmalloc.h:
|
||
(bmalloc::api::heapSize):
|
||
(bmalloc::api::heapCapacity):
|
||
|
||
2014-09-02 Geoffrey Garen <ggaren@apple.com>
|
||
|
||
bmalloc crashes on the EWS bots (due to bad large object allocation)
|
||
https://bugs.webkit.org/show_bug.cgi?id=136469
|
||
|
||
Reviewed by Andreas Kling.
|
||
|
||
It's possible to convince bmalloc to perform a bad large object allocation,
|
||
through these steps:
|
||
|
||
(1) Insert object A into freelist F0.
|
||
|
||
(2) Split, merge and split again A's neighbors such that object B is
|
||
inserted into freelist F0, with boundary tag and size equal to object A,
|
||
but pointer not completely equal to object A. Put object B at the head of F0.
|
||
|
||
(3) Allocate some other object from F0, swapping its position in the
|
||
freelist with object B, such that object A is now ahead of object B.
|
||
|
||
--> Now, the next allocation for size A/B will allocate object A, which
|
||
has a slightly wrong idea about where the object actually begins.
|
||
Immediately, you'll corrupt a little memory, and over time, you'll also
|
||
corrupt boundary tag metadata.
|
||
|
||
The solution is to store the begin pointer in the boundary tag. Luckily,
|
||
this doesn't make the tag any bigger, and it's not a noticeable slowdown
|
||
on MallocBench.
|
||
|
||
* bmalloc/Algorithm.h:
|
||
(bmalloc::rightShift):
|
||
* bmalloc/BeginTag.h:
|
||
(bmalloc::BeginTag::isInFreeList): This is the bug fix. Make sure to
|
||
validate the start pointer when popping off the free list. Through a
|
||
very uncommon set of steps, it is possible to have an item in the free
|
||
list that is valid by all accounts except for its start pointer.
|
||
|
||
* bmalloc/BoundaryTag.h:
|
||
(bmalloc::BoundaryTag::compactBegin):
|
||
(bmalloc::BoundaryTag::setRange):
|
||
(bmalloc::BoundaryTag::setSize): Deleted. Record a compact version of the
|
||
start pointer. We don't need the whole pointer -- just the offset, in
|
||
largeAlignment increments, into the relevant boundary tag bucket.
|
||
|
||
* bmalloc/BoundaryTagInlines.h:
|
||
(bmalloc::validateNext):
|
||
(bmalloc::BoundaryTag::init):
|
||
(bmalloc::BoundaryTag::mergeLarge):
|
||
(bmalloc::BoundaryTag::splitLarge):
|
||
* bmalloc/SegregatedFreeList.cpp:
|
||
(bmalloc::SegregatedFreeList::insert):
|
||
(bmalloc::SegregatedFreeList::takeGreedy):
|
||
(bmalloc::SegregatedFreeList::take): Provide the whole range instead of
|
||
the size when establishing a boundary tag, as required by the new
|
||
interface.
|
||
|
||
* bmalloc/Sizes.h:
|
||
|
||
2014-08-14 Geoffrey Garen <ggaren@apple.com>
|
||
|
||
Fixed a bmalloc crash seen on the EWS bot
|
||
https://bugs.webkit.org/show_bug.cgi?id=135955
|
||
|
||
Reviewed by Andreas Kling.
|
||
|
||
* bmalloc/Syscall.h: Some CG APIs vm_copy their input buffers. If the
|
||
input buffer is a malloc region, that region will get marked Copy-On-Write
|
||
by the kernel. Calls to madvise() for COW regions fail and return EINVAL
|
||
on older OS X's. In 10.10, they still fail, but they do not return
|
||
EINVAL.
|
||
|
||
So, we can only ASSERT that our syscalls succeed starting with 10.10.
|
||
|
||
2014-08-14 Geoffrey Garen <ggaren@apple.com>
|
||
|
||
Fixed the bmalloc build
|
||
https://bugs.webkit.org/show_bug.cgi?id=135953
|
||
|
||
Reviewed by Andreas Kling.
|
||
|
||
* bmalloc.xcodeproj/project.pbxproj: Marked a few headers as private.
|
||
These headers are used, so they must be available outside the project.
|
||
|
||
2014-08-13 Daniel Bates <dabates@apple.com>
|
||
|
||
Attempt to fix the build following <http://trac.webkit.org/changeset/172576>
|
||
(https://bugs.webkit.org/show_bug.cgi?id=135895)
|
||
|
||
Substitute PerThreadStorage<T>::initSharedKeyIfNeeded() for initSharedKeyIfNeeded() in
|
||
implementation of PerThread<T>::getFastCase().
|
||
|
||
* bmalloc/PerThread.h:
|
||
(bmalloc::PerThread<T>::getFastCase):
|
||
|
||
2014-08-13 Daniel Bates <dabates@apple.com>
|
||
|
||
Make bmalloc::PerThread work without C++ thread local storage
|
||
https://bugs.webkit.org/show_bug.cgi?id=135895
|
||
|
||
Reviewed by Geoffrey Garen.
|
||
|
||
Implement support for building bmalloc without C++ thread local storage.
|
||
|
||
* bmalloc/BPlatform.h: Remove macro define BPLATFORM_IOS_SIMULATOR. Added macro function
|
||
BCOMPILER_SUPPORTS() and macro define BCOMPILER_SUPPORTS_CXX_THREAD_LOCAL that can be used
|
||
to determine whether the compiler supports C++ thread local storage.
|
||
* bmalloc/PerThread.h:
|
||
(bmalloc::PerThreadStorage::get): Modified to call pthread_getspecific() when building
|
||
without C++ thread local storage.
|
||
(bmalloc::PerThreadStorage::initSharedKeyIfNeeded): Added.
|
||
(bmalloc::PerThreadStorage::init): Moved logic to initialize shared Pthread key from here to
|
||
PerThreadStorage::initSharedKeyIfNeeded().
|
||
(bmalloc::PerThread<T>::getFastCase): Modified to call PerThreadStorage::initSharedKeyIfNeeded()
|
||
before querying PerThreadStorage::get() when building without C++ thread local storage so as to
|
||
ensure that the shared key has been initialized.
|
||
(_pthread_setspecific_direct): Deleted.
|
||
(_pthread_getspecific_direct): Deleted.
|
||
|
||
2014-08-13 Daniel Bates <dabates@apple.com>
|
||
|
||
[iOS] Make JavaScriptCore and bmalloc build with the public SDK
|
||
https://bugs.webkit.org/show_bug.cgi?id=135848
|
||
|
||
Reviewed by Geoffrey Garen.
|
||
|
||
* bmalloc/BPlatform.h: Added macro BPLATFORM_IOS_SIMULATOR, which evaluates to true
|
||
when building for the iOS Simulator.
|
||
* bmalloc/PerThread.h: Use pthread_machdep.h code path when building for iOS Simulator
|
||
using the public SDK.
|
||
(_pthread_setspecific_direct): Added; only defined when building for the iOS Simulator
|
||
using the public SDK.
|
||
(_pthread_getspecific_direct): Added; only defined when building for the iOS Simulator
|
||
using the public SDK.
|
||
|
||
2014-08-12 Daniel Bates <dabates@apple.com>
|
||
|
||
BPLATFORM(IOS) always evaluates to false
|
||
https://bugs.webkit.org/show_bug.cgi?id=135843
|
||
|
||
Reviewed by Geoffrey Garen.
|
||
|
||
Fix typo in definition of BPLATFORM() and include system header TargetConditionals.h
|
||
(when building on an Apple platform) so that BPLATFORM(X) evaluates to true when
|
||
building for platform X. In particular, so that BPLATFORM(IOS) evaluates to true when
|
||
building for iOS.
|
||
|
||
As a side effect of this change, the change made in <http://trac.webkit.org/changeset/167289>
|
||
will be honored and iOS will assume a VM page size of 16kB (again) instead of 4kB.
|
||
|
||
* bmalloc/BPlatform.h:
|
||
|
||
2014-08-11 Andy Estes <aestes@apple.com>
|
||
|
||
[iOS] Get rid of iOS.xcconfig
|
||
https://bugs.webkit.org/show_bug.cgi?id=135809
|
||
|
||
Reviewed by Joseph Pecoraro.
|
||
|
||
All iOS.xcconfig did was include AspenFamily.xcconfig, so there's no need for the indirection.
|
||
|
||
* Configurations/Base.xcconfig:
|
||
* Configurations/iOS.xcconfig: Removed.
|
||
* bmalloc.xcodeproj/project.pbxproj:
|
||
|
||
2014-05-01 Dan Bernstein <mitz@apple.com>
|
||
|
||
Fixed production builds for the iOS Simulator.
|
||
<rdar://problem/16792221>
|
||
|
||
* Configurations/bmalloc.xcconfig: Include INSTALL_PATH_PREFIX in
|
||
PRIVATE_HEADERS_FOLDER_PATH when installing.
|
||
|
||
2014-04-20 Geoffrey Garen <ggaren@apple.com>
|
||
|
||
bmalloc: Segregate pages by objects size
|
||
https://bugs.webkit.org/show_bug.cgi?id=131909
|
||
|
||
Reviewed by Andreas Kling.
|
||
|
||
2% reduction in memory-at-end on the Membuster memory_warning benchmarks.
|
||
|
||
* bmalloc/Allocator.cpp:
|
||
(bmalloc::Allocator::allocateSlowCase):
|
||
* bmalloc/Allocator.h:
|
||
(bmalloc::Allocator::allocateFastCase):
|
||
(bmalloc::Allocator::smallAllocatorFor): Use the new shared helper
|
||
function for size class calculation.
|
||
|
||
* bmalloc/Deallocator.cpp:
|
||
(bmalloc::Deallocator::Deallocator):
|
||
(bmalloc::Deallocator::scavenge):
|
||
(bmalloc::Deallocator::deallocateSmallLine):
|
||
(bmalloc::Deallocator::allocateSmallLine):
|
||
* bmalloc/Deallocator.h: Keep a cache for every size class, since the
|
||
cache can't be shared anymore.
|
||
|
||
* bmalloc/Heap.cpp:
|
||
(bmalloc::Heap::allocateSmallLineSlowCase):
|
||
* bmalloc/Heap.h:
|
||
(bmalloc::Heap::deallocateSmallLine): Ditto.
|
||
|
||
(bmalloc::Heap::allocateSmallLine): Check size class in addition to
|
||
page refcount when allocating a line because we might have deallocated
|
||
the page and the recycled it for another size class.
|
||
|
||
(bmalloc::Heap::deallocateMediumLine):
|
||
(bmalloc::Heap::allocateMediumLine):
|
||
* bmalloc/Line.h:
|
||
(bmalloc::Line::refCount):
|
||
* bmalloc/Page.h:
|
||
(bmalloc::Page::refCount):
|
||
(bmalloc::Page::smallSizeClass):
|
||
(bmalloc::Page::setSmallSizeClass):
|
||
(bmalloc::Page<Traits>::refCount): Deleted.
|
||
* bmalloc/Sizes.h:
|
||
(bmalloc::Sizes::smallSizeClassFor): New shared API for computing
|
||
an index into an array from a size.
|
||
|
||
2014-04-19 Geoffrey Garen <ggaren@apple.com>
|
||
|
||
bmalloc: Improved alignment in LargeChunk
|
||
https://bugs.webkit.org/show_bug.cgi?id=131895
|
||
|
||
Reviewed by Andreas Kling.
|
||
|
||
* bmalloc/Chunk.h:
|
||
* bmalloc/LargeChunk.h: Align to vmPageSize just like Chunk does.
|
||
Technically, the previous alignment was harmless, but I would prefer,
|
||
dear reader, not to have to explain the interlocking set of
|
||
circumstances that made it so.
|
||
|
||
2014-04-19 Geoffrey Garen <ggaren@apple.com>
|
||
|
||
Rolled out r167502 because it caused a crash on the facebook benchmark.
|
||
|
||
Unreviewed.
|
||
|
||
bmalloc: Added an XSmall line size
|
||
https://bugs.webkit.org/show_bug.cgi?id=131851
|
||
|
||
Reviewed by Sam Weinig.
|
||
|
||
2014-04-19 Geoffrey Garen <ggaren@apple.com>
|
||
|
||
bmalloc: Mutex should be harder to use wrong
|
||
https://bugs.webkit.org/show_bug.cgi?id=131879
|
||
|
||
Reviewed by Andreas Kling.
|
||
|
||
Mutex now has a proper constructor, so you can't deadlock by forgetting
|
||
to initialize it.
|
||
|
||
* bmalloc.xcodeproj/project.pbxproj:
|
||
* bmalloc/Allocator.cpp:
|
||
(bmalloc::Allocator::processXSmallAllocatorLog):
|
||
(bmalloc::Allocator::processSmallAllocatorLog):
|
||
(bmalloc::Allocator::processMediumAllocatorLog):
|
||
(bmalloc::Allocator::allocateLarge):
|
||
(bmalloc::Allocator::allocateXLarge): Global replace Mutex => StaticMutex,
|
||
since the Heap mutex is a static.
|
||
|
||
* bmalloc/AsyncTask.h:
|
||
(bmalloc::Function>::AsyncTask): Use Mutex, since we're not static. No
|
||
need for explicit initialization anymore.
|
||
|
||
* bmalloc/Deallocator.cpp:
|
||
(bmalloc::Deallocator::scavenge):
|
||
(bmalloc::Deallocator::deallocateLarge):
|
||
(bmalloc::Deallocator::deallocateXLarge):
|
||
(bmalloc::Deallocator::processObjectLog):
|
||
(bmalloc::Deallocator::deallocateSmallLine):
|
||
(bmalloc::Deallocator::deallocateXSmallLine):
|
||
(bmalloc::Deallocator::allocateSmallLine):
|
||
(bmalloc::Deallocator::allocateXSmallLine):
|
||
(bmalloc::Deallocator::deallocateMediumLine):
|
||
(bmalloc::Deallocator::allocateMediumLine):
|
||
* bmalloc/Deallocator.h:
|
||
* bmalloc/Heap.cpp:
|
||
(bmalloc::sleep):
|
||
(bmalloc::Heap::Heap):
|
||
(bmalloc::Heap::concurrentScavenge):
|
||
(bmalloc::Heap::scavenge):
|
||
(bmalloc::Heap::scavengeSmallPages):
|
||
(bmalloc::Heap::scavengeXSmallPages):
|
||
(bmalloc::Heap::scavengeMediumPages):
|
||
(bmalloc::Heap::scavengeLargeRanges):
|
||
(bmalloc::Heap::allocateXSmallLineSlowCase):
|
||
(bmalloc::Heap::allocateSmallLineSlowCase):
|
||
(bmalloc::Heap::allocateMediumLineSlowCase):
|
||
(bmalloc::Heap::allocateXLarge):
|
||
(bmalloc::Heap::deallocateXLarge):
|
||
(bmalloc::Heap::allocateLarge):
|
||
(bmalloc::Heap::deallocateLarge):
|
||
* bmalloc/Heap.h:
|
||
(bmalloc::Heap::deallocateXSmallLine):
|
||
(bmalloc::Heap::allocateXSmallLine):
|
||
(bmalloc::Heap::deallocateSmallLine):
|
||
(bmalloc::Heap::allocateSmallLine):
|
||
(bmalloc::Heap::deallocateMediumLine):
|
||
(bmalloc::Heap::allocateMediumLine):
|
||
* bmalloc/Line.h:
|
||
(bmalloc::Line<Traits>::deref):
|
||
* bmalloc/Mutex.cpp: Removed.
|
||
* bmalloc/Mutex.h:
|
||
(bmalloc::Mutex::Mutex):
|
||
(bmalloc::Mutex::init): Deleted.
|
||
(bmalloc::Mutex::try_lock): Deleted.
|
||
(bmalloc::Mutex::lock): Deleted.
|
||
(bmalloc::Mutex::unlock): Deleted.
|
||
* bmalloc/Page.h:
|
||
(bmalloc::Page<Traits>::ref):
|
||
(bmalloc::Page<Traits>::deref):
|
||
(bmalloc::Page<Traits>::refCount):
|
||
* bmalloc/PerProcess.h:
|
||
(bmalloc::PerProcess::mutex):
|
||
(bmalloc::PerProcess<T>::getSlowCase):
|
||
* bmalloc/StaticMutex.cpp: Added.
|
||
(bmalloc::StaticMutex::lockSlowCase):
|
||
* bmalloc/StaticMutex.h: Added.
|
||
(bmalloc::StaticMutex::init):
|
||
(bmalloc::StaticMutex::try_lock):
|
||
(bmalloc::StaticMutex::lock):
|
||
(bmalloc::StaticMutex::unlock):
|
||
* bmalloc/VMHeap.h:
|
||
(bmalloc::VMHeap::deallocateXSmallPage):
|
||
(bmalloc::VMHeap::deallocateSmallPage):
|
||
(bmalloc::VMHeap::deallocateMediumPage):
|
||
(bmalloc::VMHeap::deallocateLargeRange):
|
||
* bmalloc/bmalloc.h:
|
||
(bmalloc::api::scavenge): Global replace Mutex => StaticMutex,
|
||
since the Heap mutex is a static.
|
||
|
||
2014-04-18 Geoffrey Garen <ggaren@apple.com>
|
||
|
||
bmalloc: AsyncTask should use Mutex instead of std::mutex
|
||
https://bugs.webkit.org/show_bug.cgi?id=131865
|
||
|
||
Reviewed by Gavin Barraclough.
|
||
|
||
std::mutex is so slow that it makes parallelizing simple tasks through
|
||
AsyncTask a net regression. Mutex fixes this.
|
||
|
||
* bmalloc/AsyncTask.h:
|
||
(bmalloc::Function>::AsyncTask):
|
||
(bmalloc::Function>::join):
|
||
(bmalloc::Function>::runSlowCase):
|
||
(bmalloc::Function>::entryPoint):
|
||
* bmalloc/Mutex.h:
|
||
(bmalloc::Mutex::init):
|
||
|
||
2014-04-18 Geoffrey Garen <ggaren@apple.com>
|
||
|
||
bmalloc: Added an XSmall line size
|
||
https://bugs.webkit.org/show_bug.cgi?id=131851
|
||
|
||
Reviewed by Sam Weinig.
|
||
|
||
Reduces malloc footprint on Membuster recordings by 10%.
|
||
|
||
This is a throughput regression, but we're still way ahead of TCMalloc.
|
||
I have some ideas for how to recover the regression -- but I wanted to
|
||
get this win in first.
|
||
|
||
Full set of benchmark results:
|
||
|
||
bmalloc> ~/webkit/PerformanceTests/MallocBench/run-malloc-benchmarks --measure-heap nopatch:~/scratch/Build-nopatch/Release/ patch:~/webkit/WebKitBuild/Release/
|
||
|
||
nopatch patch Δ
|
||
Peak Memory:
|
||
reddit_memory_warning 7,896kB 7,532kB ^ 1.05x smaller
|
||
flickr_memory_warning 12,968kB 12,324kB ^ 1.05x smaller
|
||
theverge_memory_warning 16,672kB 15,200kB ^ 1.1x smaller
|
||
|
||
<geometric mean> 11,952kB 11,216kB ^ 1.07x smaller
|
||
<arithmetic mean> 12,512kB 11,685kB ^ 1.07x smaller
|
||
<harmonic mean> 11,375kB 10,726kB ^ 1.06x smaller
|
||
|
||
Memory at End:
|
||
reddit_memory_warning 7,320kB 6,856kB ^ 1.07x smaller
|
||
flickr_memory_warning 10,848kB 9,692kB ^ 1.12x smaller
|
||
theverge_memory_warning 16,380kB 14,872kB ^ 1.1x smaller
|
||
|
||
<geometric mean> 10,916kB 9,961kB ^ 1.1x smaller
|
||
<arithmetic mean> 11,516kB 10,473kB ^ 1.1x smaller
|
||
<harmonic mean> 10,350kB 9,485kB ^ 1.09x smaller
|
||
|
||
MallocBench> ~/webkit/PerformanceTests/MallocBench/run-malloc-benchmarks nopatch:~/scratch/Build-nopatch/Release/ patch:~/webkit/WebKitBuild/Release/
|
||
|
||
nopatch patch Δ
|
||
Execution Time:
|
||
churn 127ms 151ms ! 1.19x slower
|
||
list_allocate 130ms 164ms ! 1.26x slower
|
||
tree_allocate 109ms 127ms ! 1.17x slower
|
||
tree_churn 115ms 120ms ! 1.04x slower
|
||
facebook 240ms 259ms ! 1.08x slower
|
||
fragment 91ms 131ms ! 1.44x slower
|
||
fragment_iterate 105ms 106ms ! 1.01x slower
|
||
message_one 260ms 259ms ^ 1.0x faster
|
||
message_many 149ms 154ms ! 1.03x slower
|
||
medium 194ms 248ms ! 1.28x slower
|
||
big 157ms 160ms ! 1.02x slower
|
||
|
||
<geometric mean> 144ms 163ms ! 1.13x slower
|
||
<arithmetic mean> 152ms 171ms ! 1.12x slower
|
||
<harmonic mean> 137ms 156ms ! 1.14x slower
|
||
|
||
MallocBench> ~/webkit/PerformanceTests/MallocBench/run-malloc-benchmarks nopatch:~/scratch/Build-nopatch/Release/ patch:~/webkit/WebKitBuild/Release/
|
||
|
||
nopatch patch Δ
|
||
Execution Time:
|
||
churn 126ms 148ms ! 1.17x slower
|
||
churn --parallel 62ms 76ms ! 1.23x slower
|
||
list_allocate 130ms 164ms ! 1.26x slower
|
||
list_allocate --parallel 120ms 175ms ! 1.46x slower
|
||
tree_allocate 111ms 127ms ! 1.14x slower
|
||
tree_allocate --parallel 95ms 135ms ! 1.42x slower
|
||
tree_churn 115ms 124ms ! 1.08x slower
|
||
tree_churn --parallel 107ms 126ms ! 1.18x slower
|
||
facebook 240ms 276ms ! 1.15x slower
|
||
facebook --parallel 802ms 1,088ms ! 1.36x slower
|
||
fragment 92ms 130ms ! 1.41x slower
|
||
fragment --parallel 66ms 124ms ! 1.88x slower
|
||
fragment_iterate 109ms 127ms ! 1.17x slower
|
||
fragment_iterate --parallel 55ms 64ms ! 1.16x slower
|
||
message_one 260ms 260ms
|
||
message_many 170ms 238ms ! 1.4x slower
|
||
medium 185ms 250ms ! 1.35x slower
|
||
medium --parallel 210ms 334ms ! 1.59x slower
|
||
big 150ms 169ms ! 1.13x slower
|
||
big --parallel 138ms 144ms ! 1.04x slower
|
||
|
||
<geometric mean> 135ms 170ms ! 1.26x slower
|
||
<arithmetic mean> 167ms 214ms ! 1.28x slower
|
||
<harmonic mean> 117ms 148ms ! 1.26x slower
|
||
|
||
MallocBench> ~/webkit/PerformanceTests/MallocBench/run-malloc-benchmarks TC:~/scratch/Build-TCMalloc/Release/ patch:~/webkit/WebKitBuild/Release/
|
||
|
||
TC patch Δ
|
||
Peak Memory:
|
||
reddit_memory_warning 13,836kB 13,436kB ^ 1.03x smaller
|
||
flickr_memory_warning 24,868kB 25,188kB ! 1.01x bigger
|
||
theverge_memory_warning 24,504kB 26,636kB ! 1.09x bigger
|
||
|
||
<geometric mean> 20,353kB 20,812kB ! 1.02x bigger
|
||
<arithmetic mean> 21,069kB 21,753kB ! 1.03x bigger
|
||
<harmonic mean> 19,570kB 19,780kB ! 1.01x bigger
|
||
|
||
Memory at End:
|
||
reddit_memory_warning 8,656kB 10,016kB ! 1.16x bigger
|
||
flickr_memory_warning 11,844kB 13,784kB ! 1.16x bigger
|
||
theverge_memory_warning 18,516kB 22,748kB ! 1.23x bigger
|
||
|
||
<geometric mean> 12,382kB 14,644kB ! 1.18x bigger
|
||
<arithmetic mean> 13,005kB 15,516kB ! 1.19x bigger
|
||
<harmonic mean> 11,813kB 13,867kB ! 1.17x bigger
|
||
|
||
MallocBench> ~/webkit/PerformanceTests/MallocBench/run-malloc-benchmarks TC:~/scratch/Build-TCMalloc/Release/ patch:~/webkit/WebKitBuild/Release/
|
||
|
||
TC patch Δ
|
||
Execution Time:
|
||
churn 416ms 148ms ^ 2.81x faster
|
||
list_allocate 463ms 164ms ^ 2.82x faster
|
||
tree_allocate 292ms 127ms ^ 2.3x faster
|
||
tree_churn 157ms 120ms ^ 1.31x faster
|
||
facebook 327ms 276ms ^ 1.18x faster
|
||
fragment 335ms 129ms ^ 2.6x faster
|
||
fragment_iterate 344ms 108ms ^ 3.19x faster
|
||
message_one 386ms 258ms ^ 1.5x faster
|
||
message_many 410ms 154ms ^ 2.66x faster
|
||
medium 391ms 245ms ^ 1.6x faster
|
||
big 261ms 167ms ^ 1.56x faster
|
||
|
||
<geometric mean> 332ms 164ms ^ 2.02x faster
|
||
<arithmetic mean> 344ms 172ms ^ 1.99x faster
|
||
<harmonic mean> 317ms 157ms ^ 2.02x faster
|
||
|
||
* bmalloc.xcodeproj/project.pbxproj:
|
||
* bmalloc/Allocator.cpp:
|
||
(bmalloc::Allocator::Allocator): Don't assume that each allocator's
|
||
index corresponds with its size. Instead, use the size selection function
|
||
explicitly. Now that we have XSmall, some small allocator entries are
|
||
unused.
|
||
|
||
(bmalloc::Allocator::scavenge):
|
||
(bmalloc::Allocator::log):
|
||
(bmalloc::Allocator::processXSmallAllocatorLog):
|
||
(bmalloc::Allocator::allocateSlowCase):
|
||
* bmalloc/Allocator.h:
|
||
(bmalloc::Allocator::xSmallAllocatorFor):
|
||
(bmalloc::Allocator::allocateFastCase):
|
||
* bmalloc/Chunk.h:
|
||
* bmalloc/Deallocator.cpp:
|
||
(bmalloc::Deallocator::scavenge):
|
||
(bmalloc::Deallocator::processObjectLog):
|
||
(bmalloc::Deallocator::deallocateSlowCase):
|
||
(bmalloc::Deallocator::deallocateXSmallLine):
|
||
(bmalloc::Deallocator::allocateXSmallLine):
|
||
* bmalloc/Deallocator.h:
|
||
(bmalloc::Deallocator::deallocateFastCase):
|
||
* bmalloc/Heap.cpp:
|
||
(bmalloc::Heap::scavenge):
|
||
(bmalloc::Heap::scavengeXSmallPages):
|
||
(bmalloc::Heap::allocateXSmallLineSlowCase):
|
||
* bmalloc/Heap.h:
|
||
(bmalloc::Heap::deallocateXSmallLine):
|
||
(bmalloc::Heap::allocateXSmallLine):
|
||
* bmalloc/LargeChunk.h:
|
||
(bmalloc::LargeChunk::get):
|
||
(bmalloc::LargeChunk::endTag):
|
||
* bmalloc/Line.h:
|
||
* bmalloc/MediumAllocator.h:
|
||
(bmalloc::MediumAllocator::allocate):
|
||
(bmalloc::MediumAllocator::refill):
|
||
* bmalloc/ObjectType.cpp:
|
||
(bmalloc::objectType):
|
||
* bmalloc/ObjectType.h:
|
||
(bmalloc::isXSmall):
|
||
(bmalloc::isSmall):
|
||
(bmalloc::isMedium):
|
||
(bmalloc::isLarge):
|
||
(bmalloc::isSmallOrMedium): Deleted.
|
||
* bmalloc/SegregatedFreeList.h: I boiler-plate copied existing code for
|
||
handling small objects. There's probably a reasonable way to share this
|
||
code in the future -- I'll look into that once it's stopped changing.
|
||
|
||
* bmalloc/Sizes.h: Tweaked size classes to make Membuster happy. This
|
||
is the main reason things got slower.
|
||
|
||
* bmalloc/SmallAllocator.h:
|
||
(bmalloc::SmallAllocator::allocate):
|
||
* bmalloc/SmallTraits.h:
|
||
* bmalloc/VMHeap.cpp:
|
||
(bmalloc::VMHeap::allocateXSmallChunk):
|
||
* bmalloc/VMHeap.h:
|
||
(bmalloc::VMHeap::allocateXSmallPage):
|
||
(bmalloc::VMHeap::deallocateXSmallPage):
|
||
* bmalloc/XSmallAllocator.h: Added.
|
||
(bmalloc::XSmallAllocator::isNull):
|
||
(bmalloc::XSmallAllocator::canAllocate):
|
||
(bmalloc::XSmallAllocator::XSmallAllocator):
|
||
(bmalloc::XSmallAllocator::line):
|
||
(bmalloc::XSmallAllocator::allocate):
|
||
(bmalloc::XSmallAllocator::objectCount):
|
||
(bmalloc::XSmallAllocator::derefCount):
|
||
(bmalloc::XSmallAllocator::refill):
|
||
* bmalloc/XSmallChunk.h: Added.
|
||
* bmalloc/XSmallLine.h: Added.
|
||
* bmalloc/XSmallPage.h: Added.
|
||
* bmalloc/XSmallTraits.h: Added.
|
||
* bmalloc/bmalloc.h:
|
||
(bmalloc::api::realloc): Boiler-plate copy, as above.
|
||
|
||
2014-04-14 Geoffrey Garen <ggaren@apple.com>
|
||
|
||
MallocBench should scavenge explicitly instead of waiting
|
||
https://bugs.webkit.org/show_bug.cgi?id=131661
|
||
|
||
Reviewed by Andreas Kling.
|
||
|
||
Added explicit scavenge support to bmalloc. This isn't a memory win,
|
||
since bmalloc's per-thread cache is so small. But it makes testing
|
||
simpler.
|
||
|
||
* bmalloc/Allocator.cpp:
|
||
(bmalloc::Allocator::~Allocator):
|
||
(bmalloc::Allocator::scavenge):
|
||
* bmalloc/Allocator.h:
|
||
* bmalloc/Cache.cpp:
|
||
(bmalloc::Cache::operator new):
|
||
(bmalloc::Cache::operator delete):
|
||
(bmalloc::Cache::Cache):
|
||
(bmalloc::Cache::scavenge):
|
||
* bmalloc/Cache.h:
|
||
* bmalloc/Deallocator.cpp:
|
||
(bmalloc::Deallocator::~Deallocator):
|
||
(bmalloc::Deallocator::scavenge):
|
||
* bmalloc/Deallocator.h: Factored existing scavenging code into helper
|
||
functions, for reuse.
|
||
|
||
* bmalloc/Heap.cpp:
|
||
(bmalloc::sleep):
|
||
(bmalloc::Heap::concurrentScavenge):
|
||
(bmalloc::Heap::scavenge):
|
||
(bmalloc::Heap::scavengeSmallPages):
|
||
(bmalloc::Heap::scavengeMediumPages):
|
||
(bmalloc::Heap::scavengeLargeRanges):
|
||
* bmalloc/Heap.h: Made scavenge sleep duration a parameter. Forced
|
||
scavenging -- in response to a benchmark or a low memory warning --
|
||
wants to complete as soon as possible, so its sleep duration is 0.
|
||
|
||
* bmalloc/bmalloc.h:
|
||
(bmalloc::api::scavenge):
|
||
* bmalloc/mbmalloc.cpp: Exported the scavenge API for MallocBench's use.
|
||
|
||
2014-04-14 Geoffrey Garen <ggaren@apple.com>
|
||
|
||
Use 4kB pages on Mac
|
||
https://bugs.webkit.org/show_bug.cgi?id=131658
|
||
|
||
Reviewed by Sam Weinig.
|
||
|
||
This reduces memory use a lot on Membuster:
|
||
|
||
base patch Δ
|
||
Execution Time:
|
||
reddit_memory_warning 18ms 17ms ^ 1.06x faster
|
||
flickr_memory_warning 34ms 36ms ! 1.06x slower
|
||
theverge_memory_warning 39ms 41ms ! 1.05x slower
|
||
|
||
<geometric mean> 29ms 29ms ! 1.02x slower
|
||
<arithmetic mean> 30ms 31ms ! 1.03x slower
|
||
<harmonic mean> 27ms 27ms ^ 1.0x faster
|
||
|
||
Peak Memory:
|
||
reddit_memory_warning 16,412kB 16,436kB ! 1.0x bigger
|
||
flickr_memory_warning 30,120kB 30,184kB ! 1.0x bigger
|
||
theverge_memory_warning 33,408kB 33,420kB ! 1.0x bigger
|
||
|
||
<geometric mean> 25,466kB 25,499kB ! 1.0x bigger
|
||
<arithmetic mean> 26,647kB 26,680kB ! 1.0x bigger
|
||
<harmonic mean> 24,181kB 24,214kB ! 1.0x bigger
|
||
|
||
Memory at End:
|
||
reddit_memory_warning 2,404kB 1,920kB ^ 1.25x smaller
|
||
flickr_memory_warning 3,764kB 3,072kB ^ 1.23x smaller
|
||
theverge_memory_warning 3,648kB 3,132kB ^ 1.16x smaller
|
||
|
||
<geometric mean> 3,208kB 2,644kB ^ 1.21x smaller
|
||
<arithmetic mean> 3,272kB 2,708kB ^ 1.21x smaller
|
||
<harmonic mean> 3,139kB 2,574kB ^ 1.22x smaller
|
||
|
||
|
||
* bmalloc.xcodeproj/project.pbxproj:
|
||
* bmalloc/BPlatform.h: Added.
|
||
* bmalloc/VMAllocate.h: Only use 16kB pages on iOS because the page size
|
||
is 4kB on Mac.
|
||
|
||
2014-04-14 Alexey Proskuryakov <ap@apple.com>
|
||
|
||
Fixed svn:ignore on bmalloc.xcodeproj, it had erroneous leading spaces.
|
||
|
||
* bmalloc.xcodeproj: Modified property svn:ignore.
|
||
|
||
2014-04-13 Geoffrey Garen <ggaren@apple.com>
|
||
|
||
Fixed some mbmalloc exports
|
||
https://bugs.webkit.org/show_bug.cgi?id=131599
|
||
|
||
Reviewed by Ryosuke Niwa.
|
||
|
||
* bmalloc.xcodeproj/project.pbxproj: Made some headers a private part
|
||
of the project, so we can call them from API.
|
||
|
||
* bmalloc/mbmalloc.cpp: Marked the mbmalloc functions with default
|
||
visibility, so they show up as exported in the .dylib.
|
||
|
||
2014-04-09 Geoffrey Garen <ggaren@apple.com>
|
||
|
||
Put bmalloc headers in the right place
|
||
https://bugs.webkit.org/show_bug.cgi?id=131464
|
||
|
||
Reviewed by Mark Rowe.
|
||
|
||
* Configurations/bmalloc.xcconfig: Set PRIVATE_HEADERS_FOLDER_PATH to
|
||
specify that we don't just want to dump all of our generically-named
|
||
headers into /usr/local/include.
|
||
|
||
2014-04-08 Geoffrey Garen <ggaren@apple.com>
|
||
|
||
Made bmalloc more #include friendly
|
||
https://bugs.webkit.org/show_bug.cgi?id=131386
|
||
|
||
Reviewed by Andreas Kling.
|
||
|
||
Marked a bunch of headers private so they can be used from client code
|
||
that #includes bmalloc.h.
|
||
|
||
Renamed ASSERT macros to BASSERT. This matches their header, which already
|
||
had to be renamed, and fixes conflicts with WTF's ASSERT macros.
|
||
|
||
* bmalloc.xcodeproj/project.pbxproj:
|
||
* bmalloc/Allocator.cpp:
|
||
(bmalloc::Allocator::allocateSlowCase):
|
||
* bmalloc/AsyncTask.h:
|
||
(bmalloc::Function>::runSlowCase):
|
||
* bmalloc/BAssert.h:
|
||
* bmalloc/BoundaryTag.h:
|
||
(bmalloc::BoundaryTag::setSize):
|
||
* bmalloc/BoundaryTagInlines.h:
|
||
(bmalloc::validate):
|
||
(bmalloc::BoundaryTag::init):
|
||
(bmalloc::BoundaryTag::deallocate):
|
||
(bmalloc::BoundaryTag::splitLarge):
|
||
(bmalloc::BoundaryTag::allocate):
|
||
* bmalloc/Chunk.h:
|
||
* bmalloc/Deallocator.cpp:
|
||
(bmalloc::Deallocator::processObjectLog):
|
||
(bmalloc::Deallocator::deallocateSlowCase):
|
||
* bmalloc/Deallocator.h:
|
||
(bmalloc::Deallocator::deallocateFastCase):
|
||
* bmalloc/FixedVector.h:
|
||
(bmalloc::Capacity>::operator):
|
||
(bmalloc::Capacity>::push):
|
||
(bmalloc::Capacity>::pop):
|
||
(bmalloc::Capacity>::shrink):
|
||
* bmalloc/Heap.cpp:
|
||
(bmalloc::Heap::allocateLarge):
|
||
* bmalloc/LargeChunk.h:
|
||
(bmalloc::LargeChunk::get):
|
||
(bmalloc::LargeChunk::endTag):
|
||
* bmalloc/Line.h:
|
||
(bmalloc::Line<Traits>::concurrentRef):
|
||
(bmalloc::Line<Traits>::deref):
|
||
* bmalloc/MediumAllocator.h:
|
||
(bmalloc::MediumAllocator::allocate):
|
||
* bmalloc/ObjectType.h:
|
||
(bmalloc::isSmall):
|
||
* bmalloc/Page.h:
|
||
(bmalloc::Page<Traits>::ref):
|
||
(bmalloc::Page<Traits>::deref):
|
||
* bmalloc/PerThread.h:
|
||
(bmalloc::PerThread<T>::getSlowCase):
|
||
* bmalloc/SegregatedFreeList.cpp:
|
||
(bmalloc::SegregatedFreeList::SegregatedFreeList):
|
||
(bmalloc::SegregatedFreeList::insert):
|
||
* bmalloc/SmallAllocator.h:
|
||
(bmalloc::SmallAllocator::allocate):
|
||
(bmalloc::SmallAllocator::refill):
|
||
* bmalloc/Syscall.h:
|
||
* bmalloc/VMAllocate.h:
|
||
(bmalloc::vmValidate):
|
||
(bmalloc::vmAllocate):
|
||
(bmalloc::vmDeallocatePhysicalPagesSloppy):
|
||
* bmalloc/Vector.h:
|
||
(bmalloc::Vector<T>::operator):
|
||
(bmalloc::Vector<T>::pop):
|
||
(bmalloc::Vector<T>::shrink):
|
||
* bmalloc/XLargeChunk.h:
|
||
(bmalloc::XLargeChunk::range):
|
||
(bmalloc::XLargeChunk::size):
|
||
|
||
2014-04-08 Geoffrey Garen <ggaren@apple.com>
|
||
|
||
Removed an unused file.
|
||
|
||
Unreviewed.
|
||
|
||
* bmalloc/AsyncTask.cpp: Removed.
|
||
|
||
2014-04-07 Geoffrey Garen <ggaren@apple.com>
|
||
|
||
Build bmalloc on Mac
|
||
https://bugs.webkit.org/show_bug.cgi?id=131333
|
||
|
||
Reviewed by Mark Rowe.
|
||
|
||
* Makefile: Added. For make clients.
|
||
|
||
These files are required for building any project in WebKit. I copied
|
||
them from WTF:
|
||
* Configurations: Added.
|
||
* Configurations/Base.xcconfig: Added.
|
||
* Configurations/DebugRelease.xcconfig: Added.
|
||
* Configurations/bmalloc.xcconfig: Added.
|
||
* Configurations/iOS.xcconfig: Added.
|
||
* Configurations/mbmalloc.xcconfig: Added.
|
||
|
||
* bmalloc.xcodeproj/project.pbxproj: I removed per-project-file stuff
|
||
from here because everything is in .xcconfig files now.
|
||
|
||
I had to fix a bunch of minor warnings, since they're enabled in our
|
||
.xcconfig files:
|
||
|
||
* bmalloc/AsyncTask.h:
|
||
(bmalloc::Function>::AsyncTask):
|
||
* bmalloc/BAssert.h:
|
||
* bmalloc/BoundaryTagInlines.h:
|
||
(bmalloc::validate):
|
||
* bmalloc/Heap.cpp:
|
||
(bmalloc::Heap::Heap):
|
||
(bmalloc::Heap::allocateLarge):
|
||
(bmalloc::Heap::deallocateLarge):
|
||
* bmalloc/Mutex.h:
|
||
(bmalloc::Mutex::Mutex): Deleted.
|
||
* bmalloc/VMAllocate.h:
|
||
(bmalloc::vmValidate):
|
||
* bmalloc/mbmalloc.cpp:
|
||
|
||
2014-04-07 Geoffrey Garen <ggaren@apple.com>
|
||
|
||
bmalloc: Fixed a leak in the per-thread cache
|
||
https://bugs.webkit.org/show_bug.cgi?id=131330
|
||
|
||
Reviewed by Andreas Kling.
|
||
|
||
Remember to deallocate our line caches upon thread exit.
|
||
|
||
* bmalloc/Deallocator.cpp:
|
||
(bmalloc::Deallocator::~Deallocator):
|
||
|
||
2014-04-07 Geoffrey Garen <ggaren@apple.com>
|
||
|
||
bmalloc: rolled out the tryLock experiment
|
||
https://bugs.webkit.org/show_bug.cgi?id=131328
|
||
|
||
Reviewed by Andreas Kling.
|
||
|
||
It wasn't a speedup.
|
||
|
||
* bmalloc.xcodeproj/project.pbxproj:
|
||
* bmalloc/Allocator.cpp:
|
||
(bmalloc::Allocator::processSmallAllocatorLog):
|
||
(bmalloc::Allocator::processMediumAllocatorLog):
|
||
* bmalloc/Deallocator.cpp:
|
||
(bmalloc::Deallocator::processObjectLog):
|
||
(bmalloc::Deallocator::deallocateSlowCase):
|
||
(bmalloc::Deallocator::deallocateSmallLine):
|
||
(bmalloc::Deallocator::deallocateMediumLine):
|
||
* bmalloc/Deallocator.h:
|
||
(bmalloc::Deallocator::deallocateFastCase):
|
||
* bmalloc/Heap.h:
|
||
(bmalloc::Heap::deallocateSmallLine):
|
||
(bmalloc::Heap::deallocateMediumLine):
|
||
* bmalloc/Line.h:
|
||
(bmalloc::Line<Traits>::deref):
|
||
* bmalloc/Page.h:
|
||
(bmalloc::Page<Traits>::deref):
|
||
|
||
2014-04-07 Geoffrey Garen <ggaren@apple.com>
|
||
|
||
bmalloc
|
||
https://bugs.webkit.org/show_bug.cgi?id=131170
|
||
|
||
Reviewed by Andreas Kling.
|
||
|
||
Initial commit.
|
||
|
||
* bmalloc: Added.
|
||
* bmalloc.xcodeproj: Added.
|
||
* bmalloc.xcodeproj/project.pbxproj: Added.
|
||
* bmalloc/Algorithm.h: Added.
|
||
(bmalloc::max):
|
||
(bmalloc::min):
|
||
(bmalloc::mask):
|
||
(bmalloc::test):
|
||
(bmalloc::roundUpToMultipleOf):
|
||
(bmalloc::roundDownToMultipleOf):
|
||
(bmalloc::sizeOf):
|
||
(bmalloc::bitCount):
|
||
(bmalloc::isPowerOfTwo):
|
||
* bmalloc/Allocator.cpp: Added.
|
||
(bmalloc::Allocator::Allocator):
|
||
(bmalloc::Allocator::~Allocator):
|
||
(bmalloc::Allocator::log):
|
||
(bmalloc::Allocator::processSmallAllocatorLog):
|
||
(bmalloc::Allocator::processMediumAllocatorLog):
|
||
(bmalloc::Allocator::allocateLarge):
|
||
(bmalloc::Allocator::allocateXLarge):
|
||
(bmalloc::Allocator::allocateMedium):
|
||
(bmalloc::Allocator::allocateSlowCase):
|
||
* bmalloc/Allocator.h: Added.
|
||
(bmalloc::Allocator::smallAllocatorFor):
|
||
(bmalloc::Allocator::allocateFastCase):
|
||
(bmalloc::Allocator::allocate):
|
||
* bmalloc/AsyncTask.cpp: Added.
|
||
(bmalloc::AsyncTask<Function>::runSlowCase):
|
||
(bmalloc::AsyncTask<Function>::pthreadEntryPoint):
|
||
(bmalloc::AsyncTask<Function>::entryPoint):
|
||
* bmalloc/AsyncTask.h: Added.
|
||
(bmalloc::Function>::AsyncTask):
|
||
(bmalloc::Function>::join):
|
||
(bmalloc::Function>::run):
|
||
(bmalloc::Function>::runSlowCase):
|
||
(bmalloc::Function>::pthreadEntryPoint):
|
||
(bmalloc::Function>::entryPoint):
|
||
* bmalloc/BAssert.h: Added.
|
||
* bmalloc/BeginTag.h: Added.
|
||
(bmalloc::BeginTag::isInFreeList):
|
||
* bmalloc/BoundaryTag.h: Added.
|
||
(bmalloc::BoundaryTag::isXLarge):
|
||
(bmalloc::BoundaryTag::setXLarge):
|
||
(bmalloc::BoundaryTag::isFree):
|
||
(bmalloc::BoundaryTag::setFree):
|
||
(bmalloc::BoundaryTag::isEnd):
|
||
(bmalloc::BoundaryTag::setEnd):
|
||
(bmalloc::BoundaryTag::hasPhysicalPages):
|
||
(bmalloc::BoundaryTag::setHasPhysicalPages):
|
||
(bmalloc::BoundaryTag::isNull):
|
||
(bmalloc::BoundaryTag::clear):
|
||
(bmalloc::BoundaryTag::size):
|
||
(bmalloc::BoundaryTag::setSize):
|
||
(bmalloc::BoundaryTag::prev):
|
||
(bmalloc::BoundaryTag::next):
|
||
* bmalloc/BoundaryTagInlines.h: Added.
|
||
(bmalloc::validate):
|
||
(bmalloc::validatePrev):
|
||
(bmalloc::validateNext):
|
||
(bmalloc::BoundaryTag::init):
|
||
(bmalloc::BoundaryTag::mergeLargeLeft):
|
||
(bmalloc::BoundaryTag::mergeLargeRight):
|
||
(bmalloc::BoundaryTag::mergeLarge):
|
||
(bmalloc::BoundaryTag::deallocate):
|
||
(bmalloc::BoundaryTag::splitLarge):
|
||
(bmalloc::BoundaryTag::allocate):
|
||
* bmalloc/Cache.cpp: Added.
|
||
(bmalloc::Cache::operator new):
|
||
(bmalloc::Cache::operator delete):
|
||
(bmalloc::Cache::Cache):
|
||
(bmalloc::Cache::allocateSlowCase):
|
||
(bmalloc::Cache::allocateSlowCaseNullCache):
|
||
(bmalloc::Cache::deallocateSlowCase):
|
||
(bmalloc::Cache::deallocateSlowCaseNullCache):
|
||
* bmalloc/Cache.h: Added.
|
||
(bmalloc::Cache::allocator):
|
||
(bmalloc::Cache::deallocator):
|
||
(bmalloc::Cache::allocateFastCase):
|
||
(bmalloc::Cache::deallocateFastCase):
|
||
(bmalloc::Cache::allocate):
|
||
(bmalloc::Cache::deallocate):
|
||
* bmalloc/Chunk.h: Added.
|
||
(bmalloc::Chunk::begin):
|
||
(bmalloc::Chunk::end):
|
||
(bmalloc::Chunk::lines):
|
||
(bmalloc::Chunk::pages):
|
||
* bmalloc/Deallocator.cpp: Added.
|
||
(bmalloc::Deallocator::Deallocator):
|
||
(bmalloc::Deallocator::~Deallocator):
|
||
(bmalloc::Deallocator::deallocateLarge):
|
||
(bmalloc::Deallocator::deallocateXLarge):
|
||
(bmalloc::Deallocator::processObjectLog):
|
||
(bmalloc::Deallocator::deallocateSlowCase):
|
||
(bmalloc::Deallocator::deallocateSmallLine):
|
||
(bmalloc::Deallocator::allocateSmallLine):
|
||
(bmalloc::Deallocator::deallocateMediumLine):
|
||
(bmalloc::Deallocator::allocateMediumLine):
|
||
* bmalloc/Deallocator.h: Added.
|
||
(bmalloc::Deallocator::deallocateFastCase):
|
||
(bmalloc::Deallocator::deallocate):
|
||
* bmalloc/EndTag.h: Added.
|
||
(bmalloc::EndTag::operator=):
|
||
* bmalloc/FixedVector.h: Added.
|
||
(bmalloc::FixedVector::begin):
|
||
(bmalloc::FixedVector::end):
|
||
(bmalloc::FixedVector::size):
|
||
(bmalloc::FixedVector::capacity):
|
||
(bmalloc::FixedVector::clear):
|
||
(bmalloc::FixedVector::isEmpty):
|
||
(bmalloc::Capacity>::FixedVector):
|
||
(bmalloc::Capacity>::operator):
|
||
(bmalloc::Capacity>::push):
|
||
(bmalloc::Capacity>::pop):
|
||
(bmalloc::Capacity>::shrink):
|
||
* bmalloc/Heap.cpp: Added.
|
||
(bmalloc::sleep):
|
||
(bmalloc::Heap::Heap):
|
||
(bmalloc::Heap::concurrentScavenge):
|
||
(bmalloc::Heap::scavengeSmallPages):
|
||
(bmalloc::Heap::scavengeMediumPages):
|
||
(bmalloc::Heap::scavengeLargeRanges):
|
||
(bmalloc::Heap::allocateSmallLineSlowCase):
|
||
(bmalloc::Heap::allocateMediumLineSlowCase):
|
||
(bmalloc::Heap::allocateXLarge):
|
||
(bmalloc::Heap::deallocateXLarge):
|
||
(bmalloc::Heap::allocateLarge):
|
||
(bmalloc::Heap::deallocateLarge):
|
||
* bmalloc/Heap.h: Added.
|
||
(bmalloc::Heap::deallocateSmallLine):
|
||
(bmalloc::Heap::allocateSmallLine):
|
||
(bmalloc::Heap::deallocateMediumLine):
|
||
(bmalloc::Heap::allocateMediumLine):
|
||
* bmalloc/Inline.h: Added.
|
||
* bmalloc/LargeChunk.h: Added.
|
||
(bmalloc::LargeChunk::begin):
|
||
(bmalloc::LargeChunk::end):
|
||
(bmalloc::LargeChunk::create):
|
||
(bmalloc::LargeChunk::get):
|
||
(bmalloc::LargeChunk::beginTag):
|
||
(bmalloc::LargeChunk::endTag):
|
||
* bmalloc/Line.h: Added.
|
||
(bmalloc::Line<Traits>::begin):
|
||
(bmalloc::Line<Traits>::end):
|
||
(bmalloc::Line<Traits>::concurrentRef):
|
||
(bmalloc::Line<Traits>::deref):
|
||
* bmalloc/MediumAllocator.h: Added.
|
||
(bmalloc::MediumAllocator::isNull):
|
||
(bmalloc::MediumAllocator::MediumAllocator):
|
||
(bmalloc::MediumAllocator::line):
|
||
(bmalloc::MediumAllocator::allocate):
|
||
(bmalloc::MediumAllocator::derefCount):
|
||
(bmalloc::MediumAllocator::refill):
|
||
* bmalloc/MediumChunk.h: Added.
|
||
* bmalloc/MediumLine.h: Added.
|
||
* bmalloc/MediumPage.h: Added.
|
||
* bmalloc/MediumTraits.h: Added.
|
||
* bmalloc/Mutex.cpp: Added.
|
||
(bmalloc::Mutex::lockSlowCase):
|
||
* bmalloc/Mutex.h: Added.
|
||
(bmalloc::Mutex::Mutex):
|
||
(bmalloc::Mutex::try_lock):
|
||
(bmalloc::Mutex::lock):
|
||
(bmalloc::Mutex::unlock):
|
||
* bmalloc/ObjectType.cpp: Added.
|
||
(bmalloc::objectType):
|
||
* bmalloc/ObjectType.h: Added.
|
||
(bmalloc::isSmallOrMedium):
|
||
(bmalloc::isSmall):
|
||
* bmalloc/Page.h: Added.
|
||
(bmalloc::Page<Traits>::ref):
|
||
(bmalloc::Page<Traits>::deref):
|
||
(bmalloc::Page<Traits>::refCount):
|
||
* bmalloc/PerProcess.h: Added.
|
||
(bmalloc::PerProcess::mutex):
|
||
(bmalloc::PerProcess<T>::getFastCase):
|
||
(bmalloc::PerProcess<T>::get):
|
||
(bmalloc::PerProcess<T>::getSlowCase):
|
||
* bmalloc/PerThread.h: Added.
|
||
(bmalloc::PerThreadStorage<Cache>::get):
|
||
(bmalloc::PerThreadStorage<Cache>::init):
|
||
(bmalloc::PerThreadStorage::get):
|
||
(bmalloc::PerThreadStorage::init):
|
||
(bmalloc::PerThread<T>::getFastCase):
|
||
(bmalloc::PerThread<T>::get):
|
||
(bmalloc::PerThread<T>::destructor):
|
||
(bmalloc::PerThread<T>::getSlowCase):
|
||
* bmalloc/Range.h: Added.
|
||
(bmalloc::Range::Range):
|
||
(bmalloc::Range::begin):
|
||
(bmalloc::Range::end):
|
||
(bmalloc::Range::size):
|
||
(bmalloc::Range::operator!):
|
||
(bmalloc::Range::operator<):
|
||
* bmalloc/SegregatedFreeList.cpp: Added.
|
||
(bmalloc::SegregatedFreeList::SegregatedFreeList):
|
||
(bmalloc::SegregatedFreeList::insert):
|
||
(bmalloc::SegregatedFreeList::takeGreedy):
|
||
(bmalloc::SegregatedFreeList::take):
|
||
* bmalloc/SegregatedFreeList.h: Added.
|
||
* bmalloc/Sizes.h: Added.
|
||
* bmalloc/SmallAllocator.h: Added.
|
||
(bmalloc::SmallAllocator::isNull):
|
||
(bmalloc::SmallAllocator::canAllocate):
|
||
(bmalloc::SmallAllocator::SmallAllocator):
|
||
(bmalloc::SmallAllocator::line):
|
||
(bmalloc::SmallAllocator::allocate):
|
||
(bmalloc::SmallAllocator::objectCount):
|
||
(bmalloc::SmallAllocator::derefCount):
|
||
(bmalloc::SmallAllocator::refill):
|
||
* bmalloc/SmallChunk.h: Added.
|
||
* bmalloc/SmallLine.h: Added.
|
||
* bmalloc/SmallPage.h: Added.
|
||
* bmalloc/SmallTraits.h: Added.
|
||
* bmalloc/Syscall.h: Added.
|
||
* bmalloc/VMAllocate.h: Added.
|
||
(bmalloc::vmSize):
|
||
(bmalloc::vmValidate):
|
||
(bmalloc::vmAllocate):
|
||
(bmalloc::vmDeallocate):
|
||
(bmalloc::vmDeallocatePhysicalPages):
|
||
(bmalloc::vmAllocatePhysicalPages):
|
||
(bmalloc::vmDeallocatePhysicalPagesSloppy):
|
||
(bmalloc::vmAllocatePhysicalPagesSloppy):
|
||
* bmalloc/VMHeap.cpp: Added.
|
||
(bmalloc::VMHeap::VMHeap):
|
||
(bmalloc::VMHeap::allocateSmallChunk):
|
||
(bmalloc::VMHeap::allocateMediumChunk):
|
||
(bmalloc::VMHeap::allocateLargeChunk):
|
||
* bmalloc/VMHeap.h: Added.
|
||
(bmalloc::VMHeap::allocateSmallPage):
|
||
(bmalloc::VMHeap::allocateMediumPage):
|
||
(bmalloc::VMHeap::allocateLargeRange):
|
||
(bmalloc::VMHeap::deallocateSmallPage):
|
||
(bmalloc::VMHeap::deallocateMediumPage):
|
||
(bmalloc::VMHeap::deallocateLargeRange):
|
||
* bmalloc/Vector.h: Added.
|
||
(bmalloc::Vector::begin):
|
||
(bmalloc::Vector::end):
|
||
(bmalloc::Vector::size):
|
||
(bmalloc::Vector::capacity):
|
||
(bmalloc::Vector::last):
|
||
(bmalloc::Vector::pop):
|
||
(bmalloc::Vector<T>::Vector):
|
||
(bmalloc::Vector<T>::~Vector):
|
||
(bmalloc::Vector<T>::operator):
|
||
(bmalloc::Vector<T>::push):
|
||
(bmalloc::Vector<T>::pop):
|
||
(bmalloc::Vector<T>::shrink):
|
||
(bmalloc::Vector<T>::reallocateBuffer):
|
||
(bmalloc::Vector<T>::shrinkCapacity):
|
||
(bmalloc::Vector<T>::growCapacity):
|
||
* bmalloc/XLargeChunk.h: Added.
|
||
(bmalloc::XLargeChunk::get):
|
||
(bmalloc::XLargeChunk::begin):
|
||
(bmalloc::XLargeChunk::XLargeChunk):
|
||
(bmalloc::XLargeChunk::create):
|
||
(bmalloc::XLargeChunk::destroy):
|
||
(bmalloc::XLargeChunk::range):
|
||
(bmalloc::XLargeChunk::size):
|
||
* bmalloc/bmalloc.h: Added.
|
||
(bmalloc::api::malloc):
|
||
(bmalloc::api::free):
|
||
(bmalloc::api::realloc):
|
||
* bmalloc/mbmalloc.cpp: Added.
|
||
|