The tracking of the coarse-grain Heap state (allocating or not, collector or not, eden vs full) should respect the orthogonality between allocating and collecting
https://bugs.webkit.org/show_bug.cgi?id=163738
Reviewed by Geoffrey Garen.
Source/JavaScriptCore:
We need to know if we're currently in an allocation slow path, so that code can assert that
it's not being used from inside a destructor that runs during a sweep. We need to know if
we're currently collecting, because some code behaves differently during collection, and
other code wants to assert that it's not being used from inside a visitChildren method that
runs during marking. If we are collecting, we need to know if it's an eden collection or a
full collection. If we are requesting a collection, we need to know if we're requesting an
eden collection, a full collection, or any kind of collection.
Prior to this change, you would reason about all of these things using the HeapOperation. It
had the following states: NoOperation, Allocation, FullCollection, EdenCollection, and
AnyCollection. NoOperation versus Allocation was primarily for asserting that sweep didn't
call arbitrary JS. FullCollection versus EdenCollection was about describing generations. We
would even use HeapOperation in places where we knew that it could only be either Full or
Eden, because we just needed a variable to tell us which generation we were talking about.
It was all very confusing.
Where it completely breaks down is the fact that a concurrent GC has two logical threads, the
mutator and the collector, which can change state independently. The mutator can be
allocating. It can also be doing some work to help the GC. That's three states: running,
allocating, or helping GC. At the same time, the GC thread could either be running or not,
and if it's running, it could be a full collection or an eden collection. Because the mutator
and collector can run at the same time, it means that if we used one enum, we would need nine
states: every combination of mutator running, allocating, or helping GC, crossed with
collector not running, running eden, or running full. So, this change decouples mutator state
from collector state and uses two separate fields with two different types.
Mutator state is described using MutatorState, which can be either MutatorState::Running,
MutatorState::Allocating, or MutatorState::HelpingGC.
Collector state is described using Optional<CollectionScope>. CollectionScope describes how
big the scope of the collection is, and it can be either CollectionScope::Eden or
CollectionScope::Full. If the Optional is Nullopt, it means that we are not collecting. This
way, you can treat collectionScope as a boolean (an Optional is true iff it's engaged). You
can pass around just a CollectionScope if you know that you must be collecting and you just
want to know about the generation. Also, we can use Nullopt in methods that request
collection, which those methods take to mean that they can run any kind of collection (the
old AnyCollection).
Another use of HeapOperation was to answer questions about whether the caller is running as
part of the GC or as part of the mutator. Optional<CollectionScope> does not answer this,
since code that runs in the mutator while the mutator is not HelpingGC at the same time as
the collector is running should run as if it was part of the mutator not as if it was part of
the GC. MutatorState is needed to answer this question, but it doesn't tell the whole story
since code that runs in the collector thread at the same time as the mutator is running
should run as if it was part of the GC not as if it was part of the mutator. So, we need to
know if we're on the collector thread or the mutator thread. We already have a WTF facility
for this, which answers if a thread is a GC thread. But we already use this to answer a
stronger question: are we part of the parallel GC helpers? Some functions in the GC, like
mark bit queries, will work fine in a concurrent collector thread so long as there is no
parallel marking. So, this change also changes WTF's mayBeGCThread to tell what kind of GC
thread we may be: either GCThreadType::Main or GCThreadType::Helper. The parallel GC safety
checks look for GCThreadType::Helper. The "should I run as mutator" query can now be answered
by checking with mayBeGCThread, which returns Optional<GCThreadType>; if engaged, then run as
GC, else run as GC if MutatorState is HelpingGC, else run as mutator.
This doesn't change the way that the GC behaves, but it does change how the GC represents a
fundamental piece of state. So, it's a big change. It should be perf-neutral (still testing).
* API/JSBase.cpp:
(JSSynchronousEdenCollectForDebugging):
* CMakeLists.txt:
* JavaScriptCore.xcodeproj/project.pbxproj:
* bytecode/CodeBlock.cpp:
(JSC::CodeBlock::jettison):
* dfg/DFGWorklist.cpp:
* ftl/FTLCompile.cpp:
(JSC::FTL::compile):
* heap/AllocatingScope.h: Added.
(JSC::AllocatingScope::AllocatingScope):
(JSC::AllocatingScope::~AllocatingScope):
* heap/AllocationScope.h: Removed.
* heap/CodeBlockSet.cpp:
(JSC::CodeBlockSet::deleteUnmarkedAndUnreferenced):
* heap/CodeBlockSet.h:
* heap/CollectionScope.cpp: Added.
(JSC::collectionScopeName):
(WTF::printInternal):
* heap/CollectionScope.h: Added.
* heap/EdenGCActivityCallback.cpp:
(JSC::EdenGCActivityCallback::doCollection):
* heap/FullGCActivityCallback.cpp:
(JSC::FullGCActivityCallback::doCollection):
* heap/GCTypeMap.h:
(JSC::GCTypeMap::operator[]):
* heap/Heap.cpp:
(JSC::Heap::Heap):
(JSC::Heap::lastChanceToFinalize):
(JSC::Heap::markRoots):
(JSC::Heap::beginMarking):
(JSC::Heap::visitSmallStrings):
(JSC::Heap::updateObjectCounts):
(JSC::Heap::deleteAllCodeBlocks):
(JSC::Heap::deleteUnmarkedCompiledCode):
(JSC::Heap::collectAllGarbage):
(JSC::Heap::collect):
(JSC::Heap::collectWithoutAnySweep):
(JSC::Heap::collectImpl):
(JSC::Heap::willStartCollection):
(JSC::Heap::flushWriteBarrierBuffer):
(JSC::Heap::pruneStaleEntriesFromWeakGCMaps):
(JSC::Heap::notifyIncrementalSweeper):
(JSC::Heap::updateAllocationLimits):
(JSC::Heap::didFinishCollection):
(JSC::Heap::isValidAllocation):
(JSC::Heap::shouldDoFullCollection):
* heap/Heap.h:
(JSC::Heap::mutatorState):
(JSC::Heap::collectionScope):
(JSC::Heap::operationInProgress): Deleted.
* heap/HeapInlines.h:
(JSC::Heap::shouldCollect):
(JSC::Heap::isCurrentThreadBusy):
(JSC::Heap::isMarked):
(JSC::Heap::reportExtraMemoryVisited):
(JSC::Heap::reportExternalMemoryVisited):
(JSC::Heap::collectAccordingToDeferGCProbability):
(JSC::Heap::isBusy): Deleted.
(JSC::Heap::isCollecting): Deleted.
* heap/HeapObserver.h:
* heap/HeapOperation.cpp: Removed.
* heap/HeapOperation.h: Removed.
* heap/HeapVerifier.cpp:
(JSC::HeapVerifier::initializeGCCycle):
(JSC::HeapVerifier::reportObject):
(JSC::HeapVerifier::collectionTypeName): Deleted.
* heap/HeapVerifier.h:
(JSC::HeapVerifier::GCCycle::collectionTypeName): Deleted.
* heap/HelpingGCScope.h: Added.
(JSC::HelpingGCScope::HelpingGCScope):
(JSC::HelpingGCScope::~HelpingGCScope):
* heap/LargeAllocation.cpp:
(JSC::LargeAllocation::flip):
* heap/MarkedAllocator.cpp:
(JSC::MarkedAllocator::doTestCollectionsIfNeeded):
(JSC::MarkedAllocator::allocateSlowCaseImpl):
* heap/MarkedBlock.h:
* heap/MarkedSpace.cpp:
(JSC::MarkedSpace::prepareForAllocation):
(JSC::MarkedSpace::visitWeakSets):
(JSC::MarkedSpace::reapWeakSets):
(JSC::MarkedSpace::prepareForMarking):
(JSC::MarkedSpace::beginMarking):
(JSC::MarkedSpace::snapshotUnswept):
* heap/MutatorState.cpp: Added.
(WTF::printInternal):
* heap/MutatorState.h: Added.
* heap/SlotVisitor.cpp:
(JSC::SlotVisitor::didStartMarking):
* inspector/agents/InspectorHeapAgent.cpp:
(Inspector::protocolTypeForHeapOperation):
(Inspector::InspectorHeapAgent::didGarbageCollect):
* inspector/agents/InspectorHeapAgent.h:
* interpreter/Interpreter.cpp:
(JSC::Interpreter::execute):
(JSC::Interpreter::executeCall):
(JSC::Interpreter::executeConstruct):
(JSC::Interpreter::prepareForRepeatCall):
* jsc.cpp:
(functionFullGC):
(functionEdenGC):
* runtime/Completion.cpp:
(JSC::evaluate):
(JSC::loadAndEvaluateModule):
(JSC::loadModule):
(JSC::linkAndEvaluateModule):
* runtime/JSLock.cpp:
(JSC::JSLock::DropAllLocks::DropAllLocks):
* runtime/SmallStrings.h:
(JSC::SmallStrings::needsToBeVisited):
* runtime/VM.h:
(JSC::VM::isCollectorBusyOnCurrentThread):
(JSC::VM::isCollectorBusy): Deleted.
* tools/JSDollarVMPrototype.cpp:
(JSC::JSDollarVMPrototype::edenGC):
Source/WebCore:
No new tests because no change in behavior.
* bindings/js/GCController.cpp:
(WebCore::GCController::garbageCollectNow):
Source/WTF:
There will soon be different kinds of GC threads, and WTF's "are you a GC thread" thing
should know about this.
* wtf/MainThread.cpp:
(WTF::initializeGCThreads):
(WTF::registerGCThread):
(WTF::mayBeGCThread):
* wtf/MainThread.h:
git-svn-id: http://svn.webkit.org/repository/webkit/trunk@207653 268f45cc-cd09-0410-ab3c-d52691b4dbfc
diff --git a/Source/JavaScriptCore/heap/HeapOperation.h b/Source/JavaScriptCore/heap/AllocatingScope.h
similarity index 72%
copy from Source/JavaScriptCore/heap/HeapOperation.h
copy to Source/JavaScriptCore/heap/AllocatingScope.h
index fbb8283..1d13ae0 100644
--- a/Source/JavaScriptCore/heap/HeapOperation.h
+++ b/Source/JavaScriptCore/heap/AllocatingScope.h
@@ -1,5 +1,5 @@
/*
- * Copyright (C) 2013 Apple Inc. All rights reserved.
+ * Copyright (C) 2016 Apple Inc. All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
@@ -25,16 +25,28 @@
#pragma once
+#include "Heap.h"
+
namespace JSC {
-enum HeapOperation { NoOperation, Allocation, FullCollection, EdenCollection, AnyCollection };
+class AllocatingScope {
+public:
+ AllocatingScope(Heap& heap)
+ : m_heap(heap)
+ {
+ RELEASE_ASSERT(m_heap.m_mutatorState == MutatorState::Running);
+ m_heap.m_mutatorState = MutatorState::Allocating;
+ }
+
+ ~AllocatingScope()
+ {
+ RELEASE_ASSERT(m_heap.m_mutatorState == MutatorState::Allocating);
+ m_heap.m_mutatorState = MutatorState::Running;
+ }
+
+private:
+ Heap& m_heap;
+};
} // namespace JSC
-namespace WTF {
-
-class PrintStream;
-
-void printInternal(PrintStream& out, JSC::HeapOperation);
-
-} // namespace WTF
diff --git a/Source/JavaScriptCore/heap/AllocationScope.h b/Source/JavaScriptCore/heap/AllocationScope.h
deleted file mode 100644
index 2e4a654..0000000
--- a/Source/JavaScriptCore/heap/AllocationScope.h
+++ /dev/null
@@ -1,53 +0,0 @@
-/*
- * Copyright (C) 2016 Apple Inc. All rights reserved.
- *
- * Redistribution and use in source and binary forms, with or without
- * modification, are permitted provided that the following conditions
- * are met:
- * 1. Redistributions of source code must retain the above copyright
- * notice, this list of conditions and the following disclaimer.
- * 2. Redistributions in binary form must reproduce the above copyright
- * notice, this list of conditions and the following disclaimer in the
- * documentation and/or other materials provided with the distribution.
- *
- * THIS SOFTWARE IS PROVIDED BY APPLE INC. ``AS IS'' AND ANY
- * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
- * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
- * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR
- * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
- * EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
- * PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
- * PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
- * OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
- * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
- * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
- */
-
-#pragma once
-
-#include "Heap.h"
-
-namespace JSC {
-
-class AllocationScope {
-public:
- AllocationScope(Heap& heap)
- : m_heap(heap)
- , m_lastOperation(m_heap.m_operationInProgress)
- {
- ASSERT(m_lastOperation == NoOperation || m_lastOperation == Allocation);
- m_heap.m_operationInProgress = Allocation;
- }
-
- ~AllocationScope()
- {
- ASSERT(m_heap.m_operationInProgress == Allocation);
- m_heap.m_operationInProgress = m_lastOperation;
- }
-private:
- Heap& m_heap;
- HeapOperation m_lastOperation;
-};
-
-} // namespace JSC
-
diff --git a/Source/JavaScriptCore/heap/CodeBlockSet.cpp b/Source/JavaScriptCore/heap/CodeBlockSet.cpp
index c9ab0be..ec07db2 100644
--- a/Source/JavaScriptCore/heap/CodeBlockSet.cpp
+++ b/Source/JavaScriptCore/heap/CodeBlockSet.cpp
@@ -1,5 +1,5 @@
/*
- * Copyright (C) 2013-2015 Apple Inc. All rights reserved.
+ * Copyright (C) 2013-2016 Apple Inc. All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
@@ -77,10 +77,10 @@
codeBlock->classInfo()->methodTable.destroy(codeBlock);
}
-void CodeBlockSet::deleteUnmarkedAndUnreferenced(HeapOperation collectionType)
+void CodeBlockSet::deleteUnmarkedAndUnreferenced(CollectionScope scope)
{
LockHolder locker(&m_lock);
- HashSet<CodeBlock*>& set = collectionType == EdenCollection ? m_newCodeBlocks : m_oldCodeBlocks;
+ HashSet<CodeBlock*>& set = scope == CollectionScope::Eden ? m_newCodeBlocks : m_oldCodeBlocks;
Vector<CodeBlock*> unmarked;
for (CodeBlock* codeBlock : set) {
if (Heap::isMarked(codeBlock))
@@ -94,7 +94,7 @@
}
// Any remaining young CodeBlocks are live and need to be promoted to the set of old CodeBlocks.
- if (collectionType == EdenCollection)
+ if (scope == CollectionScope::Eden)
promoteYoungCodeBlocks(locker);
}
diff --git a/Source/JavaScriptCore/heap/CodeBlockSet.h b/Source/JavaScriptCore/heap/CodeBlockSet.h
index fd3800e..68caf30 100644
--- a/Source/JavaScriptCore/heap/CodeBlockSet.h
+++ b/Source/JavaScriptCore/heap/CodeBlockSet.h
@@ -1,5 +1,5 @@
/*
- * Copyright (C) 2013, 2014 Apple Inc. All rights reserved.
+ * Copyright (C) 2013-2014, 2016 Apple Inc. All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
@@ -25,8 +25,8 @@
#pragma once
+#include "CollectionScope.h"
#include "GCSegmentedArray.h"
-#include "HeapOperation.h"
#include <wtf/HashSet.h>
#include <wtf/Lock.h>
#include <wtf/Noncopyable.h>
@@ -67,7 +67,7 @@
// Delete all code blocks that are only referenced by this set (i.e. owned
// by this set), and that have not been marked.
- void deleteUnmarkedAndUnreferenced(HeapOperation);
+ void deleteUnmarkedAndUnreferenced(CollectionScope);
// Add all currently executing CodeBlocks to the remembered set to be
// re-scanned during the next collection.
diff --git a/Source/JavaScriptCore/heap/HeapOperation.cpp b/Source/JavaScriptCore/heap/CollectionScope.cpp
similarity index 75%
copy from Source/JavaScriptCore/heap/HeapOperation.cpp
copy to Source/JavaScriptCore/heap/CollectionScope.cpp
index 8715314..b2990e7 100644
--- a/Source/JavaScriptCore/heap/HeapOperation.cpp
+++ b/Source/JavaScriptCore/heap/CollectionScope.cpp
@@ -24,37 +24,32 @@
*/
#include "config.h"
-#include "HeapOperation.h"
+#include "CollectionScope.h"
#include <wtf/PrintStream.h>
-namespace WTF {
+namespace JSC {
-using namespace JSC;
-
-void printInternal(PrintStream& out, HeapOperation operation)
+const char* collectionScopeName(CollectionScope scope)
{
- switch (operation) {
- case NoOperation:
- out.print("None");
- return;
- case Allocation:
- out.print("Alloc");
- return;
- case FullCollection:
- out.print("Full");
- return;
- case EdenCollection:
- out.print("Eden");
- return;
- case AnyCollection:
- out.print("Any");
- return;
+ switch (scope) {
+ case CollectionScope::Eden:
+ return "Eden";
+ case CollectionScope::Full:
+ return "Full";
}
RELEASE_ASSERT_NOT_REACHED();
+ return nullptr;
+}
+
+} // namespace JSC
+
+namespace WTF {
+
+void printInternal(PrintStream& out, JSC::CollectionScope scope)
+{
+ out.print(JSC::collectionScopeName(scope));
}
} // namespace WTF
-
-
diff --git a/Source/JavaScriptCore/heap/HeapOperation.h b/Source/JavaScriptCore/heap/CollectionScope.h
similarity index 86%
rename from Source/JavaScriptCore/heap/HeapOperation.h
rename to Source/JavaScriptCore/heap/CollectionScope.h
index fbb8283..9b4f487 100644
--- a/Source/JavaScriptCore/heap/HeapOperation.h
+++ b/Source/JavaScriptCore/heap/CollectionScope.h
@@ -1,5 +1,5 @@
/*
- * Copyright (C) 2013 Apple Inc. All rights reserved.
+ * Copyright (C) 2016 Apple Inc. All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
@@ -27,7 +27,9 @@
namespace JSC {
-enum HeapOperation { NoOperation, Allocation, FullCollection, EdenCollection, AnyCollection };
+enum class CollectionScope { Eden, Full };
+
+const char* collectionScopeName(CollectionScope);
} // namespace JSC
@@ -35,6 +37,7 @@
class PrintStream;
-void printInternal(PrintStream& out, JSC::HeapOperation);
+void printInternal(PrintStream& out, JSC::CollectionScope);
} // namespace WTF
+
diff --git a/Source/JavaScriptCore/heap/EdenGCActivityCallback.cpp b/Source/JavaScriptCore/heap/EdenGCActivityCallback.cpp
index 7774075..1d2a6fc 100644
--- a/Source/JavaScriptCore/heap/EdenGCActivityCallback.cpp
+++ b/Source/JavaScriptCore/heap/EdenGCActivityCallback.cpp
@@ -1,5 +1,5 @@
/*
- * Copyright (C) 2014 Apple Inc. All rights reserved.
+ * Copyright (C) 2014, 2016 Apple Inc. All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
@@ -39,7 +39,7 @@
void EdenGCActivityCallback::doCollection()
{
- m_vm->heap.collect(EdenCollection);
+ m_vm->heap.collect(CollectionScope::Eden);
}
double EdenGCActivityCallback::lastGCLength()
diff --git a/Source/JavaScriptCore/heap/FullGCActivityCallback.cpp b/Source/JavaScriptCore/heap/FullGCActivityCallback.cpp
index 91051a6..5ddfa87 100644
--- a/Source/JavaScriptCore/heap/FullGCActivityCallback.cpp
+++ b/Source/JavaScriptCore/heap/FullGCActivityCallback.cpp
@@ -1,5 +1,5 @@
/*
- * Copyright (C) 2014 Apple Inc. All rights reserved.
+ * Copyright (C) 2014, 2016 Apple Inc. All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
@@ -55,7 +55,7 @@
}
#endif
- heap.collect(FullCollection);
+ heap.collect(CollectionScope::Full);
}
double FullGCActivityCallback::lastGCLength()
diff --git a/Source/JavaScriptCore/heap/GCTypeMap.h b/Source/JavaScriptCore/heap/GCTypeMap.h
index d0df25c..9e19e67 100644
--- a/Source/JavaScriptCore/heap/GCTypeMap.h
+++ b/Source/JavaScriptCore/heap/GCTypeMap.h
@@ -25,7 +25,7 @@
#pragma once
-#include "HeapOperation.h"
+#include "CollectionScope.h"
#include <wtf/Assertions.h>
namespace JSC {
@@ -35,20 +35,28 @@
T eden;
T full;
- T& operator[](HeapOperation operation)
+ T& operator[](CollectionScope scope)
{
- if (operation == FullCollection)
+ switch (scope) {
+ case CollectionScope::Full:
return full;
- ASSERT(operation == EdenCollection);
- return eden;
+ case CollectionScope::Eden:
+ return eden;
+ }
+ ASSERT_NOT_REACHED();
+ return full;
}
- const T& operator[](HeapOperation operation) const
+ const T& operator[](CollectionScope scope) const
{
- if (operation == FullCollection)
+ switch (scope) {
+ case CollectionScope::Full:
return full;
- ASSERT(operation == EdenCollection);
- return eden;
+ case CollectionScope::Eden:
+ return eden;
+ }
+ RELEASE_ASSERT_NOT_REACHED();
+ return full;
}
};
diff --git a/Source/JavaScriptCore/heap/Heap.cpp b/Source/JavaScriptCore/heap/Heap.cpp
index 3e408f6..35a6b48 100644
--- a/Source/JavaScriptCore/heap/Heap.cpp
+++ b/Source/JavaScriptCore/heap/Heap.cpp
@@ -39,6 +39,7 @@
#include "HeapSnapshot.h"
#include "HeapStatistics.h"
#include "HeapVerifier.h"
+#include "HelpingGCScope.h"
#include "IncrementalSweeper.h"
#include "Interpreter.h"
#include "JITStubRoutineSet.h"
@@ -140,15 +141,15 @@
return *result;
}
-SimpleStats& timingStats(const char* name, HeapOperation operation)
+SimpleStats& timingStats(const char* name, CollectionScope scope)
{
- return timingStats().add(name, GCTypeMap<SimpleStats>()).iterator->value[operation];
+ return timingStats().add(name, GCTypeMap<SimpleStats>()).iterator->value[scope];
}
class TimingScope {
public:
- TimingScope(HeapOperation operation, const char* name)
- : m_operation(operation)
+ TimingScope(Optional<CollectionScope> scope, const char* name)
+ : m_scope(scope)
, m_name(name)
{
if (measurePhaseTiming())
@@ -156,18 +157,18 @@
}
TimingScope(Heap& heap, const char* name)
- : TimingScope(heap.operationInProgress(), name)
+ : TimingScope(heap.collectionScope(), name)
{
}
- void setOperation(HeapOperation operation)
+ void setScope(Optional<CollectionScope> scope)
{
- m_operation = operation;
+ m_scope = scope;
}
- void setOperation(Heap& heap)
+ void setScope(Heap& heap)
{
- setOperation(heap.operationInProgress());
+ setScope(heap.collectionScope());
}
~TimingScope()
@@ -175,13 +176,13 @@
if (measurePhaseTiming()) {
double after = monotonicallyIncreasingTimeMS();
double timing = after - m_before;
- SimpleStats& stats = timingStats(m_name, m_operation);
+ SimpleStats& stats = timingStats(m_name, *m_scope);
stats.add(timing);
- dataLog("[GC:", m_operation, "] ", m_name, " took: ", timing, " ms (average ", stats.mean(), " ms).\n");
+ dataLog("[GC:", *m_scope, "] ", m_name, " took: ", timing, " ms (average ", stats.mean(), " ms).\n");
}
}
private:
- HeapOperation m_operation;
+ Optional<CollectionScope> m_scope;
double m_before;
const char* m_name;
};
@@ -203,7 +204,6 @@
, m_maxHeapSize(m_minBytesPerCycle)
, m_shouldDoFullCollection(false)
, m_totalBytesVisited(0)
- , m_operationInProgress(NoOperation)
, m_objectSpace(this)
, m_extraMemorySize(0)
, m_deprecatedExtraMemorySize(0)
@@ -252,7 +252,8 @@
void Heap::lastChanceToFinalize()
{
RELEASE_ASSERT(!m_vm->entryScope);
- RELEASE_ASSERT(m_operationInProgress == NoOperation);
+ RELEASE_ASSERT(!m_collectionScope);
+ RELEASE_ASSERT(m_mutatorState == MutatorState::Running);
m_arrayBuffers.lastChanceToFinalize();
m_codeBlocks->lastChanceToFinalize();
@@ -408,7 +409,7 @@
}
#endif // ENABLE(SAMPLING_PROFILER)
- if (m_operationInProgress == FullCollection) {
+ if (m_collectionScope == CollectionScope::Full) {
m_opaqueRoots.clear();
m_slotVisitor.clearMarkStack();
}
@@ -431,7 +432,7 @@
slotVisitor = m_availableParallelSlotVisitors.takeLast();
}
- WTF::registerGCThread();
+ WTF::registerGCThread(GCThreadType::Helper);
{
ParallelModeEnabler parallelModeEnabler(*slotVisitor);
@@ -526,7 +527,7 @@
void Heap::beginMarking()
{
TimingScope timingScope(*this, "Heap::beginMarking");
- if (m_operationInProgress == FullCollection)
+ if (m_collectionScope == CollectionScope::Full)
m_codeBlocks->clearMarksForFullCollection();
{
@@ -544,7 +545,7 @@
void Heap::visitSmallStrings()
{
- if (!m_vm->smallStrings.needsToBeVisited(m_operationInProgress))
+ if (!m_vm->smallStrings.needsToBeVisited(*m_collectionScope))
return;
m_vm->smallStrings.visitStrongReferences(m_slotVisitor);
@@ -763,7 +764,7 @@
dataLogF("\nNumber of live Objects after GC %lu, took %.6f secs\n", static_cast<unsigned long>(visitCount), WTF::monotonicallyIncreasingTime() - gcStartTime);
}
- if (m_operationInProgress == FullCollection)
+ if (m_collectionScope == CollectionScope::Full)
m_totalBytesVisited = 0;
m_totalBytesVisitedThisCycle = m_slotVisitor.bytesVisited() + threadBytesVisited();
@@ -871,7 +872,7 @@
// If JavaScript is running, it's not safe to delete all JavaScript code, since
// we'll end up returning to deleted code.
RELEASE_ASSERT(!m_vm->entryScope);
- ASSERT(m_operationInProgress == NoOperation);
+ ASSERT(!m_collectionScope);
completeAllJITPlans();
@@ -908,7 +909,7 @@
void Heap::deleteUnmarkedCompiledCode()
{
clearUnmarkedExecutables();
- m_codeBlocks->deleteUnmarkedAndUnreferenced(m_operationInProgress);
+ m_codeBlocks->deleteUnmarkedAndUnreferenced(*m_collectionScope);
m_jitStubRoutines->deleteUnmarkedJettisonedStubRoutines();
}
@@ -932,7 +933,7 @@
if (!m_isSafeToCollect)
return;
- collectWithoutAnySweep(FullCollection);
+ collectWithoutAnySweep(CollectionScope::Full);
DeferGCForAWhile deferGC(*this);
if (UNLIKELY(Options::useImmortalObjects()))
@@ -955,29 +956,29 @@
sweepAllLogicallyEmptyWeakBlocks();
}
-void Heap::collect(HeapOperation collectionType)
+void Heap::collect(Optional<CollectionScope> scope)
{
SuperSamplerScope superSamplerScope(false);
if (!m_isSafeToCollect)
return;
-
- collectWithoutAnySweep(collectionType);
+
+ collectWithoutAnySweep(scope);
}
-NEVER_INLINE void Heap::collectWithoutAnySweep(HeapOperation collectionType)
+NEVER_INLINE void Heap::collectWithoutAnySweep(Optional<CollectionScope> scope)
{
void* stackTop;
ALLOCATE_AND_GET_REGISTER_STATE(registers);
- collectImpl(collectionType, wtfThreadData().stack().origin(), &stackTop, registers);
+ collectImpl(scope, wtfThreadData().stack().origin(), &stackTop, registers);
sanitizeStackForVM(m_vm);
}
-NEVER_INLINE void Heap::collectImpl(HeapOperation collectionType, void* stackOrigin, void* stackTop, MachineThreads::RegisterState& calleeSavedRegisters)
+NEVER_INLINE void Heap::collectImpl(Optional<CollectionScope> scope, void* stackOrigin, void* stackTop, MachineThreads::RegisterState& calleeSavedRegisters)
{
SuperSamplerScope superSamplerScope(false);
- TimingScope collectImplTimingScope(collectionType, "Heap::collectImpl");
+ TimingScope collectImplTimingScope(scope, "Heap::collectImpl");
#if ENABLE(ALLOCATION_LOGGING)
dataLogF("JSC GC starting collection.\n");
@@ -990,81 +991,78 @@
}
double gcStartTime;
- {
- TimingScope earlyTimingScope(collectionType, "Heap::collectImpl before markRoots");
-
- if (vm()->typeProfiler()) {
- DeferGCForAWhile awhile(*this);
- vm()->typeProfilerLog()->processLogEntries(ASCIILiteral("GC"));
- }
-
+
+ if (vm()->typeProfiler()) {
+ DeferGCForAWhile awhile(*this);
+ vm()->typeProfilerLog()->processLogEntries(ASCIILiteral("GC"));
+ }
+
#if ENABLE(JIT)
- {
- DeferGCForAWhile awhile(*this);
- JITWorklist::instance()->completeAllForVM(*m_vm);
- }
+ {
+ DeferGCForAWhile awhile(*this);
+ JITWorklist::instance()->completeAllForVM(*m_vm);
+ }
#endif // ENABLE(JIT)
-
- vm()->shadowChicken().update(*vm(), vm()->topCallFrame);
-
- RELEASE_ASSERT(!m_deferralDepth);
- ASSERT(vm()->currentThreadIsHoldingAPILock());
- RELEASE_ASSERT(vm()->atomicStringTable() == wtfThreadData().atomicStringTable());
- ASSERT(m_isSafeToCollect);
- RELEASE_ASSERT(m_operationInProgress == NoOperation);
-
- suspendCompilerThreads();
- willStartCollection(collectionType);
+
+ vm()->shadowChicken().update(*vm(), vm()->topCallFrame);
+
+ RELEASE_ASSERT(!m_deferralDepth);
+ ASSERT(vm()->currentThreadIsHoldingAPILock());
+ RELEASE_ASSERT(vm()->atomicStringTable() == wtfThreadData().atomicStringTable());
+ ASSERT(m_isSafeToCollect);
+ RELEASE_ASSERT(!m_collectionScope);
+
+ suspendCompilerThreads();
+ willStartCollection(scope);
+ {
+ HelpingGCScope helpingHeapScope(*this);
- collectImplTimingScope.setOperation(*this);
- earlyTimingScope.setOperation(*this);
-
+ collectImplTimingScope.setScope(*this);
+
gcStartTime = WTF::monotonicallyIncreasingTime();
if (m_verifier) {
// Verify that live objects from the last GC cycle haven't been corrupted by
// mutators before we begin this new GC cycle.
m_verifier->verify(HeapVerifier::Phase::BeforeGC);
-
+
m_verifier->initializeGCCycle();
m_verifier->gatherLiveObjects(HeapVerifier::Phase::BeforeMarking);
}
-
+
flushOldStructureIDTables();
stopAllocation();
prepareForMarking();
flushWriteBarrierBuffer();
-
+
if (HasOwnPropertyCache* cache = vm()->hasOwnPropertyCache())
cache->clear();
+
+ markRoots(gcStartTime, stackOrigin, stackTop, calleeSavedRegisters);
+
+ if (m_verifier) {
+ m_verifier->gatherLiveObjects(HeapVerifier::Phase::AfterMarking);
+ m_verifier->verify(HeapVerifier::Phase::AfterMarking);
+ }
+
+ if (vm()->typeProfiler())
+ vm()->typeProfiler()->invalidateTypeSetCache();
+
+ reapWeakHandles();
+ pruneStaleEntriesFromWeakGCMaps();
+ sweepArrayBuffers();
+ snapshotUnswept();
+ finalizeUnconditionalFinalizers();
+ removeDeadCompilerWorklistEntries();
+ deleteUnmarkedCompiledCode();
+ deleteSourceProviderCaches();
+
+ notifyIncrementalSweeper();
+ m_codeBlocks->writeBarrierCurrentlyExecuting(this);
+ m_codeBlocks->clearCurrentlyExecuting();
+
+ prepareForAllocation();
+ updateAllocationLimits();
}
-
- markRoots(gcStartTime, stackOrigin, stackTop, calleeSavedRegisters);
-
- TimingScope lateTimingScope(*this, "Heap::collectImpl after markRoots");
-
- if (m_verifier) {
- m_verifier->gatherLiveObjects(HeapVerifier::Phase::AfterMarking);
- m_verifier->verify(HeapVerifier::Phase::AfterMarking);
- }
-
- if (vm()->typeProfiler())
- vm()->typeProfiler()->invalidateTypeSetCache();
-
- reapWeakHandles();
- pruneStaleEntriesFromWeakGCMaps();
- sweepArrayBuffers();
- snapshotUnswept();
- finalizeUnconditionalFinalizers();
- removeDeadCompilerWorklistEntries();
- deleteUnmarkedCompiledCode();
- deleteSourceProviderCaches();
-
- notifyIncrementalSweeper();
- m_codeBlocks->writeBarrierCurrentlyExecuting(this);
- m_codeBlocks->clearCurrentlyExecuting();
-
- prepareForAllocation();
- updateAllocationLimits();
didFinishCollection(gcStartTime);
resumeCompilerThreads();
sweepLargeAllocations();
@@ -1103,22 +1101,22 @@
#endif
}
-void Heap::willStartCollection(HeapOperation collectionType)
+void Heap::willStartCollection(Optional<CollectionScope> scope)
{
if (Options::logGC())
dataLog("=> ");
- if (shouldDoFullCollection(collectionType)) {
- m_operationInProgress = FullCollection;
+ if (shouldDoFullCollection(scope)) {
+ m_collectionScope = CollectionScope::Full;
m_shouldDoFullCollection = false;
if (Options::logGC())
dataLog("FullCollection, ");
} else {
- m_operationInProgress = EdenCollection;
+ m_collectionScope = CollectionScope::Eden;
if (Options::logGC())
dataLog("EdenCollection, ");
}
- if (m_operationInProgress == FullCollection) {
+ if (m_collectionScope == CollectionScope::Full) {
m_sizeBeforeLastFullCollect = m_sizeAfterLastCollect + m_bytesAllocatedThisCycle;
m_extraMemorySize = 0;
m_deprecatedExtraMemorySize = 0;
@@ -1129,7 +1127,7 @@
if (m_fullActivityCallback)
m_fullActivityCallback->willCollect();
} else {
- ASSERT(m_operationInProgress == EdenCollection);
+ ASSERT(m_collectionScope == CollectionScope::Eden);
m_sizeBeforeLastEdenCollect = m_sizeAfterLastCollect + m_bytesAllocatedThisCycle;
}
@@ -1147,7 +1145,7 @@
void Heap::flushWriteBarrierBuffer()
{
- if (m_operationInProgress == EdenCollection) {
+ if (m_collectionScope == CollectionScope::Eden) {
m_writeBarrierBuffer.flush(*this);
return;
}
@@ -1171,7 +1169,7 @@
void Heap::pruneStaleEntriesFromWeakGCMaps()
{
- if (m_operationInProgress != FullCollection)
+ if (m_collectionScope != CollectionScope::Full)
return;
for (auto& pruneCallback : m_weakGCMaps.values())
pruneCallback();
@@ -1195,7 +1193,7 @@
void Heap::notifyIncrementalSweeper()
{
- if (m_operationInProgress == FullCollection) {
+ if (m_collectionScope == CollectionScope::Full) {
if (!m_logicallyEmptyWeakBlocks.isEmpty())
m_indexOfNextLogicallyEmptyWeakBlockToSweep = 0;
}
@@ -1243,7 +1241,7 @@
if (Options::gcMaxHeapSize() && currentHeapSize > Options::gcMaxHeapSize())
HeapStatistics::exitWithFailure();
- if (m_operationInProgress == FullCollection) {
+ if (m_collectionScope == CollectionScope::Full) {
// To avoid pathological GC churn in very small and very large heaps, we set
// the new allocation limit based on the current size of the heap, with a
// fixed minimum.
@@ -1298,8 +1296,8 @@
void Heap::didFinishCollection(double gcStartTime)
{
double gcEndTime = WTF::monotonicallyIncreasingTime();
- HeapOperation operation = m_operationInProgress;
- if (m_operationInProgress == FullCollection)
+ CollectionScope scope = *m_collectionScope;
+ if (scope == CollectionScope::Full)
m_lastFullGCLength = gcEndTime - gcStartTime;
else
m_lastEdenGCLength = gcEndTime - gcStartTime;
@@ -1322,11 +1320,11 @@
removeDeadHeapSnapshotNodes(*heapProfiler);
}
- RELEASE_ASSERT(m_operationInProgress == EdenCollection || m_operationInProgress == FullCollection);
- m_operationInProgress = NoOperation;
+ RELEASE_ASSERT(m_collectionScope);
+ m_collectionScope = Nullopt;
for (auto* observer : m_observers)
- observer->didGarbageCollect(operation);
+ observer->didGarbageCollect(scope);
}
void Heap::resumeCompilerThreads()
@@ -1388,7 +1386,7 @@
if (!isValidThreadState(m_vm))
return false;
- if (m_operationInProgress != NoOperation)
+ if (isCurrentThreadBusy())
return false;
return true;
@@ -1465,24 +1463,14 @@
m_writeBarrierBuffer.add(cell);
}
-bool Heap::shouldDoFullCollection(HeapOperation requestedCollectionType) const
+bool Heap::shouldDoFullCollection(Optional<CollectionScope> scope) const
{
if (!Options::useGenerationalGC())
return true;
- switch (requestedCollectionType) {
- case EdenCollection:
- return false;
- case FullCollection:
- return true;
- case AnyCollection:
+ if (!scope)
return m_shouldDoFullCollection;
- default:
- RELEASE_ASSERT_NOT_REACHED();
- return false;
- }
- RELEASE_ASSERT_NOT_REACHED();
- return false;
+ return *scope == CollectionScope::Full;
}
void Heap::addLogicallyEmptyWeakBlock(WeakBlock* block)
diff --git a/Source/JavaScriptCore/heap/Heap.h b/Source/JavaScriptCore/heap/Heap.h
index 85b3525..0a490de 100644
--- a/Source/JavaScriptCore/heap/Heap.h
+++ b/Source/JavaScriptCore/heap/Heap.h
@@ -22,17 +22,18 @@
#pragma once
#include "ArrayBuffer.h"
+#include "CollectionScope.h"
#include "GCIncomingRefCountedSet.h"
#include "HandleSet.h"
#include "HandleStack.h"
#include "HeapObserver.h"
-#include "HeapOperation.h"
#include "ListableHandler.h"
#include "MachineStackMarker.h"
#include "MarkedAllocator.h"
#include "MarkedBlock.h"
#include "MarkedBlockSet.h"
#include "MarkedSpace.h"
+#include "MutatorState.h"
#include "Options.h"
#include "SlotVisitor.h"
#include "StructureIDTable.h"
@@ -48,7 +49,6 @@
namespace JSC {
-class AllocationScope;
class CodeBlock;
class CodeBlockSet;
class GCDeferralContext;
@@ -61,6 +61,7 @@
class HeapProfiler;
class HeapRootVisitor;
class HeapVerifier;
+class HelpingGCScope;
class IncrementalSweeper;
class JITStubRoutine;
class JITStubRoutineSet;
@@ -139,11 +140,13 @@
void addObserver(HeapObserver* observer) { m_observers.append(observer); }
void removeObserver(HeapObserver* observer) { m_observers.removeFirst(observer); }
- // true if collection is in progress
- bool isCollecting();
- HeapOperation operationInProgress() { return m_operationInProgress; }
- // true if an allocation or collection is in progress
- bool isBusy();
+ MutatorState mutatorState() const { return m_mutatorState; }
+ Optional<CollectionScope> collectionScope() const { return m_collectionScope; }
+
+ // We're always busy on the collection threads. On the main thread, this returns true if we're
+ // helping heap.
+ bool isCurrentThreadBusy();
+
MarkedSpace::Subspace& subspaceForObjectWithoutDestructor() { return m_objectSpace.subspaceForObjectsWithoutDestructor(); }
MarkedSpace::Subspace& subspaceForObjectDestructor() { return m_objectSpace.subspaceForObjectsWithDestructor(); }
MarkedSpace::Subspace& subspaceForAuxiliaryData() { return m_objectSpace.subspaceForAuxiliaryData(); }
@@ -171,7 +174,7 @@
JS_EXPORT_PRIVATE void collectAllGarbage();
bool shouldCollect();
- JS_EXPORT_PRIVATE void collect(HeapOperation collectionType = AnyCollection);
+ JS_EXPORT_PRIVATE void collect(Optional<CollectionScope> = Nullopt);
bool collectIfNecessaryOrDefer(GCDeferralContext* = nullptr); // Returns true if it did collect.
void collectAccordingToDeferGCProbability();
@@ -268,7 +271,7 @@
const unsigned* addressOfBarrierThreshold() const { return &m_barrierThreshold; }
private:
- friend class AllocationScope;
+ friend class AllocatingScope;
friend class CodeBlock;
friend class DeferGC;
friend class DeferGCForAWhile;
@@ -278,6 +281,7 @@
friend class HandleSet;
friend class HeapUtil;
friend class HeapVerifier;
+ friend class HelpingGCScope;
friend class JITStubRoutine;
friend class LLIntOffsetsExtractor;
friend class MarkedSpace;
@@ -293,7 +297,7 @@
template<typename T> friend void* allocateCell(Heap&, GCDeferralContext*);
template<typename T> friend void* allocateCell(Heap&, GCDeferralContext*, size_t);
- void collectWithoutAnySweep(HeapOperation collectionType = AnyCollection);
+ void collectWithoutAnySweep(Optional<CollectionScope> = Nullopt);
void* allocateWithDestructor(size_t); // For use with objects with destructors.
void* allocateWithoutDestructor(size_t); // For use with objects without destructors.
@@ -312,10 +316,10 @@
JS_EXPORT_PRIVATE void reportExtraMemoryAllocatedSlowCase(size_t);
JS_EXPORT_PRIVATE void deprecatedReportExtraMemorySlowCase(size_t);
- void collectImpl(HeapOperation, void* stackOrigin, void* stackTop, MachineThreads::RegisterState&);
+ void collectImpl(Optional<CollectionScope>, void* stackOrigin, void* stackTop, MachineThreads::RegisterState&);
void suspendCompilerThreads();
- void willStartCollection(HeapOperation collectionType);
+ void willStartCollection(Optional<CollectionScope>);
void flushOldStructureIDTables();
void flushWriteBarrierBuffer();
void stopAllocation();
@@ -366,7 +370,7 @@
void sweepAllLogicallyEmptyWeakBlocks();
bool sweepNextLogicallyEmptyWeakBlock();
- bool shouldDoFullCollection(HeapOperation requestedCollectionType) const;
+ bool shouldDoFullCollection(Optional<CollectionScope> requestedCollectionScope) const;
void incrementDeferralDepth();
void decrementDeferralDepth();
@@ -394,7 +398,8 @@
size_t m_totalBytesVisited;
size_t m_totalBytesVisitedThisCycle;
- HeapOperation m_operationInProgress;
+ Optional<CollectionScope> m_collectionScope;
+ MutatorState m_mutatorState { MutatorState::Running };
StructureIDTable m_structureIDTable;
MarkedSpace m_objectSpace;
GCIncomingRefCountedSet<ArrayBuffer> m_arrayBuffers;
diff --git a/Source/JavaScriptCore/heap/HeapInlines.h b/Source/JavaScriptCore/heap/HeapInlines.h
index d8c0db0..e9a8e92 100644
--- a/Source/JavaScriptCore/heap/HeapInlines.h
+++ b/Source/JavaScriptCore/heap/HeapInlines.h
@@ -45,21 +45,16 @@
return false;
if (!m_isSafeToCollect)
return false;
- if (m_operationInProgress != NoOperation)
+ if (collectionScope() || mutatorState() == MutatorState::HelpingGC)
return false;
if (Options::gcMaxHeapSize())
return m_bytesAllocatedThisCycle > Options::gcMaxHeapSize();
return m_bytesAllocatedThisCycle > m_maxEdenSize;
}
-inline bool Heap::isBusy()
+inline bool Heap::isCurrentThreadBusy()
{
- return m_operationInProgress != NoOperation;
-}
-
-inline bool Heap::isCollecting()
-{
- return m_operationInProgress == FullCollection || m_operationInProgress == EdenCollection;
+ return mayBeGCThread() || mutatorState() != MutatorState::Running;
}
ALWAYS_INLINE Heap* Heap::heap(const HeapCell* cell)
@@ -76,7 +71,7 @@
ALWAYS_INLINE bool Heap::isMarked(const void* rawCell)
{
- ASSERT(!mayBeGCThread());
+ ASSERT(mayBeGCThread() != GCThreadType::Helper);
HeapCell* cell = bitwise_cast<HeapCell*>(rawCell);
if (cell->isLargeAllocation())
return cell->largeAllocation().isMarked();
@@ -161,7 +156,7 @@
inline void Heap::reportExtraMemoryVisited(CellState oldState, size_t size)
{
// We don't want to double-count the extra memory that was reported in previous collections.
- if (operationInProgress() == EdenCollection && oldState == CellState::OldGrey)
+ if (collectionScope() == CollectionScope::Eden && oldState == CellState::OldGrey)
return;
size_t* counter = &m_extraMemorySize;
@@ -177,7 +172,7 @@
inline void Heap::reportExternalMemoryVisited(CellState oldState, size_t size)
{
// We don't want to double-count the external memory that was reported in previous collections.
- if (operationInProgress() == EdenCollection && oldState == CellState::OldGrey)
+ if (collectionScope() == CollectionScope::Eden && oldState == CellState::OldGrey)
return;
size_t* counter = &m_externalMemorySize;
@@ -372,7 +367,7 @@
inline void Heap::collectAccordingToDeferGCProbability()
{
- if (isDeferred() || !m_isSafeToCollect || m_operationInProgress != NoOperation)
+ if (isDeferred() || !m_isSafeToCollect || collectionScope() || mutatorState() == MutatorState::HelpingGC)
return;
if (randomNumber() < Options::deferGCProbability()) {
diff --git a/Source/JavaScriptCore/heap/HeapObserver.h b/Source/JavaScriptCore/heap/HeapObserver.h
index 7abe60d..ccbb7e6 100644
--- a/Source/JavaScriptCore/heap/HeapObserver.h
+++ b/Source/JavaScriptCore/heap/HeapObserver.h
@@ -25,7 +25,7 @@
#pragma once
-#include "HeapOperation.h"
+#include "CollectionScope.h"
namespace JSC {
@@ -33,7 +33,7 @@
public:
virtual ~HeapObserver() { }
virtual void willGarbageCollect() = 0;
- virtual void didGarbageCollect(HeapOperation) = 0;
+ virtual void didGarbageCollect(CollectionScope) = 0;
};
} // namespace JSC
diff --git a/Source/JavaScriptCore/heap/HeapVerifier.cpp b/Source/JavaScriptCore/heap/HeapVerifier.cpp
index 153881f..c779cf1 100644
--- a/Source/JavaScriptCore/heap/HeapVerifier.cpp
+++ b/Source/JavaScriptCore/heap/HeapVerifier.cpp
@@ -43,24 +43,6 @@
m_cycles = std::make_unique<GCCycle[]>(m_numberOfCycles);
}
-const char* HeapVerifier::collectionTypeName(HeapOperation type)
-{
- switch (type) {
- case NoOperation:
- return "NoOperation";
- case AnyCollection:
- return "AnyCollection";
- case Allocation:
- return "Allocation";
- case EdenCollection:
- return "EdenCollection";
- case FullCollection:
- return "FullCollection";
- }
- RELEASE_ASSERT_NOT_REACHED();
- return nullptr; // Silencing a compiler warning.
-}
-
const char* HeapVerifier::phaseName(HeapVerifier::Phase phase)
{
switch (phase) {
@@ -81,7 +63,7 @@
{
Heap* heap = m_heap;
incrementCycle();
- currentCycle().collectionType = heap->operationInProgress();
+ currentCycle().scope = *heap->collectionScope();
}
struct GatherLiveObjFunctor : MarkedBlock::CountFunctor {
@@ -195,7 +177,7 @@
if (objData.isConfirmedDead) {
dataLogF("FOUND dead obj %p in GC[%d] %s list '%s'\n",
- obj, cycleIndex, cycle.collectionTypeName(), list.name);
+ obj, cycleIndex, collectionScopeName(cycle.scope), list.name);
return;
}
@@ -206,7 +188,7 @@
dataLogF("FOUND obj %p type '%s' butterfly %p (base %p) in GC[%d] %s list '%s'\n",
obj, structure->classInfo()->className,
butterfly, butterflyBase,
- cycleIndex, cycle.collectionTypeName(), list.name);
+ cycleIndex, collectionScopeName(cycle.scope), list.name);
}
void HeapVerifier::checkIfRecorded(JSObject* obj)
diff --git a/Source/JavaScriptCore/heap/HeapVerifier.h b/Source/JavaScriptCore/heap/HeapVerifier.h
index 50551ec..d736b47 100644
--- a/Source/JavaScriptCore/heap/HeapVerifier.h
+++ b/Source/JavaScriptCore/heap/HeapVerifier.h
@@ -54,7 +54,6 @@
// object was in any of those lists.
JS_EXPORT_PRIVATE void checkIfRecorded(JSObject*);
- static const char* collectionTypeName(HeapOperation);
static const char* phaseName(Phase);
private:
@@ -65,14 +64,9 @@
{
}
- HeapOperation collectionType;
+ CollectionScope scope;
LiveObjectList before;
LiveObjectList after;
-
- const char* collectionTypeName() const
- {
- return HeapVerifier::collectionTypeName(collectionType);
- }
};
void incrementCycle() { m_currentCycle = (m_currentCycle + 1) % m_numberOfCycles; }
diff --git a/Source/JavaScriptCore/heap/HeapOperation.h b/Source/JavaScriptCore/heap/HelpingGCScope.h
similarity index 76%
copy from Source/JavaScriptCore/heap/HeapOperation.h
copy to Source/JavaScriptCore/heap/HelpingGCScope.h
index fbb8283..e8f4085 100644
--- a/Source/JavaScriptCore/heap/HeapOperation.h
+++ b/Source/JavaScriptCore/heap/HelpingGCScope.h
@@ -1,5 +1,5 @@
/*
- * Copyright (C) 2013 Apple Inc. All rights reserved.
+ * Copyright (C) 2016 Apple Inc. All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
@@ -25,16 +25,28 @@
#pragma once
+#include "Heap.h"
+
namespace JSC {
-enum HeapOperation { NoOperation, Allocation, FullCollection, EdenCollection, AnyCollection };
+class HelpingGCScope {
+public:
+ HelpingGCScope(Heap& heap)
+ : m_heap(heap)
+ , m_oldState(m_heap.m_mutatorState)
+ {
+ m_heap.m_mutatorState = MutatorState::HelpingGC;
+ }
+
+ ~HelpingGCScope()
+ {
+ m_heap.m_mutatorState = m_oldState;
+ }
+
+private:
+ Heap& m_heap;
+ MutatorState m_oldState;
+};
} // namespace JSC
-namespace WTF {
-
-class PrintStream;
-
-void printInternal(PrintStream& out, JSC::HeapOperation);
-
-} // namespace WTF
diff --git a/Source/JavaScriptCore/heap/LargeAllocation.cpp b/Source/JavaScriptCore/heap/LargeAllocation.cpp
index 09ca109..eb839dd 100644
--- a/Source/JavaScriptCore/heap/LargeAllocation.cpp
+++ b/Source/JavaScriptCore/heap/LargeAllocation.cpp
@@ -77,7 +77,7 @@
void LargeAllocation::flip()
{
- ASSERT(heap()->operationInProgress() == FullCollection);
+ ASSERT(heap()->collectionScope() == CollectionScope::Full);
clearMarked();
}
diff --git a/Source/JavaScriptCore/heap/MarkedAllocator.cpp b/Source/JavaScriptCore/heap/MarkedAllocator.cpp
index 72d9230..2ceadaa 100644
--- a/Source/JavaScriptCore/heap/MarkedAllocator.cpp
+++ b/Source/JavaScriptCore/heap/MarkedAllocator.cpp
@@ -26,7 +26,7 @@
#include "config.h"
#include "MarkedAllocator.h"
-#include "AllocationScope.h"
+#include "AllocatingScope.h"
#include "GCActivityCallback.h"
#include "Heap.h"
#include "IncrementalSweeper.h"
@@ -185,7 +185,6 @@
else
m_heap->collectAllGarbage();
}
- ASSERT(m_heap->m_operationInProgress == NoOperation);
}
if (++allocationCount >= Options::slowPathAllocsBetweenGCs())
allocationCount = 0;
@@ -214,10 +213,10 @@
didConsumeFreeList();
+ AllocatingScope healpingHeap(*m_heap);
+
m_heap->collectIfNecessaryOrDefer(deferralContext);
- AllocationScope allocationScope(*m_heap);
-
void* result = tryAllocateWithoutCollecting();
if (LIKELY(result != 0))
diff --git a/Source/JavaScriptCore/heap/MarkedBlock.h b/Source/JavaScriptCore/heap/MarkedBlock.h
index 8db44c466..90a02b8 100644
--- a/Source/JavaScriptCore/heap/MarkedBlock.h
+++ b/Source/JavaScriptCore/heap/MarkedBlock.h
@@ -25,7 +25,6 @@
#include "DestructionMode.h"
#include "FreeList.h"
#include "HeapCell.h"
-#include "HeapOperation.h"
#include "IterationStatus.h"
#include "WeakSet.h"
#include <wtf/Atomics.h>
diff --git a/Source/JavaScriptCore/heap/MarkedSpace.cpp b/Source/JavaScriptCore/heap/MarkedSpace.cpp
index a5b0397..b0500e3 100644
--- a/Source/JavaScriptCore/heap/MarkedSpace.cpp
+++ b/Source/JavaScriptCore/heap/MarkedSpace.cpp
@@ -338,7 +338,7 @@
m_activeWeakSets.takeFrom(m_newActiveWeakSets);
- if (m_heap->operationInProgress() == EdenCollection)
+ if (m_heap->collectionScope() == CollectionScope::Eden)
m_largeAllocationsNurseryOffsetForSweep = m_largeAllocationsNurseryOffset;
else
m_largeAllocationsNurseryOffsetForSweep = 0;
@@ -355,7 +355,7 @@
m_newActiveWeakSets.forEach(visit);
- if (m_heap->operationInProgress() == FullCollection)
+ if (m_heap->collectionScope() == CollectionScope::Full)
m_activeWeakSets.forEach(visit);
}
@@ -367,7 +367,7 @@
m_newActiveWeakSets.forEach(visit);
- if (m_heap->operationInProgress() == FullCollection)
+ if (m_heap->collectionScope() == CollectionScope::Full)
m_activeWeakSets.forEach(visit);
}
@@ -383,7 +383,7 @@
void MarkedSpace::prepareForMarking()
{
- if (m_heap->operationInProgress() == EdenCollection)
+ if (m_heap->collectionScope() == CollectionScope::Eden)
m_largeAllocationsOffsetForThisCollection = m_largeAllocationsNurseryOffset;
else
m_largeAllocationsOffsetForThisCollection = 0;
@@ -453,7 +453,7 @@
void MarkedSpace::beginMarking()
{
- if (m_heap->operationInProgress() == FullCollection) {
+ if (m_heap->collectionScope() == CollectionScope::Full) {
forEachAllocator(
[&] (MarkedAllocator& allocator) -> IterationStatus {
allocator.beginMarkingForFullCollection();
@@ -597,7 +597,7 @@
void MarkedSpace::snapshotUnswept()
{
- if (m_heap->operationInProgress() == EdenCollection) {
+ if (m_heap->collectionScope() == CollectionScope::Eden) {
forEachAllocator(
[&] (MarkedAllocator& allocator) -> IterationStatus {
allocator.snapshotUnsweptForEdenCollection();
diff --git a/Source/JavaScriptCore/heap/HeapOperation.cpp b/Source/JavaScriptCore/heap/MutatorState.cpp
similarity index 79%
rename from Source/JavaScriptCore/heap/HeapOperation.cpp
rename to Source/JavaScriptCore/heap/MutatorState.cpp
index 8715314..8368f43 100644
--- a/Source/JavaScriptCore/heap/HeapOperation.cpp
+++ b/Source/JavaScriptCore/heap/MutatorState.cpp
@@ -24,31 +24,25 @@
*/
#include "config.h"
-#include "HeapOperation.h"
+#include "MutatorState.h"
#include <wtf/PrintStream.h>
-namespace WTF {
-
using namespace JSC;
-void printInternal(PrintStream& out, HeapOperation operation)
+namespace WTF {
+
+void printInternal(PrintStream& out, MutatorState state)
{
- switch (operation) {
- case NoOperation:
- out.print("None");
+ switch (state) {
+ case MutatorState::Running:
+ out.print("Running");
return;
- case Allocation:
- out.print("Alloc");
+ case MutatorState::Allocating:
+ out.print("Allocating");
return;
- case FullCollection:
- out.print("Full");
- return;
- case EdenCollection:
- out.print("Eden");
- return;
- case AnyCollection:
- out.print("Any");
+ case MutatorState::HelpingGC:
+ out.print("HelpingGC");
return;
}
RELEASE_ASSERT_NOT_REACHED();
@@ -56,5 +50,3 @@
} // namespace WTF
-
-
diff --git a/Source/JavaScriptCore/heap/HeapOperation.h b/Source/JavaScriptCore/heap/MutatorState.h
similarity index 79%
copy from Source/JavaScriptCore/heap/HeapOperation.h
copy to Source/JavaScriptCore/heap/MutatorState.h
index fbb8283..65b61f7 100644
--- a/Source/JavaScriptCore/heap/HeapOperation.h
+++ b/Source/JavaScriptCore/heap/MutatorState.h
@@ -1,5 +1,5 @@
/*
- * Copyright (C) 2013 Apple Inc. All rights reserved.
+ * Copyright (C) 2016 Apple Inc. All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
@@ -27,7 +27,16 @@
namespace JSC {
-enum HeapOperation { NoOperation, Allocation, FullCollection, EdenCollection, AnyCollection };
+enum class MutatorState {
+ // The mutator is running when it's not inside a Heap slow path.
+ Running,
+
+ // The mutator is in an allocation slow path.
+ Allocating,
+
+ // The mutator was asked by the GC to do some work.
+ HelpingGC
+};
} // namespace JSC
@@ -35,6 +44,7 @@
class PrintStream;
-void printInternal(PrintStream& out, JSC::HeapOperation);
+void printInternal(PrintStream&, JSC::MutatorState);
} // namespace WTF
+
diff --git a/Source/JavaScriptCore/heap/SlotVisitor.cpp b/Source/JavaScriptCore/heap/SlotVisitor.cpp
index 6856d8f..a804e89 100644
--- a/Source/JavaScriptCore/heap/SlotVisitor.cpp
+++ b/Source/JavaScriptCore/heap/SlotVisitor.cpp
@@ -96,7 +96,7 @@
void SlotVisitor::didStartMarking()
{
- if (heap()->operationInProgress() == FullCollection)
+ if (heap()->collectionScope() == CollectionScope::Full)
ASSERT(m_opaqueRoots.isEmpty()); // Should have merged by now.
else
reset();
