GC should be able to remove things from the DFG worklist and cancel on-going compilations if it knows that the compilation would already be invalidated
https://bugs.webkit.org/show_bug.cgi?id=132166
Reviewed by Oliver Hunt and Mark Hahnenberg.
The GC can aid type inference by removing structures that are dead and jettisoning
code that relies on those structures. This can dramatically accelerate type inference
for some tricky programs.
Unfortunately, we previously pinned any structures that enqueued compilations depended
on. This means that if you're on a machine that only runs a single compilation thread
and where compilations are relatively slow, you have a high chance of large numbers of
structures being pinned during any GC since the compilation queue is likely to be full
of random stuff.
This comprehensively fixes this issue by allowing the GC to remove compilation plans
if the things they depend on are dead, and to even cancel safepointed compilations.
* bytecode/CodeBlock.cpp:
(JSC::CodeBlock::shouldImmediatelyAssumeLivenessDuringScan):
(JSC::CodeBlock::isKnownToBeLiveDuringGC):
(JSC::CodeBlock::finalizeUnconditionally):
* bytecode/CodeBlock.h:
(JSC::CodeBlock::shouldImmediatelyAssumeLivenessDuringScan): Deleted.
* dfg/DFGDesiredIdentifiers.cpp:
(JSC::DFG::DesiredIdentifiers::DesiredIdentifiers):
* dfg/DFGDesiredIdentifiers.h:
* dfg/DFGDesiredWatchpoints.h:
* dfg/DFGDesiredWeakReferences.cpp:
(JSC::DFG::DesiredWeakReferences::DesiredWeakReferences):
* dfg/DFGDesiredWeakReferences.h:
* dfg/DFGGraphSafepoint.cpp:
(JSC::DFG::GraphSafepoint::GraphSafepoint):
* dfg/DFGGraphSafepoint.h:
* dfg/DFGPlan.cpp:
(JSC::DFG::Plan::Plan):
(JSC::DFG::Plan::compileInThread):
(JSC::DFG::Plan::compileInThreadImpl):
(JSC::DFG::Plan::notifyCompiling):
(JSC::DFG::Plan::notifyCompiled):
(JSC::DFG::Plan::notifyReady):
(JSC::DFG::Plan::checkLivenessAndVisitChildren):
(JSC::DFG::Plan::isKnownToBeLiveDuringGC):
(JSC::DFG::Plan::cancel):
(JSC::DFG::Plan::visitChildren): Deleted.
* dfg/DFGPlan.h:
* dfg/DFGSafepoint.cpp:
(JSC::DFG::Safepoint::Result::~Result):
(JSC::DFG::Safepoint::Result::didGetCancelled):
(JSC::DFG::Safepoint::Safepoint):
(JSC::DFG::Safepoint::~Safepoint):
(JSC::DFG::Safepoint::checkLivenessAndVisitChildren):
(JSC::DFG::Safepoint::isKnownToBeLiveDuringGC):
(JSC::DFG::Safepoint::cancel):
(JSC::DFG::Safepoint::visitChildren): Deleted.
* dfg/DFGSafepoint.h:
(JSC::DFG::Safepoint::Result::Result):
* dfg/DFGWorklist.cpp:
(JSC::DFG::Worklist::compilationState):
(JSC::DFG::Worklist::waitUntilAllPlansForVMAreReady):
(JSC::DFG::Worklist::removeAllReadyPlansForVM):
(JSC::DFG::Worklist::completeAllReadyPlansForVM):
(JSC::DFG::Worklist::visitWeakReferences):
(JSC::DFG::Worklist::removeDeadPlans):
(JSC::DFG::Worklist::runThread):
(JSC::DFG::Worklist::visitChildren): Deleted.
* dfg/DFGWorklist.h:
* ftl/FTLCompile.cpp:
(JSC::FTL::compile):
* ftl/FTLCompile.h:
* heap/CodeBlockSet.cpp:
(JSC::CodeBlockSet::rememberCurrentlyExecutingCodeBlocks):
* heap/Heap.cpp:
(JSC::Heap::markRoots):
(JSC::Heap::visitCompilerWorklistWeakReferences):
(JSC::Heap::removeDeadCompilerWorklistEntries):
(JSC::Heap::visitWeakHandles):
(JSC::Heap::collect):
(JSC::Heap::visitCompilerWorklists): Deleted.
* heap/Heap.h:
git-svn-id: http://svn.webkit.org/repository/webkit/trunk@167897 268f45cc-cd09-0410-ab3c-d52691b4dbfc
diff --git a/Source/JavaScriptCore/ChangeLog b/Source/JavaScriptCore/ChangeLog
index 508bb68..6e3d1ea 100644
--- a/Source/JavaScriptCore/ChangeLog
+++ b/Source/JavaScriptCore/ChangeLog
@@ -1,3 +1,86 @@
+2014-04-27 Filip Pizlo <fpizlo@apple.com>
+
+ GC should be able to remove things from the DFG worklist and cancel on-going compilations if it knows that the compilation would already be invalidated
+ https://bugs.webkit.org/show_bug.cgi?id=132166
+
+ Reviewed by Oliver Hunt and Mark Hahnenberg.
+
+ The GC can aid type inference by removing structures that are dead and jettisoning
+ code that relies on those structures. This can dramatically accelerate type inference
+ for some tricky programs.
+
+ Unfortunately, we previously pinned any structures that enqueued compilations depended
+ on. This means that if you're on a machine that only runs a single compilation thread
+ and where compilations are relatively slow, you have a high chance of large numbers of
+ structures being pinned during any GC since the compilation queue is likely to be full
+ of random stuff.
+
+ This comprehensively fixes this issue by allowing the GC to remove compilation plans
+ if the things they depend on are dead, and to even cancel safepointed compilations.
+
+ * bytecode/CodeBlock.cpp:
+ (JSC::CodeBlock::shouldImmediatelyAssumeLivenessDuringScan):
+ (JSC::CodeBlock::isKnownToBeLiveDuringGC):
+ (JSC::CodeBlock::finalizeUnconditionally):
+ * bytecode/CodeBlock.h:
+ (JSC::CodeBlock::shouldImmediatelyAssumeLivenessDuringScan): Deleted.
+ * dfg/DFGDesiredIdentifiers.cpp:
+ (JSC::DFG::DesiredIdentifiers::DesiredIdentifiers):
+ * dfg/DFGDesiredIdentifiers.h:
+ * dfg/DFGDesiredWatchpoints.h:
+ * dfg/DFGDesiredWeakReferences.cpp:
+ (JSC::DFG::DesiredWeakReferences::DesiredWeakReferences):
+ * dfg/DFGDesiredWeakReferences.h:
+ * dfg/DFGGraphSafepoint.cpp:
+ (JSC::DFG::GraphSafepoint::GraphSafepoint):
+ * dfg/DFGGraphSafepoint.h:
+ * dfg/DFGPlan.cpp:
+ (JSC::DFG::Plan::Plan):
+ (JSC::DFG::Plan::compileInThread):
+ (JSC::DFG::Plan::compileInThreadImpl):
+ (JSC::DFG::Plan::notifyCompiling):
+ (JSC::DFG::Plan::notifyCompiled):
+ (JSC::DFG::Plan::notifyReady):
+ (JSC::DFG::Plan::checkLivenessAndVisitChildren):
+ (JSC::DFG::Plan::isKnownToBeLiveDuringGC):
+ (JSC::DFG::Plan::cancel):
+ (JSC::DFG::Plan::visitChildren): Deleted.
+ * dfg/DFGPlan.h:
+ * dfg/DFGSafepoint.cpp:
+ (JSC::DFG::Safepoint::Result::~Result):
+ (JSC::DFG::Safepoint::Result::didGetCancelled):
+ (JSC::DFG::Safepoint::Safepoint):
+ (JSC::DFG::Safepoint::~Safepoint):
+ (JSC::DFG::Safepoint::checkLivenessAndVisitChildren):
+ (JSC::DFG::Safepoint::isKnownToBeLiveDuringGC):
+ (JSC::DFG::Safepoint::cancel):
+ (JSC::DFG::Safepoint::visitChildren): Deleted.
+ * dfg/DFGSafepoint.h:
+ (JSC::DFG::Safepoint::Result::Result):
+ * dfg/DFGWorklist.cpp:
+ (JSC::DFG::Worklist::compilationState):
+ (JSC::DFG::Worklist::waitUntilAllPlansForVMAreReady):
+ (JSC::DFG::Worklist::removeAllReadyPlansForVM):
+ (JSC::DFG::Worklist::completeAllReadyPlansForVM):
+ (JSC::DFG::Worklist::visitWeakReferences):
+ (JSC::DFG::Worklist::removeDeadPlans):
+ (JSC::DFG::Worklist::runThread):
+ (JSC::DFG::Worklist::visitChildren): Deleted.
+ * dfg/DFGWorklist.h:
+ * ftl/FTLCompile.cpp:
+ (JSC::FTL::compile):
+ * ftl/FTLCompile.h:
+ * heap/CodeBlockSet.cpp:
+ (JSC::CodeBlockSet::rememberCurrentlyExecutingCodeBlocks):
+ * heap/Heap.cpp:
+ (JSC::Heap::markRoots):
+ (JSC::Heap::visitCompilerWorklistWeakReferences):
+ (JSC::Heap::removeDeadCompilerWorklistEntries):
+ (JSC::Heap::visitWeakHandles):
+ (JSC::Heap::collect):
+ (JSC::Heap::visitCompilerWorklists): Deleted.
+ * heap/Heap.h:
+
2014-04-28 Mark Hahnenberg <mhahnenberg@apple.com>
Deleting properties poisons objects
diff --git a/Source/JavaScriptCore/bytecode/CodeBlock.cpp b/Source/JavaScriptCore/bytecode/CodeBlock.cpp
index b7a7c9c..99fc515 100644
--- a/Source/JavaScriptCore/bytecode/CodeBlock.cpp
+++ b/Source/JavaScriptCore/bytecode/CodeBlock.cpp
@@ -1972,6 +1972,49 @@
#endif // ENABLE(DFG_JIT)
}
+bool CodeBlock::shouldImmediatelyAssumeLivenessDuringScan()
+{
+#if ENABLE(DFG_JIT)
+ // Interpreter and Baseline JIT CodeBlocks don't need to be jettisoned when
+ // their weak references go stale. So if a basline JIT CodeBlock gets
+ // scanned, we can assume that this means that it's live.
+ if (!JITCode::isOptimizingJIT(jitType()))
+ return true;
+
+ // For simplicity, we don't attempt to jettison code blocks during GC if
+ // they are executing. Instead we strongly mark their weak references to
+ // allow them to continue to execute soundly.
+ if (m_mayBeExecuting)
+ return true;
+
+ if (Options::forceDFGCodeBlockLiveness())
+ return true;
+
+ return false;
+#else
+ return true;
+#endif
+}
+
+bool CodeBlock::isKnownToBeLiveDuringGC()
+{
+#if ENABLE(DFG_JIT)
+ // This should return true for:
+ // - Code blocks that behave like normal objects - i.e. if they are referenced then they
+ // are live.
+ // - Code blocks that were running on the stack.
+ // - Code blocks that survived the last GC if the current GC is an Eden GC. This is
+ // because either livenessHasBeenProved would have survived as true or m_mayBeExecuting
+ // would survive as true.
+ // - Code blocks that don't have any dead weak references.
+
+ return shouldImmediatelyAssumeLivenessDuringScan()
+ || m_jitCode->dfgCommon()->livenessHasBeenProved;
+#else
+ return true;
+#endif
+}
+
void CodeBlock::propagateTransitions(SlotVisitor& visitor)
{
UNUSED_PARAM(visitor);
@@ -2215,7 +2258,7 @@
#if ENABLE(DFG_JIT)
// Check if we're not live. If we are, then jettison.
- if (!(shouldImmediatelyAssumeLivenessDuringScan() || m_jitCode->dfgCommon()->livenessHasBeenProved)) {
+ if (!isKnownToBeLiveDuringGC()) {
if (Options::verboseOSR())
dataLog(*this, " has dead weak references, jettisoning during GC.\n");
diff --git a/Source/JavaScriptCore/bytecode/CodeBlock.h b/Source/JavaScriptCore/bytecode/CodeBlock.h
index 765fa0b0..29fda45 100644
--- a/Source/JavaScriptCore/bytecode/CodeBlock.h
+++ b/Source/JavaScriptCore/bytecode/CodeBlock.h
@@ -947,6 +947,8 @@
void beginValidationDidFail();
NO_RETURN_DUE_TO_CRASH void endValidationDidFail();
+ bool isKnownToBeLiveDuringGC(); // Will only return valid results when called during GC. Assumes that you've already established that the owner executable is live.
+
protected:
virtual void visitWeakReferences(SlotVisitor&) override;
virtual void finalizeUnconditionally() override;
@@ -1001,29 +1003,7 @@
void dumpArrayProfiling(PrintStream&, const Instruction*&, bool& hasPrintedProfiling);
void dumpRareCaseProfile(PrintStream&, const char* name, RareCaseProfile*, bool& hasPrintedProfiling);
-#if ENABLE(DFG_JIT)
- bool shouldImmediatelyAssumeLivenessDuringScan()
- {
- // Interpreter and Baseline JIT CodeBlocks don't need to be jettisoned when
- // their weak references go stale. So if a basline JIT CodeBlock gets
- // scanned, we can assume that this means that it's live.
- if (!JITCode::isOptimizingJIT(jitType()))
- return true;
-
- // For simplicity, we don't attempt to jettison code blocks during GC if
- // they are executing. Instead we strongly mark their weak references to
- // allow them to continue to execute soundly.
- if (m_mayBeExecuting)
- return true;
-
- if (Options::forceDFGCodeBlockLiveness())
- return true;
-
- return false;
- }
-#else
- bool shouldImmediatelyAssumeLivenessDuringScan() { return true; }
-#endif
+ bool shouldImmediatelyAssumeLivenessDuringScan();
void propagateTransitions(SlotVisitor&);
void determineLiveness(SlotVisitor&);
diff --git a/Source/JavaScriptCore/dfg/DFGDesiredIdentifiers.cpp b/Source/JavaScriptCore/dfg/DFGDesiredIdentifiers.cpp
index 60aa6b6..dfa8faa 100644
--- a/Source/JavaScriptCore/dfg/DFGDesiredIdentifiers.cpp
+++ b/Source/JavaScriptCore/dfg/DFGDesiredIdentifiers.cpp
@@ -1,5 +1,5 @@
/*
- * Copyright (C) 2013 Apple Inc. All rights reserved.
+ * Copyright (C) 2013, 2014 Apple Inc. All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
@@ -33,6 +33,11 @@
namespace JSC { namespace DFG {
+DesiredIdentifiers::DesiredIdentifiers()
+ : m_codeBlock(nullptr)
+{
+}
+
DesiredIdentifiers::DesiredIdentifiers(CodeBlock* codeBlock)
: m_codeBlock(codeBlock)
{
diff --git a/Source/JavaScriptCore/dfg/DFGDesiredIdentifiers.h b/Source/JavaScriptCore/dfg/DFGDesiredIdentifiers.h
index 53fba60..10c2159 100644
--- a/Source/JavaScriptCore/dfg/DFGDesiredIdentifiers.h
+++ b/Source/JavaScriptCore/dfg/DFGDesiredIdentifiers.h
@@ -1,5 +1,5 @@
/*
- * Copyright (C) 2013 Apple Inc. All rights reserved.
+ * Copyright (C) 2013, 2014 Apple Inc. All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
@@ -40,6 +40,7 @@
class DesiredIdentifiers {
public:
+ DesiredIdentifiers();
DesiredIdentifiers(CodeBlock*);
~DesiredIdentifiers();
diff --git a/Source/JavaScriptCore/dfg/DFGDesiredWatchpoints.h b/Source/JavaScriptCore/dfg/DFGDesiredWatchpoints.h
index cc1b733..3e07f9b 100644
--- a/Source/JavaScriptCore/dfg/DFGDesiredWatchpoints.h
+++ b/Source/JavaScriptCore/dfg/DFGDesiredWatchpoints.h
@@ -81,7 +81,6 @@
template<typename WatchpointSetType, typename Adaptor = GenericSetAdaptor<WatchpointSetType>>
class GenericDesiredWatchpoints {
- WTF_MAKE_NONCOPYABLE(GenericDesiredWatchpoints);
#if !ASSERT_DISABLED
typedef HashMap<WatchpointSetType*, bool> StateMap;
#endif
diff --git a/Source/JavaScriptCore/dfg/DFGDesiredWeakReferences.cpp b/Source/JavaScriptCore/dfg/DFGDesiredWeakReferences.cpp
index 22cb3e4..1ee02ba 100644
--- a/Source/JavaScriptCore/dfg/DFGDesiredWeakReferences.cpp
+++ b/Source/JavaScriptCore/dfg/DFGDesiredWeakReferences.cpp
@@ -34,6 +34,11 @@
namespace JSC { namespace DFG {
+DesiredWeakReferences::DesiredWeakReferences()
+ : m_codeBlock(nullptr)
+{
+}
+
DesiredWeakReferences::DesiredWeakReferences(CodeBlock* codeBlock)
: m_codeBlock(codeBlock)
{
diff --git a/Source/JavaScriptCore/dfg/DFGDesiredWeakReferences.h b/Source/JavaScriptCore/dfg/DFGDesiredWeakReferences.h
index 7b3198c..48ad44c 100644
--- a/Source/JavaScriptCore/dfg/DFGDesiredWeakReferences.h
+++ b/Source/JavaScriptCore/dfg/DFGDesiredWeakReferences.h
@@ -43,6 +43,7 @@
class DesiredWeakReferences {
public:
+ DesiredWeakReferences();
DesiredWeakReferences(CodeBlock*);
~DesiredWeakReferences();
diff --git a/Source/JavaScriptCore/dfg/DFGGraphSafepoint.cpp b/Source/JavaScriptCore/dfg/DFGGraphSafepoint.cpp
index 5233cf6..e021e99 100644
--- a/Source/JavaScriptCore/dfg/DFGGraphSafepoint.cpp
+++ b/Source/JavaScriptCore/dfg/DFGGraphSafepoint.cpp
@@ -33,8 +33,8 @@
namespace JSC { namespace DFG {
-GraphSafepoint::GraphSafepoint(Graph& graph)
- : m_safepoint(graph.m_plan)
+GraphSafepoint::GraphSafepoint(Graph& graph, Safepoint::Result& result)
+ : m_safepoint(graph.m_plan, result)
{
m_safepoint.add(&graph);
m_safepoint.begin();
diff --git a/Source/JavaScriptCore/dfg/DFGGraphSafepoint.h b/Source/JavaScriptCore/dfg/DFGGraphSafepoint.h
index 1dabe3d..1759b6e 100644
--- a/Source/JavaScriptCore/dfg/DFGGraphSafepoint.h
+++ b/Source/JavaScriptCore/dfg/DFGGraphSafepoint.h
@@ -36,7 +36,7 @@
class GraphSafepoint {
public:
- GraphSafepoint(Graph&);
+ GraphSafepoint(Graph&, Safepoint::Result&);
~GraphSafepoint();
private:
diff --git a/Source/JavaScriptCore/dfg/DFGPlan.cpp b/Source/JavaScriptCore/dfg/DFGPlan.cpp
index 5f429e4..e58c8f6 100644
--- a/Source/JavaScriptCore/dfg/DFGPlan.cpp
+++ b/Source/JavaScriptCore/dfg/DFGPlan.cpp
@@ -124,7 +124,7 @@
, identifiers(codeBlock.get())
, weakReferences(codeBlock.get())
, willTryToTierUp(false)
- , isCompiled(false)
+ , stage(Preparing)
{
}
@@ -143,8 +143,11 @@
this->threadData = threadData;
double before = 0;
- if (reportCompileTimes())
+ CString codeBlockName;
+ if (reportCompileTimes()) {
before = currentTimeMS();
+ codeBlockName = toCString(*codeBlock);
+ }
SamplingRegion samplingRegion("DFG Compilation (Plan)");
CompilationScope compilationScope;
@@ -154,7 +157,8 @@
CompilationPath path = compileInThreadImpl(longLivedState);
- RELEASE_ASSERT(finalizer);
+ RELEASE_ASSERT(path == CancelPath || finalizer);
+ RELEASE_ASSERT((path == CancelPath) == (stage == Cancelled));
if (reportCompileTimes()) {
const char* pathName;
@@ -168,13 +172,16 @@
case FTLPath:
pathName = "FTL";
break;
+ case CancelPath:
+ pathName = "Cancelled";
+ break;
default:
RELEASE_ASSERT_NOT_REACHED();
pathName = "";
break;
}
double now = currentTimeMS();
- dataLog("Optimized ", *codeBlock, " using ", mode, " with ", pathName, " into ", finalizer->codeSize(), " bytes in ", now - before, " ms");
+ dataLog("Optimized ", codeBlockName, " using ", mode, " with ", pathName, " into ", finalizer ? finalizer->codeSize() : 0, " bytes in ", now - before, " ms");
if (path == FTLPath)
dataLog(" (DFG: ", beforeFTL - before, ", LLVM: ", now - beforeFTL, ")");
dataLog(".\n");
@@ -322,10 +329,13 @@
dumpAndVerifyGraph(dfg, "Graph just before FTL lowering:");
bool haveLLVM;
+ Safepoint::Result safepointResult;
{
- GraphSafepoint safepoint(dfg);
+ GraphSafepoint safepoint(dfg, safepointResult);
haveLLVM = initializeLLVM();
}
+ if (safepointResult.didGetCancelled())
+ return CancelPath;
if (!haveLLVM) {
finalizer = adoptPtr(new FailedFinalizer(*this));
@@ -343,8 +353,10 @@
return FTLPath;
}
- FTL::compile(state);
-
+ FTL::compile(state, safepointResult);
+ if (safepointResult.didGetCancelled())
+ return CancelPath;
+
if (Options::llvmAlwaysFailsBeforeLink()) {
FTL::fail(state);
return FTLPath;
@@ -384,10 +396,20 @@
writeBarriers.trigger(vm);
}
+void Plan::notifyCompiling()
+{
+ stage = Compiling;
+}
+
+void Plan::notifyCompiled()
+{
+ stage = Compiled;
+}
+
void Plan::notifyReady()
{
callback->compilationDidBecomeReadyAsynchronously(codeBlock.get());
- isCompiled = true;
+ stage = Ready;
}
CompilationResult Plan::finalizeWithoutNotifyingCallback()
@@ -419,8 +441,11 @@
return CompilationKey(codeBlock->alternative(), mode);
}
-void Plan::visitChildren(SlotVisitor& visitor, CodeBlockSet& codeBlocks)
+void Plan::checkLivenessAndVisitChildren(SlotVisitor& visitor, CodeBlockSet& codeBlocks)
{
+ if (!isKnownToBeLiveDuringGC())
+ return;
+
for (unsigned i = mustHandleValues.size(); i--;)
visitor.appendUnbarrieredValue(&mustHandleValues[i]);
@@ -434,6 +459,37 @@
transitions.visitChildren(visitor);
}
+bool Plan::isKnownToBeLiveDuringGC()
+{
+ if (stage == Cancelled)
+ return false;
+ if (!Heap::isMarked(codeBlock->ownerExecutable()))
+ return false;
+ if (!codeBlock->alternative()->isKnownToBeLiveDuringGC())
+ return false;
+ if (!!profiledDFGCodeBlock && !profiledDFGCodeBlock->isKnownToBeLiveDuringGC())
+ return false;
+ return true;
+}
+
+void Plan::cancel()
+{
+ codeBlock = nullptr;
+ profiledDFGCodeBlock = nullptr;
+ mustHandleValues.clear();
+ compilation = nullptr;
+ finalizer.clear();
+ inlineCallFrames = nullptr;
+ watchpoints = DesiredWatchpoints();
+ identifiers = DesiredIdentifiers();
+ chains = DesiredStructureChains();
+ weakReferences = DesiredWeakReferences();
+ writeBarriers = DesiredWriteBarriers();
+ transitions = DesiredTransitions();
+ callback = nullptr;
+ stage = Cancelled;
+}
+
} } // namespace JSC::DFG
#endif // ENABLE(DFG_JIT)
diff --git a/Source/JavaScriptCore/dfg/DFGPlan.h b/Source/JavaScriptCore/dfg/DFGPlan.h
index cbf228e..a2fdced 100644
--- a/Source/JavaScriptCore/dfg/DFGPlan.h
+++ b/Source/JavaScriptCore/dfg/DFGPlan.h
@@ -60,17 +60,21 @@
CompilationMode, unsigned osrEntryBytecodeIndex,
const Operands<JSValue>& mustHandleValues);
~Plan();
-
+
void compileInThread(LongLivedState&, ThreadData*);
CompilationResult finalizeWithoutNotifyingCallback();
void finalizeAndNotifyCallback();
+ void notifyCompiling();
+ void notifyCompiled();
void notifyReady();
CompilationKey key();
- void visitChildren(SlotVisitor&, CodeBlockSet&);
+ void checkLivenessAndVisitChildren(SlotVisitor&, CodeBlockSet&);
+ bool isKnownToBeLiveDuringGC();
+ void cancel();
VM& vm;
RefPtr<CodeBlock> codeBlock;
@@ -96,15 +100,16 @@
bool willTryToTierUp;
double beforeFTL;
-
- bool isCompiled;
+
+ enum Stage { Preparing, Compiling, Compiled, Ready, Cancelled };
+ Stage stage;
RefPtr<DeferredCompilationCallback> callback;
private:
bool reportCompileTimes() const;
- enum CompilationPath { FailPath, DFGPath, FTLPath };
+ enum CompilationPath { FailPath, DFGPath, FTLPath, CancelPath };
CompilationPath compileInThreadImpl(LongLivedState&);
bool isStillValid();
diff --git a/Source/JavaScriptCore/dfg/DFGSafepoint.cpp b/Source/JavaScriptCore/dfg/DFGSafepoint.cpp
index fb0c69d..7ada849 100644
--- a/Source/JavaScriptCore/dfg/DFGSafepoint.cpp
+++ b/Source/JavaScriptCore/dfg/DFGSafepoint.cpp
@@ -35,10 +35,25 @@
namespace JSC { namespace DFG {
-Safepoint::Safepoint(Plan& plan)
+Safepoint::Result::~Result()
+{
+ RELEASE_ASSERT(m_wasChecked);
+}
+
+bool Safepoint::Result::didGetCancelled()
+{
+ m_wasChecked = true;
+ return m_didGetCancelled;
+}
+
+Safepoint::Safepoint(Plan& plan, Result& result)
: m_plan(plan)
, m_didCallBegin(false)
+ , m_result(result)
{
+ RELEASE_ASSERT(result.m_wasChecked);
+ result.m_wasChecked = false;
+ result.m_didGetCancelled = false;
}
Safepoint::~Safepoint()
@@ -46,8 +61,8 @@
RELEASE_ASSERT(m_didCallBegin);
if (ThreadData* data = m_plan.threadData) {
RELEASE_ASSERT(data->m_safepoint == this);
- data->m_safepoint = nullptr;
data->m_rightToRun.lock();
+ data->m_safepoint = nullptr;
}
}
@@ -68,13 +83,39 @@
m_didCallBegin = true;
}
-void Safepoint::visitChildren(SlotVisitor& visitor)
+void Safepoint::checkLivenessAndVisitChildren(SlotVisitor& visitor)
{
RELEASE_ASSERT(m_didCallBegin);
+
+ if (m_result.m_didGetCancelled)
+ return; // We were cancelled during a previous GC!
+
+ if (!isKnownToBeLiveDuringGC())
+ return;
+
for (unsigned i = m_scannables.size(); i--;)
m_scannables[i]->visitChildren(visitor);
}
+bool Safepoint::isKnownToBeLiveDuringGC()
+{
+ RELEASE_ASSERT(m_didCallBegin);
+
+ if (m_result.m_didGetCancelled)
+ return true; // We were cancelled during a previous GC, so let's not mess with it this time around - pretend it's live and move on.
+
+ return m_plan.isKnownToBeLiveDuringGC();
+}
+
+void Safepoint::cancel()
+{
+ RELEASE_ASSERT(m_didCallBegin);
+ RELEASE_ASSERT(!m_result.m_didGetCancelled); // We cannot get cancelled twice because subsequent GCs will think that we're alive and they will not do anything to us.
+
+ m_plan.cancel();
+ m_result.m_didGetCancelled = true;
+}
+
VM& Safepoint::vm() const
{
return m_plan.vm;
diff --git a/Source/JavaScriptCore/dfg/DFGSafepoint.h b/Source/JavaScriptCore/dfg/DFGSafepoint.h
index 17a1307..96f4b8e 100644
--- a/Source/JavaScriptCore/dfg/DFGSafepoint.h
+++ b/Source/JavaScriptCore/dfg/DFGSafepoint.h
@@ -42,14 +42,35 @@
class Safepoint {
public:
- Safepoint(Plan&);
+ class Result {
+ public:
+ Result()
+ : m_didGetCancelled(false)
+ , m_wasChecked(true)
+ {
+ }
+
+ ~Result();
+
+ bool didGetCancelled();
+
+ private:
+ friend class Safepoint;
+
+ bool m_didGetCancelled;
+ bool m_wasChecked;
+ };
+
+ Safepoint(Plan&, Result&);
~Safepoint();
void add(Scannable*);
void begin();
- void visitChildren(SlotVisitor&);
+ void checkLivenessAndVisitChildren(SlotVisitor&);
+ bool isKnownToBeLiveDuringGC();
+ void cancel();
VM& vm() const;
@@ -57,6 +78,7 @@
Plan& m_plan;
Vector<Scannable*> m_scannables;
bool m_didCallBegin;
+ Result& m_result;
};
} } // namespace JSC::DFG
diff --git a/Source/JavaScriptCore/dfg/DFGWorklist.cpp b/Source/JavaScriptCore/dfg/DFGWorklist.cpp
index 336293f..81676fa 100644
--- a/Source/JavaScriptCore/dfg/DFGWorklist.cpp
+++ b/Source/JavaScriptCore/dfg/DFGWorklist.cpp
@@ -105,7 +105,7 @@
PlanMap::iterator iter = m_plans.find(key);
if (iter == m_plans.end())
return NotKnown;
- return iter->value->isCompiled ? Compiled : Compiling;
+ return iter->value->stage == Plan::Ready ? Compiled : Compiling;
}
void Worklist::waitUntilAllPlansForVMAreReady(VM& vm)
@@ -130,7 +130,7 @@
for (PlanMap::iterator iter = m_plans.begin(); iter != end; ++iter) {
if (&iter->value->vm != &vm)
continue;
- if (!iter->value->isCompiled) {
+ if (iter->value->stage != Plan::Ready) {
allAreCompiled = false;
break;
}
@@ -151,7 +151,7 @@
RefPtr<Plan> plan = m_readyPlans[i];
if (&plan->vm != &vm)
continue;
- if (!plan->isCompiled)
+ if (plan->stage != Plan::Ready)
continue;
myReadyPlans.append(plan);
m_readyPlans[i--] = m_readyPlans.last();
@@ -182,7 +182,7 @@
if (Options::verboseCompilationQueue())
dataLog(*this, ": Completing ", currentKey, "\n");
- RELEASE_ASSERT(plan->isCompiled);
+ RELEASE_ASSERT(plan->stage == Plan::Ready);
plan->finalizeAndNotifyCallback();
@@ -220,7 +220,7 @@
m_suspensionLock.unlock();
}
-void Worklist::visitChildren(SlotVisitor& visitor, CodeBlockSet& codeBlocks)
+void Worklist::visitWeakReferences(SlotVisitor& visitor, CodeBlockSet& codeBlocks)
{
VM* vm = visitor.heap()->vm();
{
@@ -229,11 +229,10 @@
Plan* plan = iter->value.get();
if (&plan->vm != vm)
continue;
- iter->value->visitChildren(visitor, codeBlocks);
+ iter->value->checkLivenessAndVisitChildren(visitor, codeBlocks);
}
}
-
- // This loop doesn't need further locking because:
+ // This loop doesn't need locking because:
// (1) no new threads can be added to m_threads. Hence, it is immutable and needs no locks.
// (2) ThreadData::m_safepoint is protected by that thread's m_rightToRun which we must be
// holding here because of a prior call to suspendAllThreads().
@@ -241,7 +240,55 @@
ThreadData* data = m_threads[i].get();
Safepoint* safepoint = data->m_safepoint;
if (safepoint && &safepoint->vm() == vm)
- safepoint->visitChildren(visitor);
+ safepoint->checkLivenessAndVisitChildren(visitor);
+ }
+}
+
+void Worklist::removeDeadPlans(VM& vm)
+{
+ {
+ MutexLocker locker(m_lock);
+ HashSet<CompilationKey> deadPlanKeys;
+ for (PlanMap::iterator iter = m_plans.begin(); iter != m_plans.end(); ++iter) {
+ Plan* plan = iter->value.get();
+ if (&plan->vm != &vm)
+ continue;
+ if (plan->isKnownToBeLiveDuringGC())
+ continue;
+ RELEASE_ASSERT(plan->stage != Plan::Cancelled); // Should not be cancelled, yet.
+ ASSERT(!deadPlanKeys.contains(plan->key()));
+ deadPlanKeys.add(plan->key());
+ }
+ if (!deadPlanKeys.isEmpty()) {
+ for (HashSet<CompilationKey>::iterator iter = deadPlanKeys.begin(); iter != deadPlanKeys.end(); ++iter)
+ m_plans.take(*iter)->cancel();
+ Deque<RefPtr<Plan>> newQueue;
+ while (!m_queue.isEmpty()) {
+ RefPtr<Plan> plan = m_queue.takeFirst();
+ if (plan->stage != Plan::Cancelled)
+ newQueue.append(plan);
+ }
+ m_queue.swap(newQueue);
+ for (unsigned i = 0; i < m_readyPlans.size(); ++i) {
+ if (m_readyPlans[i]->stage != Plan::Cancelled)
+ continue;
+ m_readyPlans[i] = m_readyPlans.last();
+ m_readyPlans.removeLast();
+ }
+ }
+ }
+
+ // No locking needed for this part, see comment in visitWeakReferences().
+ for (unsigned i = m_threads.size(); i--;) {
+ ThreadData* data = m_threads[i].get();
+ Safepoint* safepoint = data->m_safepoint;
+ if (!safepoint)
+ continue;
+ if (&safepoint->vm() != &vm)
+ continue;
+ if (safepoint->isKnownToBeLiveDuringGC())
+ continue;
+ safepoint->cancel();
}
}
@@ -294,15 +341,43 @@
{
MutexLocker locker(data->m_rightToRun);
+ {
+ MutexLocker locker(m_lock);
+ if (plan->stage == Plan::Cancelled) {
+ m_numberOfActiveThreads--;
+ continue;
+ }
+ plan->notifyCompiling();
+ }
if (Options::verboseCompilationQueue())
dataLog(*this, ": Compiling ", plan->key(), " asynchronously\n");
+ RELEASE_ASSERT(!plan->vm.heap.isCollecting());
plan->compileInThread(longLivedState, data);
+ RELEASE_ASSERT(!plan->vm.heap.isCollecting());
+
+ {
+ MutexLocker locker(m_lock);
+ if (plan->stage == Plan::Cancelled) {
+ m_numberOfActiveThreads--;
+ continue;
+ }
+ plan->notifyCompiled();
+ }
+ RELEASE_ASSERT(!plan->vm.heap.isCollecting());
}
-
+
{
MutexLocker locker(m_lock);
+
+ // We could have been cancelled between releasing rightToRun and acquiring m_lock.
+ // This would mean that we might be in the middle of GC right now.
+ if (plan->stage == Plan::Cancelled) {
+ m_numberOfActiveThreads--;
+ continue;
+ }
+
plan->notifyReady();
if (Options::verboseCompilationQueue()) {
diff --git a/Source/JavaScriptCore/dfg/DFGWorklist.h b/Source/JavaScriptCore/dfg/DFGWorklist.h
index e437199..0a30451 100644
--- a/Source/JavaScriptCore/dfg/DFGWorklist.h
+++ b/Source/JavaScriptCore/dfg/DFGWorklist.h
@@ -71,7 +71,9 @@
bool isActiveForVM(VM&) const;
- void visitChildren(SlotVisitor&, CodeBlockSet&); // Only called on the main thread after suspending all threads.
+ // Only called on the main thread after suspending all threads.
+ void visitWeakReferences(SlotVisitor&, CodeBlockSet&);
+ void removeDeadPlans(VM&);
void dump(PrintStream&) const;
diff --git a/Source/JavaScriptCore/ftl/FTLCompile.cpp b/Source/JavaScriptCore/ftl/FTLCompile.cpp
index f8681dc..cba2728 100644
--- a/Source/JavaScriptCore/ftl/FTLCompile.cpp
+++ b/Source/JavaScriptCore/ftl/FTLCompile.cpp
@@ -490,12 +490,12 @@
}
}
-void compile(State& state)
+void compile(State& state, Safepoint::Result& safepointResult)
{
char* error = 0;
{
- GraphSafepoint safepoint(state.graph);
+ GraphSafepoint safepoint(state.graph, safepointResult);
LLVMMCJITCompilerOptions options;
llvm->InitializeMCJITCompilerOptions(&options, sizeof(options));
@@ -566,7 +566,10 @@
llvm->DisposePassManager(modulePasses);
llvm->DisposeExecutionEngine(engine);
}
-
+ if (safepointResult.didGetCancelled())
+ return;
+ RELEASE_ASSERT(!state.graph.m_vm.heap.isCollecting());
+
if (shouldShowDisassembly()) {
for (unsigned i = 0; i < state.jitCode->handles().size(); ++i) {
ExecutableMemoryHandle* handle = state.jitCode->handles()[i].get();
diff --git a/Source/JavaScriptCore/ftl/FTLCompile.h b/Source/JavaScriptCore/ftl/FTLCompile.h
index 066a2f4..e4da4bd 100644
--- a/Source/JavaScriptCore/ftl/FTLCompile.h
+++ b/Source/JavaScriptCore/ftl/FTLCompile.h
@@ -1,5 +1,5 @@
/*
- * Copyright (C) 2013 Apple Inc. All rights reserved.
+ * Copyright (C) 2013, 2014 Apple Inc. All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
@@ -28,11 +28,12 @@
#if ENABLE(FTL_JIT)
+#include "DFGSafepoint.h"
#include "FTLState.h"
namespace JSC { namespace FTL {
-void compile(State&);
+void compile(State&, DFG::Safepoint::Result&);
} } // namespace JSC::FTL
diff --git a/Source/JavaScriptCore/heap/CodeBlockSet.cpp b/Source/JavaScriptCore/heap/CodeBlockSet.cpp
index 57ff34f..9c5fab2 100644
--- a/Source/JavaScriptCore/heap/CodeBlockSet.cpp
+++ b/Source/JavaScriptCore/heap/CodeBlockSet.cpp
@@ -145,6 +145,8 @@
void CodeBlockSet::rememberCurrentlyExecutingCodeBlocks(Heap* heap)
{
#if ENABLE(GGC)
+ if (verbose)
+ dataLog("Remembering ", m_currentlyExecuting.size(), " code blocks.\n");
for (CodeBlock* codeBlock : m_currentlyExecuting) {
heap->addToRememberedSet(codeBlock->ownerExecutable());
ASSERT(codeBlock->m_mayBeExecuting);
diff --git a/Source/JavaScriptCore/heap/Heap.cpp b/Source/JavaScriptCore/heap/Heap.cpp
index 20d1437..24de3ab 100644
--- a/Source/JavaScriptCore/heap/Heap.cpp
+++ b/Source/JavaScriptCore/heap/Heap.cpp
@@ -516,7 +516,6 @@
visitExternalRememberedSet();
visitSmallStrings();
visitConservativeRoots(conservativeRoots);
- visitCompilerWorklists();
visitProtectedObjects(heapRootVisitor);
visitTempSortVectors(heapRootVisitor);
visitArgumentBuffers(heapRootVisitor);
@@ -622,17 +621,23 @@
m_slotVisitor.donateAndDrain();
}
-void Heap::visitCompilerWorklists()
+void Heap::visitCompilerWorklistWeakReferences()
{
#if ENABLE(DFG_JIT)
- GCPHASE(VisitDFGWorklists);
for (auto worklist : m_suspendedCompilerWorklists)
- worklist->visitChildren(m_slotVisitor, m_codeBlocks);
+ worklist->visitWeakReferences(m_slotVisitor, m_codeBlocks);
if (Options::logGC() == GCLogging::Verbose)
dataLog("DFG Worklists:\n", m_slotVisitor);
+#endif
+}
- m_slotVisitor.donateAndDrain();
+void Heap::removeDeadCompilerWorklistEntries()
+{
+#if ENABLE(DFG_JIT)
+ GCPHASE(FinalizeDFGWorklists);
+ for (auto worklist : m_suspendedCompilerWorklists)
+ worklist->removeDeadPlans(*m_vm);
#endif
}
@@ -743,6 +748,8 @@
while (true) {
m_objectSpace.visitWeakSets(visitor);
harvestWeakReferences();
+ visitCompilerWorklistWeakReferences();
+ m_codeBlocks.traceMarked(m_slotVisitor); // New "executing" code blocks may be discovered.
if (m_slotVisitor.isEmpty())
break;
@@ -998,6 +1005,7 @@
copyBackingStores();
finalizeUnconditionalFinalizers();
+ removeDeadCompilerWorklistEntries();
deleteUnmarkedCompiledCode();
deleteSourceProviderCaches();
notifyIncrementalSweeper();
diff --git a/Source/JavaScriptCore/heap/Heap.h b/Source/JavaScriptCore/heap/Heap.h
index 64c6ae9..0effee0 100644
--- a/Source/JavaScriptCore/heap/Heap.h
+++ b/Source/JavaScriptCore/heap/Heap.h
@@ -274,7 +274,8 @@
void visitExternalRememberedSet();
void visitSmallStrings();
void visitConservativeRoots(ConservativeRoots&);
- void visitCompilerWorklists();
+ void visitCompilerWorklistWeakReferences();
+ void removeDeadCompilerWorklistEntries();
void visitProtectedObjects(HeapRootVisitor&);
void visitTempSortVectors(HeapRootVisitor&);
void visitArgumentBuffers(HeapRootVisitor&);
