The DFG should be able to tier-up and OSR enter into the FTL
https://bugs.webkit.org/show_bug.cgi?id=112838
Source/JavaScriptCore:
Reviewed by Mark Hahnenberg.
This adds the ability for the DFG to tier-up into the FTL. This works in both
of the expected tier-up modes:
Replacement: frequently called functions eventually have their entrypoint
replaced with one that goes into FTL-compiled code. Note, this will be a
slow-down for now since we don't yet have LLVM calling convention integration.
OSR entry: code stuck in hot loops gets OSR'd into the FTL from the DFG.
This means that if the DFG detects that a function is an FTL candidate, it
inserts execution counting code similar to the kind that the baseline JIT
would use. If you trip on a loop count in a loop header that is an OSR
candidate (it's not an inlined loop), we do OSR; otherwise we do replacement.
OSR almost always also implies future replacement.
OSR entry into the FTL is really cool. It uses a specialized FTL compile of
the code, where early in the DFG pipeline we replace the original root block
with an OSR entrypoint block that jumps to the pre-header of the hot loop.
The OSR entrypoint loads all live state at the loop pre-header using loads
from a scratch buffer, which gets populated by the runtime's OSR entry
preparation code (FTL::prepareOSREntry()). This approach appears to work well
with all of our subsequent optimizations, including prediction propagation,
CFA, and LICM. LLVM seems happy with it, too. Best of all, it works naturally
with concurrent compilation: when we hit the tier-up trigger we spawn a
compilation plan at the bytecode index from which we triggered; once the
compilation finishes the next trigger will try to enter, at that bytecode
index. If it can't - for example because the code has moved on to another
loop - then we just try again. Loops that get hot enough for OSR entry (about
25,000 iterations) will probably still be running when a concurrent compile
finishes, so this doesn't appear to be a big problem.
This immediately gives us a 70% speed-up on imaging-gaussian-blur. We could
get a bigger speed-up by adding some more intelligence and tweaking LLVM to
compile code faster. Those things will happen eventually but this is a good
start. Probably this code will see more tuning as we get more coverage in the
FTL JIT, but I'll worry about that in future patches.
* CMakeLists.txt:
* GNUmakefile.list.am:
* JavaScriptCore.xcodeproj/project.pbxproj:
* Target.pri:
* bytecode/CodeBlock.cpp:
(JSC::CodeBlock::CodeBlock):
(JSC::CodeBlock::hasOptimizedReplacement):
(JSC::CodeBlock::setOptimizationThresholdBasedOnCompilationResult):
* bytecode/CodeBlock.h:
* dfg/DFGAbstractInterpreterInlines.h:
(JSC::DFG::::executeEffects):
* dfg/DFGByteCodeParser.cpp:
(JSC::DFG::ByteCodeParser::parseBlock):
(JSC::DFG::ByteCodeParser::parse):
* dfg/DFGCFGSimplificationPhase.cpp:
(JSC::DFG::CFGSimplificationPhase::run):
* dfg/DFGClobberize.h:
(JSC::DFG::clobberize):
* dfg/DFGDriver.cpp:
(JSC::DFG::compileImpl):
(JSC::DFG::compile):
* dfg/DFGDriver.h:
* dfg/DFGFixupPhase.cpp:
(JSC::DFG::FixupPhase::fixupNode):
* dfg/DFGGraph.cpp:
(JSC::DFG::Graph::dump):
(JSC::DFG::Graph::killBlockAndItsContents):
(JSC::DFG::Graph::killUnreachableBlocks):
* dfg/DFGGraph.h:
* dfg/DFGInPlaceAbstractState.cpp:
(JSC::DFG::InPlaceAbstractState::initialize):
* dfg/DFGJITCode.cpp:
(JSC::DFG::JITCode::reconstruct):
(JSC::DFG::JITCode::checkIfOptimizationThresholdReached):
(JSC::DFG::JITCode::optimizeNextInvocation):
(JSC::DFG::JITCode::dontOptimizeAnytimeSoon):
(JSC::DFG::JITCode::optimizeAfterWarmUp):
(JSC::DFG::JITCode::optimizeSoon):
(JSC::DFG::JITCode::forceOptimizationSlowPathConcurrently):
(JSC::DFG::JITCode::setOptimizationThresholdBasedOnCompilationResult):
* dfg/DFGJITCode.h:
* dfg/DFGJITFinalizer.cpp:
(JSC::DFG::JITFinalizer::finalize):
(JSC::DFG::JITFinalizer::finalizeFunction):
(JSC::DFG::JITFinalizer::finalizeCommon):
* dfg/DFGLoopPreHeaderCreationPhase.cpp:
(JSC::DFG::createPreHeader):
(JSC::DFG::LoopPreHeaderCreationPhase::run):
* dfg/DFGLoopPreHeaderCreationPhase.h:
* dfg/DFGNode.h:
(JSC::DFG::Node::hasUnlinkedLocal):
(JSC::DFG::Node::unlinkedLocal):
* dfg/DFGNodeType.h:
* dfg/DFGOSREntry.cpp:
(JSC::DFG::prepareOSREntry):
* dfg/DFGOSREntrypointCreationPhase.cpp: Added.
(JSC::DFG::OSREntrypointCreationPhase::OSREntrypointCreationPhase):
(JSC::DFG::OSREntrypointCreationPhase::run):
(JSC::DFG::performOSREntrypointCreation):
* dfg/DFGOSREntrypointCreationPhase.h: Added.
* dfg/DFGOperations.cpp:
* dfg/DFGOperations.h:
* dfg/DFGPlan.cpp:
(JSC::DFG::Plan::Plan):
(JSC::DFG::Plan::compileInThread):
(JSC::DFG::Plan::compileInThreadImpl):
* dfg/DFGPlan.h:
* dfg/DFGPredictionInjectionPhase.cpp:
(JSC::DFG::PredictionInjectionPhase::run):
* dfg/DFGPredictionPropagationPhase.cpp:
(JSC::DFG::PredictionPropagationPhase::propagate):
* dfg/DFGSafeToExecute.h:
(JSC::DFG::safeToExecute):
* dfg/DFGSpeculativeJIT32_64.cpp:
(JSC::DFG::SpeculativeJIT::compile):
* dfg/DFGSpeculativeJIT64.cpp:
(JSC::DFG::SpeculativeJIT::compile):
* dfg/DFGTierUpCheckInjectionPhase.cpp: Added.
(JSC::DFG::TierUpCheckInjectionPhase::TierUpCheckInjectionPhase):
(JSC::DFG::TierUpCheckInjectionPhase::run):
(JSC::DFG::performTierUpCheckInjection):
* dfg/DFGTierUpCheckInjectionPhase.h: Added.
* dfg/DFGToFTLDeferredCompilationCallback.cpp: Added.
(JSC::DFG::ToFTLDeferredCompilationCallback::ToFTLDeferredCompilationCallback):
(JSC::DFG::ToFTLDeferredCompilationCallback::~ToFTLDeferredCompilationCallback):
(JSC::DFG::ToFTLDeferredCompilationCallback::create):
(JSC::DFG::ToFTLDeferredCompilationCallback::compilationDidBecomeReadyAsynchronously):
(JSC::DFG::ToFTLDeferredCompilationCallback::compilationDidComplete):
* dfg/DFGToFTLDeferredCompilationCallback.h: Added.
* dfg/DFGToFTLForOSREntryDeferredCompilationCallback.cpp: Added.
(JSC::DFG::ToFTLForOSREntryDeferredCompilationCallback::ToFTLForOSREntryDeferredCompilationCallback):
(JSC::DFG::ToFTLForOSREntryDeferredCompilationCallback::~ToFTLForOSREntryDeferredCompilationCallback):
(JSC::DFG::ToFTLForOSREntryDeferredCompilationCallback::create):
(JSC::DFG::ToFTLForOSREntryDeferredCompilationCallback::compilationDidBecomeReadyAsynchronously):
(JSC::DFG::ToFTLForOSREntryDeferredCompilationCallback::compilationDidComplete):
* dfg/DFGToFTLForOSREntryDeferredCompilationCallback.h: Added.
* dfg/DFGWorklist.cpp:
(JSC::DFG::globalWorklist):
* dfg/DFGWorklist.h:
* ftl/FTLCapabilities.cpp:
(JSC::FTL::canCompile):
* ftl/FTLCapabilities.h:
* ftl/FTLForOSREntryJITCode.cpp: Added.
(JSC::FTL::ForOSREntryJITCode::ForOSREntryJITCode):
(JSC::FTL::ForOSREntryJITCode::~ForOSREntryJITCode):
(JSC::FTL::ForOSREntryJITCode::ftlForOSREntry):
(JSC::FTL::ForOSREntryJITCode::initializeEntryBuffer):
* ftl/FTLForOSREntryJITCode.h: Added.
(JSC::FTL::ForOSREntryJITCode::entryBuffer):
(JSC::FTL::ForOSREntryJITCode::setBytecodeIndex):
(JSC::FTL::ForOSREntryJITCode::bytecodeIndex):
(JSC::FTL::ForOSREntryJITCode::countEntryFailure):
(JSC::FTL::ForOSREntryJITCode::entryFailureCount):
* ftl/FTLJITFinalizer.cpp:
(JSC::FTL::JITFinalizer::finalizeFunction):
* ftl/FTLLink.cpp:
(JSC::FTL::link):
* ftl/FTLLowerDFGToLLVM.cpp:
(JSC::FTL::LowerDFGToLLVM::compileBlock):
(JSC::FTL::LowerDFGToLLVM::compileNode):
(JSC::FTL::LowerDFGToLLVM::compileExtractOSREntryLocal):
(JSC::FTL::LowerDFGToLLVM::compileGetLocal):
(JSC::FTL::LowerDFGToLLVM::addWeakReference):
* ftl/FTLOSREntry.cpp: Added.
(JSC::FTL::prepareOSREntry):
* ftl/FTLOSREntry.h: Added.
* ftl/FTLOutput.h:
(JSC::FTL::Output::crashNonTerminal):
(JSC::FTL::Output::crash):
* ftl/FTLState.cpp:
(JSC::FTL::State::State):
* interpreter/Register.h:
(JSC::Register::unboxedDouble):
* jit/JIT.cpp:
(JSC::JIT::emitEnterOptimizationCheck):
* jit/JITCode.cpp:
(JSC::JITCode::ftlForOSREntry):
* jit/JITCode.h:
* jit/JITStubs.cpp:
(JSC::DEFINE_STUB_FUNCTION):
* runtime/Executable.cpp:
(JSC::ScriptExecutable::newReplacementCodeBlockFor):
* runtime/Options.h:
* runtime/VM.cpp:
(JSC::VM::ensureWorklist):
* runtime/VM.h:
LayoutTests:
Reviewed by Mark Hahnenberg.
Fix marsaglia to check the result instead of printing, and add a second
version that relies on OSR entry.
* fast/js/regress/marsaglia-osr-entry-expected.txt: Added.
* fast/js/regress/marsaglia-osr-entry.html: Added.
* fast/js/regress/script-tests/marsaglia-osr-entry.js: Added.
(marsaglia):
* fast/js/regress/script-tests/marsaglia.js:
git-svn-id: http://svn.webkit.org/repository/webkit/trunk@155023 268f45cc-cd09-0410-ab3c-d52691b4dbfc
diff --git a/Source/JavaScriptCore/dfg/DFGAbstractInterpreterInlines.h b/Source/JavaScriptCore/dfg/DFGAbstractInterpreterInlines.h
index c721d36..e7ba132 100644
--- a/Source/JavaScriptCore/dfg/DFGAbstractInterpreterInlines.h
+++ b/Source/JavaScriptCore/dfg/DFGAbstractInterpreterInlines.h
@@ -150,6 +150,20 @@
forNode(node) = value;
break;
}
+
+ case ExtractOSREntryLocal: {
+ forNode(node).makeTop();
+ if (!operandIsArgument(node->unlinkedLocal())
+ && m_graph.m_lazyVars.get(node->unlinkedLocal())) {
+ // This is kind of pessimistic - we could know in some cases that the
+ // DFG code at the point of the OSR had already initialized the lazy
+ // variable. But maybe this is fine, since we're inserting OSR
+ // entrypoints very early in the pipeline - so any lazy initializations
+ // ought to be hoisted out anyway.
+ forNode(node).merge(SpecEmpty);
+ }
+ break;
+ }
case GetLocal: {
VariableAccessData* variableAccessData = node->variableAccessData();
@@ -1515,6 +1529,14 @@
case Phantom:
case InlineStart:
case CountExecution:
+ case CheckTierUpInLoop:
+ case CheckTierUpAtReturn:
+ break;
+
+ case CheckTierUpAndOSREnter:
+ case LoopHint:
+ // We pretend that it can exit because it may want to get all state.
+ node->setCanExit(true);
break;
case Unreachable:
diff --git a/Source/JavaScriptCore/dfg/DFGByteCodeParser.cpp b/Source/JavaScriptCore/dfg/DFGByteCodeParser.cpp
index 7a0a56d..833092f 100644
--- a/Source/JavaScriptCore/dfg/DFGByteCodeParser.cpp
+++ b/Source/JavaScriptCore/dfg/DFGByteCodeParser.cpp
@@ -3140,19 +3140,18 @@
if (!m_inlineStackTop->m_caller)
m_currentBlock->isOSRTarget = true;
+ addToGraph(LoopHint);
+
if (m_vm->watchdog.isEnabled())
addToGraph(CheckWatchdogTimer);
- else {
- // Emit a phantom node to ensure that there is a placeholder
- // node for this bytecode op.
- addToGraph(Phantom);
- }
NEXT_OPCODE(op_loop_hint);
}
case op_init_lazy_reg: {
set(currentInstruction[1].u.operand, getJSConstantForValue(JSValue()));
+ ASSERT(currentInstruction[1].u.operand >= 0);
+ m_graph.m_lazyVars.set(currentInstruction[1].u.operand);
NEXT_OPCODE(op_init_lazy_reg);
}
@@ -3634,7 +3633,7 @@
BasicBlock* block = m_graph.block(blockIndex);
ASSERT(block);
if (!block->isReachable) {
- m_graph.killBlock(block);
+ m_graph.killBlockAndItsContents(block);
continue;
}
diff --git a/Source/JavaScriptCore/dfg/DFGCFGSimplificationPhase.cpp b/Source/JavaScriptCore/dfg/DFGCFGSimplificationPhase.cpp
index 63fff9b..ef69fed 100644
--- a/Source/JavaScriptCore/dfg/DFGCFGSimplificationPhase.cpp
+++ b/Source/JavaScriptCore/dfg/DFGCFGSimplificationPhase.cpp
@@ -270,16 +270,7 @@
m_graph.invalidateCFG();
m_graph.resetReachability();
-
- for (BlockIndex blockIndex = 0; blockIndex < m_graph.numBlocks(); ++blockIndex) {
- BasicBlock* block = m_graph.block(blockIndex);
- if (!block)
- continue;
- if (block->isReachable)
- continue;
-
- killUnreachable(block);
- }
+ m_graph.killUnreachableBlocks();
}
if (Options::validateGraphAtEachPhase())
@@ -320,19 +311,6 @@
}
}
- void killUnreachable(BasicBlock* block)
- {
- ASSERT(block);
- ASSERT(!block->isReachable);
-
- for (unsigned phiIndex = block->phis.size(); phiIndex--;)
- m_graph.m_allocator.free(block->phis[phiIndex]);
- for (unsigned nodeIndex = block->size(); nodeIndex--;)
- m_graph.m_allocator.free(block->at(nodeIndex));
-
- m_graph.killBlock(block);
- }
-
void keepOperandAlive(BasicBlock* block, BasicBlock* jettisonedBlock, CodeOrigin codeOrigin, int operand)
{
Node* livenessNode = jettisonedBlock->variablesAtHead.operand(operand);
diff --git a/Source/JavaScriptCore/dfg/DFGClobberize.h b/Source/JavaScriptCore/dfg/DFGClobberize.h
index 0ed6490..e82fd71 100644
--- a/Source/JavaScriptCore/dfg/DFGClobberize.h
+++ b/Source/JavaScriptCore/dfg/DFGClobberize.h
@@ -111,6 +111,7 @@
case IsString:
case LogicalNot:
case Int32ToDouble:
+ case ExtractOSREntryLocal:
return;
case MovHintAndCheck:
@@ -133,6 +134,10 @@
case ForceOSRExit:
case Return:
case Unreachable:
+ case CheckTierUpInLoop:
+ case CheckTierUpAtReturn:
+ case CheckTierUpAndOSREnter:
+ case LoopHint:
write(SideState);
return;
diff --git a/Source/JavaScriptCore/dfg/DFGDriver.cpp b/Source/JavaScriptCore/dfg/DFGDriver.cpp
index dc147ed6..dadc2b9 100644
--- a/Source/JavaScriptCore/dfg/DFGDriver.cpp
+++ b/Source/JavaScriptCore/dfg/DFGDriver.cpp
@@ -55,9 +55,9 @@
#if ENABLE(DFG_JIT)
static CompilationResult compileImpl(
- ExecState* exec, CodeBlock* codeBlock, CompilationMode mode,
- unsigned osrEntryBytecodeIndex, PassRefPtr<DeferredCompilationCallback> callback,
- Worklist* worklist)
+ VM& vm, CodeBlock* codeBlock, CompilationMode mode, unsigned osrEntryBytecodeIndex,
+ const Operands<JSValue>& mustHandleValues,
+ PassRefPtr<DeferredCompilationCallback> callback, Worklist* worklist)
{
SamplingRegion samplingRegion("DFG Compilation (Driver)");
@@ -67,8 +67,6 @@
ASSERT(codeBlock->alternative());
ASSERT(codeBlock->alternative()->jitType() == JITCode::BaselineJIT);
- ASSERT(osrEntryBytecodeIndex != UINT_MAX);
-
if (!Options::useDFGJIT() || !MacroAssembler::supportsFloatingPoint())
return CompilationFailed;
@@ -76,9 +74,7 @@
return CompilationFailed;
if (logCompilationChanges())
- dataLog("DFG(Driver) compiling ", *codeBlock, ", number of instructions = ", codeBlock->instructionCount(), "\n");
-
- VM& vm = exec->vm();
+ dataLog("DFG(Driver) compiling ", *codeBlock, " with ", mode, ", number of instructions = ", codeBlock->instructionCount(), "\n");
// Make sure that any stubs that the DFG is going to use are initialized. We want to
// make sure that al JIT code generation does finalization on the main thread.
@@ -93,29 +89,8 @@
vm.getCTIStub(FTL::osrExitGenerationThunkGenerator);
#endif
- // Derive our set of must-handle values. The compilation must be at least conservative
- // enough to allow for OSR entry with these values.
- unsigned numVarsWithValues;
- if (osrEntryBytecodeIndex)
- numVarsWithValues = codeBlock->m_numVars;
- else
- numVarsWithValues = 0;
RefPtr<Plan> plan = adoptRef(
- new Plan(codeBlock, mode, osrEntryBytecodeIndex, numVarsWithValues));
- for (size_t i = 0; i < plan->mustHandleValues.size(); ++i) {
- int operand = plan->mustHandleValues.operandForIndex(i);
- if (operandIsArgument(operand)
- && !operandToArgument(operand)
- && codeBlock->codeType() == FunctionCode
- && codeBlock->specializationKind() == CodeForConstruct) {
- // Ugh. If we're in a constructor, the 'this' argument may hold garbage. It will
- // also never be used. It doesn't matter what we put into the value for this,
- // but it has to be an actual value that can be grokked by subsequent DFG passes,
- // so we sanitize it here by turning it into Undefined.
- plan->mustHandleValues[i] = jsUndefined();
- } else
- plan->mustHandleValues[i] = exec->uncheckedR(operand).jsValue();
- }
+ new Plan(codeBlock, mode, osrEntryBytecodeIndex, mustHandleValues));
if (worklist) {
plan->callback = callback;
@@ -130,7 +105,7 @@
}
#else // ENABLE(DFG_JIT)
static CompilationResult compileImpl(
- ExecState*, CodeBlock*, CompilationMode, unsigned,
+ VM&, CodeBlock*, CompilationMode, unsigned, const Operands<JSValue>&,
PassRefPtr<DeferredCompilationCallback>, Worklist*)
{
return CompilationFailed;
@@ -138,13 +113,13 @@
#endif // ENABLE(DFG_JIT)
CompilationResult compile(
- ExecState* exec, CodeBlock* codeBlock, CompilationMode mode,
- unsigned osrEntryBytecodeIndex, PassRefPtr<DeferredCompilationCallback> passedCallback,
- Worklist* worklist)
+ VM& vm, CodeBlock* codeBlock, CompilationMode mode, unsigned osrEntryBytecodeIndex,
+ const Operands<JSValue>& mustHandleValues,
+ PassRefPtr<DeferredCompilationCallback> passedCallback, Worklist* worklist)
{
RefPtr<DeferredCompilationCallback> callback = passedCallback;
CompilationResult result = compileImpl(
- exec, codeBlock, mode, osrEntryBytecodeIndex, callback, worklist);
+ vm, codeBlock, mode, osrEntryBytecodeIndex, mustHandleValues, callback, worklist);
if (result != CompilationDeferred)
callback->compilationDidComplete(codeBlock, result);
return result;
diff --git a/Source/JavaScriptCore/dfg/DFGDriver.h b/Source/JavaScriptCore/dfg/DFGDriver.h
index 03204c8..9d43638 100644
--- a/Source/JavaScriptCore/dfg/DFGDriver.h
+++ b/Source/JavaScriptCore/dfg/DFGDriver.h
@@ -46,7 +46,10 @@
// If the worklist is non-null, we do a concurrent compile. Otherwise we do a synchronous
// compile. Even if we do a synchronous compile, we call the callback with the result.
-CompilationResult compile(ExecState*, CodeBlock*, CompilationMode, unsigned osrEntryBytecodeIndex, PassRefPtr<DeferredCompilationCallback>, Worklist*);
+CompilationResult compile(
+ VM&, CodeBlock*, CompilationMode, unsigned osrEntryBytecodeIndex,
+ const Operands<JSValue>& mustHandleValues,
+ PassRefPtr<DeferredCompilationCallback>, Worklist*);
} } // namespace JSC::DFG
diff --git a/Source/JavaScriptCore/dfg/DFGFixupPhase.cpp b/Source/JavaScriptCore/dfg/DFGFixupPhase.cpp
index ddaa642..feb27c0 100644
--- a/Source/JavaScriptCore/dfg/DFGFixupPhase.cpp
+++ b/Source/JavaScriptCore/dfg/DFGFixupPhase.cpp
@@ -844,6 +844,9 @@
case MovHint:
case MovHintAndCheck:
case ZombieHint:
+ case CheckTierUpInLoop:
+ case CheckTierUpAtReturn:
+ case CheckTierUpAndOSREnter:
RELEASE_ASSERT_NOT_REACHED();
break;
@@ -896,6 +899,8 @@
case ForceOSRExit:
case CheckWatchdogTimer:
case Unreachable:
+ case ExtractOSREntryLocal:
+ case LoopHint:
break;
#else
default:
diff --git a/Source/JavaScriptCore/dfg/DFGGraph.cpp b/Source/JavaScriptCore/dfg/DFGGraph.cpp
index 8baa03c..e2c0561 100644
--- a/Source/JavaScriptCore/dfg/DFGGraph.cpp
+++ b/Source/JavaScriptCore/dfg/DFGGraph.cpp
@@ -236,6 +236,13 @@
else
out.print(comma, "r", operand, "(", VariableAccessDataDump(*this, variableAccessData), ")");
}
+ if (node->hasUnlinkedLocal()) {
+ int operand = node->unlinkedLocal();
+ if (operandIsArgument(operand))
+ out.print(comma, "arg", operandToArgument(operand));
+ else
+ out.print(comma, "r", operand);
+ }
if (node->hasConstantBuffer()) {
out.print(comma);
out.print(node->startConstant(), ":[");
@@ -487,6 +494,29 @@
determineReachability();
}
+void Graph::killBlockAndItsContents(BasicBlock* block)
+{
+ for (unsigned phiIndex = block->phis.size(); phiIndex--;)
+ m_allocator.free(block->phis[phiIndex]);
+ for (unsigned nodeIndex = block->size(); nodeIndex--;)
+ m_allocator.free(block->at(nodeIndex));
+
+ killBlock(block);
+}
+
+void Graph::killUnreachableBlocks()
+{
+ for (BlockIndex blockIndex = 0; blockIndex < numBlocks(); ++blockIndex) {
+ BasicBlock* block = this->block(blockIndex);
+ if (!block)
+ continue;
+ if (block->isReachable)
+ continue;
+
+ killBlockAndItsContents(block);
+ }
+}
+
void Graph::resetExitStates()
{
for (BlockIndex blockIndex = 0; blockIndex < m_blocks.size(); ++blockIndex) {
diff --git a/Source/JavaScriptCore/dfg/DFGGraph.h b/Source/JavaScriptCore/dfg/DFGGraph.h
index 026cffe..d25fa08 100644
--- a/Source/JavaScriptCore/dfg/DFGGraph.h
+++ b/Source/JavaScriptCore/dfg/DFGGraph.h
@@ -489,6 +489,10 @@
killBlock(basicBlock->index);
}
+ void killBlockAndItsContents(BasicBlock*);
+
+ void killUnreachableBlocks();
+
bool isPredictedNumerical(Node* node)
{
return isNumerical(node->child1().useKind()) && isNumerical(node->child2().useKind());
@@ -698,6 +702,7 @@
bool m_hasArguments;
HashSet<ExecutableBase*> m_executablesWhoseArgumentsEscaped;
BitVector m_preservedVars;
+ BitVector m_lazyVars;
Dominators m_dominators;
NaturalLoops m_naturalLoops;
unsigned m_localVars;
diff --git a/Source/JavaScriptCore/dfg/DFGInPlaceAbstractState.cpp b/Source/JavaScriptCore/dfg/DFGInPlaceAbstractState.cpp
index f4c4a9f..68530a7 100644
--- a/Source/JavaScriptCore/dfg/DFGInPlaceAbstractState.cpp
+++ b/Source/JavaScriptCore/dfg/DFGInPlaceAbstractState.cpp
@@ -107,6 +107,7 @@
root->cfaHasVisited = false;
root->cfaFoundConstants = false;
for (size_t i = 0; i < root->valuesAtHead.numberOfArguments(); ++i) {
+ root->valuesAtTail.argument(i).clear();
if (m_graph.m_form == SSA) {
root->valuesAtHead.argument(i).makeTop();
continue;
@@ -129,8 +130,6 @@
root->valuesAtHead.argument(i).setType(SpecCell);
else
root->valuesAtHead.argument(i).makeTop();
-
- root->valuesAtTail.argument(i).clear();
}
for (size_t i = 0; i < root->valuesAtHead.numberOfLocals(); ++i) {
Node* node = root->variablesAtHead.local(i);
diff --git a/Source/JavaScriptCore/dfg/DFGJITCode.cpp b/Source/JavaScriptCore/dfg/DFGJITCode.cpp
index ba80553..a3f1f37 100644
--- a/Source/JavaScriptCore/dfg/DFGJITCode.cpp
+++ b/Source/JavaScriptCore/dfg/DFGJITCode.cpp
@@ -28,6 +28,8 @@
#if ENABLE(DFG_JIT)
+#include "CodeBlock.h"
+
namespace JSC { namespace DFG {
JITCode::JITCode()
@@ -59,6 +61,145 @@
variableEventStream.shrinkToFit();
}
+void JITCode::reconstruct(
+ CodeBlock* codeBlock, CodeOrigin codeOrigin, unsigned streamIndex,
+ Operands<ValueRecovery>& result)
+{
+ variableEventStream.reconstruct(
+ codeBlock, codeOrigin, minifiedDFG, streamIndex, result);
+}
+
+void JITCode::reconstruct(
+ ExecState* exec, CodeBlock* codeBlock, CodeOrigin codeOrigin, unsigned streamIndex,
+ Operands<JSValue>& result)
+{
+ Operands<ValueRecovery> recoveries;
+ reconstruct(codeBlock, codeOrigin, streamIndex, recoveries);
+
+ result = Operands<JSValue>(OperandsLike, recoveries);
+ for (size_t i = result.size(); i--;) {
+ int operand = result.operandForIndex(i);
+
+ if (operandIsArgument(operand)
+ && !operandToArgument(operand)
+ && codeBlock->codeType() == FunctionCode
+ && codeBlock->specializationKind() == CodeForConstruct) {
+ // Ugh. If we're in a constructor, the 'this' argument may hold garbage. It will
+ // also never be used. It doesn't matter what we put into the value for this,
+ // but it has to be an actual value that can be grokked by subsequent DFG passes,
+ // so we sanitize it here by turning it into Undefined.
+ result[i] = jsUndefined();
+ continue;
+ }
+
+ ValueRecovery recovery = recoveries[i];
+ JSValue value;
+ switch (recovery.technique()) {
+ case AlreadyInJSStack:
+ case AlreadyInJSStackAsUnboxedCell:
+ case AlreadyInJSStackAsUnboxedBoolean:
+ value = exec->r(operand).jsValue();
+ break;
+ case AlreadyInJSStackAsUnboxedInt32:
+ value = jsNumber(exec->r(operand).unboxedInt32());
+ break;
+ case AlreadyInJSStackAsUnboxedDouble:
+ value = jsDoubleNumber(exec->r(operand).unboxedDouble());
+ break;
+ case Constant:
+ value = recovery.constant();
+ break;
+ default:
+ RELEASE_ASSERT_NOT_REACHED();
+ break;
+ }
+ result[i] = value;
+ }
+}
+
+#if ENABLE(FTL_JIT)
+bool JITCode::checkIfOptimizationThresholdReached(CodeBlock* codeBlock)
+{
+ ASSERT(codeBlock->jitType() == JITCode::DFGJIT);
+ return tierUpCounter.checkIfThresholdCrossedAndSet(codeBlock->baselineVersion());
+}
+
+void JITCode::optimizeNextInvocation(CodeBlock* codeBlock)
+{
+ ASSERT(codeBlock->jitType() == JITCode::DFGJIT);
+ if (Options::verboseOSR())
+ dataLog(*codeBlock, ": FTL-optimizing next invocation.\n");
+ tierUpCounter.setNewThreshold(0, codeBlock->baselineVersion());
+}
+
+void JITCode::dontOptimizeAnytimeSoon(CodeBlock* codeBlock)
+{
+ ASSERT(codeBlock->jitType() == JITCode::DFGJIT);
+ if (Options::verboseOSR())
+ dataLog(*codeBlock, ": Not FTL-optimizing anytime soon.\n");
+ tierUpCounter.deferIndefinitely();
+}
+
+void JITCode::optimizeAfterWarmUp(CodeBlock* codeBlock)
+{
+ ASSERT(codeBlock->jitType() == JITCode::DFGJIT);
+ if (Options::verboseOSR())
+ dataLog(*codeBlock, ": FTL-optimizing after warm-up.\n");
+ CodeBlock* baseline = codeBlock->baselineVersion();
+ tierUpCounter.setNewThreshold(
+ baseline->adjustedCounterValue(Options::thresholdForFTLOptimizeAfterWarmUp()),
+ baseline);
+}
+
+void JITCode::optimizeSoon(CodeBlock* codeBlock)
+{
+ ASSERT(codeBlock->jitType() == JITCode::DFGJIT);
+ if (Options::verboseOSR())
+ dataLog(*codeBlock, ": FTL-optimizing soon.\n");
+ CodeBlock* baseline = codeBlock->baselineVersion();
+ tierUpCounter.setNewThreshold(
+ baseline->adjustedCounterValue(Options::thresholdForFTLOptimizeSoon()),
+ baseline);
+}
+
+void JITCode::forceOptimizationSlowPathConcurrently(CodeBlock* codeBlock)
+{
+ ASSERT(codeBlock->jitType() == JITCode::DFGJIT);
+ if (Options::verboseOSR())
+ dataLog(*codeBlock, ": Forcing slow path concurrently for FTL entry.\n");
+ tierUpCounter.forceSlowPathConcurrently();
+}
+
+void JITCode::setOptimizationThresholdBasedOnCompilationResult(
+ CodeBlock* codeBlock, CompilationResult result)
+{
+ ASSERT(codeBlock->jitType() == JITCode::DFGJIT);
+ switch (result) {
+ case CompilationSuccessful:
+ optimizeNextInvocation(codeBlock);
+ return;
+ case CompilationFailed:
+ dontOptimizeAnytimeSoon(codeBlock);
+ codeBlock->baselineVersion()->m_didFailFTLCompilation = true;
+ return;
+ case CompilationDeferred:
+ optimizeAfterWarmUp(codeBlock);
+ return;
+ case CompilationInvalidated:
+ // This is weird - it will only happen in cases when the DFG code block (i.e.
+ // the code block that this JITCode belongs to) is also invalidated. So it
+ // doesn't really matter what we do. But, we do the right thing anyway. Note
+ // that us counting the reoptimization actually means that we might count it
+ // twice. But that's generally OK. It's better to overcount reoptimizations
+ // than it is to undercount them.
+ codeBlock->baselineVersion()->countReoptimization();
+ optimizeAfterWarmUp(codeBlock);
+ return;
+ }
+ RELEASE_ASSERT_NOT_REACHED();
+}
+#endif // ENABLE(FTL_JIT)
+
} } // namespace JSC::DFG
#endif // ENABLE(DFG_JIT)
diff --git a/Source/JavaScriptCore/dfg/DFGJITCode.h b/Source/JavaScriptCore/dfg/DFGJITCode.h
index fb2f2b6..864cca8 100644
--- a/Source/JavaScriptCore/dfg/DFGJITCode.h
+++ b/Source/JavaScriptCore/dfg/DFGJITCode.h
@@ -30,11 +30,13 @@
#if ENABLE(DFG_JIT)
+#include "CompilationResult.h"
#include "DFGCommonData.h"
#include "DFGMinifiedGraph.h"
#include "DFGOSREntry.h"
#include "DFGOSRExit.h"
#include "DFGVariableEventStream.h"
+#include "ExecutionCounter.h"
#include "JITCode.h"
#include "JumpReplacementWatchpoint.h"
#include <wtf/SegmentedVector.h>
@@ -93,8 +95,30 @@
return result;
}
+ void reconstruct(
+ CodeBlock*, CodeOrigin, unsigned streamIndex, Operands<ValueRecovery>& result);
+
+ // This is only applicable if we're at a point where all values are spilled to the
+ // stack. Currently, it also has the restriction that the values must be in their
+ // bytecode-designated stack slots.
+ void reconstruct(
+ ExecState*, CodeBlock*, CodeOrigin, unsigned streamIndex, Operands<JSValue>& result);
+
+#if ENABLE(FTL_JIT)
+ // NB. All of these methods take CodeBlock* because they may want to use
+ // CodeBlock's logic about scaling thresholds. It should be a DFG CodeBlock.
+
+ bool checkIfOptimizationThresholdReached(CodeBlock*);
+ void optimizeNextInvocation(CodeBlock*);
+ void dontOptimizeAnytimeSoon(CodeBlock*);
+ void optimizeAfterWarmUp(CodeBlock*);
+ void optimizeSoon(CodeBlock*);
+ void forceOptimizationSlowPathConcurrently(CodeBlock*);
+ void setOptimizationThresholdBasedOnCompilationResult(CodeBlock*, CompilationResult);
+#endif // ENABLE(FTL_JIT)
+
void shrinkToFit();
-
+
private:
friend class JITCompiler; // Allow JITCompiler to call setCodeRef().
@@ -106,6 +130,10 @@
SegmentedVector<JumpReplacementWatchpoint, 1, 0> watchpoints;
DFG::VariableEventStream variableEventStream;
DFG::MinifiedGraph minifiedDFG;
+#if ENABLE(FTL_JIT)
+ ExecutionCounter tierUpCounter;
+ RefPtr<CodeBlock> osrEntryBlock;
+#endif // ENABLE(FTL_JIT)
};
} } // namespace JSC::DFG
diff --git a/Source/JavaScriptCore/dfg/DFGJITFinalizer.cpp b/Source/JavaScriptCore/dfg/DFGJITFinalizer.cpp
index 150eea6..c2d0702 100644
--- a/Source/JavaScriptCore/dfg/DFGJITFinalizer.cpp
+++ b/Source/JavaScriptCore/dfg/DFGJITFinalizer.cpp
@@ -48,27 +48,31 @@
bool JITFinalizer::finalize()
{
- finalizeCommon();
-
m_jitCode->initializeCodeRef(m_linkBuffer->finalizeCodeWithoutDisassembly());
m_plan.codeBlock->setJITCode(m_jitCode, MacroAssemblerCodePtr());
+ finalizeCommon();
+
return true;
}
bool JITFinalizer::finalizeFunction()
{
- finalizeCommon();
-
MacroAssemblerCodePtr withArityCheck = m_linkBuffer->locationOf(m_arityCheck);
m_jitCode->initializeCodeRef(m_linkBuffer->finalizeCodeWithoutDisassembly());
m_plan.codeBlock->setJITCode(m_jitCode, withArityCheck);
+ finalizeCommon();
+
return true;
}
void JITFinalizer::finalizeCommon()
{
+#if ENABLE(FTL_JIT)
+ m_jitCode->optimizeAfterWarmUp(m_plan.codeBlock.get());
+#endif // ENABLE(FTL_JIT)
+
if (m_plan.compilation)
m_plan.vm.m_perBytecodeProfiler->addCompilation(m_plan.compilation);
}
diff --git a/Source/JavaScriptCore/dfg/DFGLoopPreHeaderCreationPhase.cpp b/Source/JavaScriptCore/dfg/DFGLoopPreHeaderCreationPhase.cpp
index 14d8b19..507e00f 100644
--- a/Source/JavaScriptCore/dfg/DFGLoopPreHeaderCreationPhase.cpp
+++ b/Source/JavaScriptCore/dfg/DFGLoopPreHeaderCreationPhase.cpp
@@ -37,6 +37,31 @@
namespace JSC { namespace DFG {
+BasicBlock* createPreHeader(Graph& graph, BlockInsertionSet& insertionSet, BasicBlock* block)
+{
+ BasicBlock* preHeader = insertionSet.insertBefore(block);
+ preHeader->appendNode(
+ graph, SpecNone, Jump, block->at(0)->codeOrigin, OpInfo(block));
+
+ for (unsigned predecessorIndex = 0; predecessorIndex < block->predecessors.size(); predecessorIndex++) {
+ BasicBlock* predecessor = block->predecessors[predecessorIndex];
+ if (graph.m_dominators.dominates(block, predecessor))
+ continue;
+ block->predecessors[predecessorIndex--] = block->predecessors.last();
+ block->predecessors.removeLast();
+ for (unsigned successorIndex = predecessor->numSuccessors(); successorIndex--;) {
+ BasicBlock*& successor = predecessor->successor(successorIndex);
+ if (successor != block)
+ continue;
+ successor = preHeader;
+ preHeader->predecessors.append(predecessor);
+ }
+ }
+
+ block->predecessors.append(preHeader);
+ return preHeader;
+}
+
class LoopPreHeaderCreationPhase : public Phase {
public:
LoopPreHeaderCreationPhase(Graph& graph)
@@ -70,26 +95,7 @@
if (!needsNewPreHeader)
continue;
- BasicBlock* preHeader = m_insertionSet.insertBefore(loop.header());
- preHeader->appendNode(
- m_graph, SpecNone, Jump, loop.header()->at(0)->codeOrigin, OpInfo(loop.header()));
-
- for (unsigned predecessorIndex = 0; predecessorIndex < loop.header()->predecessors.size(); predecessorIndex++) {
- BasicBlock* predecessor = loop.header()->predecessors[predecessorIndex];
- if (m_graph.m_dominators.dominates(loop.header(), predecessor))
- continue;
- loop.header()->predecessors[predecessorIndex--] = loop.header()->predecessors.last();
- loop.header()->predecessors.takeLast();
- for (unsigned successorIndex = predecessor->numSuccessors(); successorIndex--;) {
- BasicBlock*& successor = predecessor->successor(successorIndex);
- if (successor != loop.header())
- continue;
- successor = preHeader;
- preHeader->predecessors.append(predecessor);
- }
- }
-
- loop.header()->predecessors.append(preHeader);
+ createPreHeader(m_graph, m_insertionSet, loop.header());
}
return m_insertionSet.execute();
diff --git a/Source/JavaScriptCore/dfg/DFGLoopPreHeaderCreationPhase.h b/Source/JavaScriptCore/dfg/DFGLoopPreHeaderCreationPhase.h
index cc362d7..a229875 100644
--- a/Source/JavaScriptCore/dfg/DFGLoopPreHeaderCreationPhase.h
+++ b/Source/JavaScriptCore/dfg/DFGLoopPreHeaderCreationPhase.h
@@ -32,13 +32,27 @@
namespace JSC { namespace DFG {
+class BlockInsertionSet;
class Graph;
+struct BasicBlock;
// Inserts dummy basic blocks before any loop headers that don't already have
// a single non-loop predecessor.
bool performLoopPreHeaderCreation(Graph&);
+// Creates a new basic block (a pre-header) that jumps to the given block. All
+// predecessors of the given block that aren't dominated by it are rerouted to
+// the pre-header.
+//
+// This function is used internally and it's used in a surgical fashion by
+// OSREntrypointCreationPhase.
+//
+// Note that executing this function requires having an intact dominators
+// analysis. You should run that analysis before doing damage to the CFG, even
+// if said damage is done before you call this.
+BasicBlock* createPreHeader(Graph&, BlockInsertionSet&, BasicBlock*);
+
} } // namespace JSC::DFG
#endif // ENABLE(DFG_JIT)
diff --git a/Source/JavaScriptCore/dfg/DFGNode.h b/Source/JavaScriptCore/dfg/DFGNode.h
index d898b70..e55d4be 100644
--- a/Source/JavaScriptCore/dfg/DFGNode.h
+++ b/Source/JavaScriptCore/dfg/DFGNode.h
@@ -527,9 +527,20 @@
return variableAccessData()->local();
}
+ bool hasUnlinkedLocal()
+ {
+ switch (op()) {
+ case GetLocalUnlinked:
+ case ExtractOSREntryLocal:
+ return true;
+ default:
+ return false;
+ }
+ }
+
VirtualRegister unlinkedLocal()
{
- ASSERT(op() == GetLocalUnlinked);
+ ASSERT(hasUnlinkedLocal());
return static_cast<VirtualRegister>(m_opInfo);
}
diff --git a/Source/JavaScriptCore/dfg/DFGNodeType.h b/Source/JavaScriptCore/dfg/DFGNodeType.h
index 6f6a462..02f4bb5 100644
--- a/Source/JavaScriptCore/dfg/DFGNodeType.h
+++ b/Source/JavaScriptCore/dfg/DFGNodeType.h
@@ -69,6 +69,21 @@
macro(Flush, NodeMustGenerate | NodeDoesNotExit) \
macro(PhantomLocal, NodeMustGenerate | NodeDoesNotExit) \
\
+ /* Hint that this is where bytecode thinks is a good place to OSR. Note that this */\
+ /* will exist even in inlined loops. This has no execution semantics but it must */\
+ /* survive all DCE. We treat this as being a can-exit because tier-up to FTL may */\
+ /* want all state. */\
+ macro(LoopHint, NodeMustGenerate) \
+ \
+ /* Special node for OSR entry into the FTL. Indicates that we're loading a local */\
+ /* variable from the scratch buffer. */\
+ macro(ExtractOSREntryLocal, NodeResultJS) \
+ \
+ /* Tier-up checks from the DFG to the FTL. */\
+ macro(CheckTierUpInLoop, NodeMustGenerate) \
+ macro(CheckTierUpAndOSREnter, NodeMustGenerate) \
+ macro(CheckTierUpAtReturn, NodeMustGenerate) \
+ \
/* Get the value of a local variable, without linking into the VariableAccessData */\
/* network. This is only valid for variable accesses whose predictions originated */\
/* as something other than a local access, and thus had their own profiling. */\
diff --git a/Source/JavaScriptCore/dfg/DFGOSREntry.cpp b/Source/JavaScriptCore/dfg/DFGOSREntry.cpp
index 46b65e7..0b70cab 100644
--- a/Source/JavaScriptCore/dfg/DFGOSREntry.cpp
+++ b/Source/JavaScriptCore/dfg/DFGOSREntry.cpp
@@ -40,18 +40,43 @@
void* prepareOSREntry(ExecState* exec, CodeBlock* codeBlock, unsigned bytecodeIndex)
{
#if DFG_ENABLE(OSR_ENTRY)
- ASSERT(codeBlock->jitType() == JITCode::DFGJIT);
+ ASSERT(JITCode::isOptimizingJIT(codeBlock->jitType()));
ASSERT(codeBlock->alternative());
ASSERT(codeBlock->alternative()->jitType() == JITCode::BaselineJIT);
ASSERT(!codeBlock->jitCodeMap());
if (Options::verboseOSR()) {
dataLog(
- "OSR in ", *codeBlock->alternative(), " -> ", *codeBlock,
+ "DFG OSR in ", *codeBlock->alternative(), " -> ", *codeBlock,
" from bc#", bytecodeIndex, "\n");
}
VM* vm = &exec->vm();
+ if (codeBlock->jitType() != JITCode::DFGJIT) {
+ RELEASE_ASSERT(codeBlock->jitType() == JITCode::FTLJIT);
+
+ // When will this happen? We could have:
+ //
+ // - An exit from the FTL JIT into the baseline JIT followed by an attempt
+ // to reenter. We're fine with allowing this to fail. If it happens
+ // enough we'll just reoptimize. It basically means that the OSR exit cost
+ // us dearly and so reoptimizing is the right thing to do.
+ //
+ // - We have recursive code with hot loops. Consider that foo has a hot loop
+ // that calls itself. We have two foo's on the stack, lets call them foo1
+ // and foo2, with foo1 having called foo2 from foo's hot loop. foo2 gets
+ // optimized all the way into the FTL. Then it returns into foo1, and then
+ // foo1 wants to get optimized. It might reach this conclusion from its
+ // hot loop and attempt to OSR enter. And we'll tell it that it can't. It
+ // might be worth addressing this case, but I just think this case will
+ // be super rare. For now, if it does happen, it'll cause some compilation
+ // thrashing.
+
+ if (Options::verboseOSR())
+ dataLog(" OSR failed because the target code block is not DFG.\n");
+ return 0;
+ }
+
OSREntryData* entry = codeBlock->jitCode()->dfg()->osrEntryDataForBytecodeIndex(bytecodeIndex);
if (!entry) {
diff --git a/Source/JavaScriptCore/dfg/DFGOSREntrypointCreationPhase.cpp b/Source/JavaScriptCore/dfg/DFGOSREntrypointCreationPhase.cpp
new file mode 100644
index 0000000..ef4f4bc
--- /dev/null
+++ b/Source/JavaScriptCore/dfg/DFGOSREntrypointCreationPhase.cpp
@@ -0,0 +1,130 @@
+/*
+ * Copyright (C) 2013 Apple Inc. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in the
+ * documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. ``AS IS'' AND ANY
+ * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+ * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR
+ * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
+ * EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
+ * PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
+ * PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
+ * OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#include "config.h"
+#include "DFGOSREntrypointCreationPhase.h"
+
+#if ENABLE(DFG_JIT)
+
+#include "DFGBasicBlockInlines.h"
+#include "DFGBlockInsertionSet.h"
+#include "DFGGraph.h"
+#include "DFGLoopPreHeaderCreationPhase.h"
+#include "DFGPhase.h"
+#include "Operations.h"
+
+namespace JSC { namespace DFG {
+
+class OSREntrypointCreationPhase : public Phase {
+public:
+ OSREntrypointCreationPhase(Graph& graph)
+ : Phase(graph, "OSR entrypoint creation")
+ {
+ }
+
+ bool run()
+ {
+ RELEASE_ASSERT(m_graph.m_plan.mode == FTLForOSREntryMode);
+ RELEASE_ASSERT(m_graph.m_form == ThreadedCPS);
+
+ unsigned bytecodeIndex = m_graph.m_plan.osrEntryBytecodeIndex;
+ RELEASE_ASSERT(bytecodeIndex);
+ RELEASE_ASSERT(bytecodeIndex != UINT_MAX);
+
+ // Needed by createPreHeader().
+ m_graph.m_dominators.computeIfNecessary(m_graph);
+
+ CodeBlock* baseline = m_graph.m_profiledBlock;
+
+ BasicBlock* target = 0;
+ for (unsigned blockIndex = m_graph.numBlocks(); blockIndex--;) {
+ BasicBlock* block = m_graph.block(blockIndex);
+ if (!block)
+ continue;
+ Node* firstNode = block->at(0);
+ if (firstNode->op() == LoopHint
+ && firstNode->codeOrigin == CodeOrigin(bytecodeIndex)) {
+ target = block;
+ break;
+ }
+ }
+
+ if (!target) {
+ // This is a terrible outcome. It shouldn't often happen but it might
+ // happen and so we should defend against it. If it happens, then this
+ // compilation is a failure.
+ return false;
+ }
+
+ BlockInsertionSet insertionSet(m_graph);
+
+ BasicBlock* newRoot = insertionSet.insert(0);
+ CodeOrigin codeOrigin = target->at(0)->codeOrigin;
+
+ for (int argument = 0; argument < baseline->numParameters(); ++argument) {
+ Node* oldNode = target->variablesAtHead.argument(argument);
+ if (!oldNode) {
+ // Just for sanity, always have a SetArgument even if it's not needed.
+ oldNode = m_graph.m_arguments[argument];
+ }
+ Node* node = newRoot->appendNode(
+ m_graph, SpecNone, SetArgument, codeOrigin,
+ OpInfo(oldNode->variableAccessData()));
+ m_graph.m_arguments[argument] = node;
+ }
+ for (int local = 0; local < baseline->m_numCalleeRegisters; ++local) {
+ Node* previousHead = target->variablesAtHead.local(local);
+ if (!previousHead)
+ continue;
+ VariableAccessData* variable = previousHead->variableAccessData();
+ Node* node = newRoot->appendNode(
+ m_graph, variable->prediction(), ExtractOSREntryLocal, codeOrigin,
+ OpInfo(variable->local()));
+ newRoot->appendNode(
+ m_graph, SpecNone, SetLocal, codeOrigin, OpInfo(variable), Edge(node));
+ }
+
+ newRoot->appendNode(
+ m_graph, SpecNone, Jump, codeOrigin,
+ OpInfo(createPreHeader(m_graph, insertionSet, target)));
+
+ insertionSet.execute();
+ m_graph.resetReachability();
+ m_graph.killUnreachableBlocks();
+ return true;
+ }
+};
+
+bool performOSREntrypointCreation(Graph& graph)
+{
+ SamplingRegion samplingRegion("DFG OSR Entrypoint Creation");
+ return runPhase<OSREntrypointCreationPhase>(graph);
+}
+
+} } // namespace JSC::DFG
+
+#endif // ENABLE(DFG_JIT)
+
+
diff --git a/Source/JavaScriptCore/dfg/DFGOSREntrypointCreationPhase.h b/Source/JavaScriptCore/dfg/DFGOSREntrypointCreationPhase.h
new file mode 100644
index 0000000..a763721
--- /dev/null
+++ b/Source/JavaScriptCore/dfg/DFGOSREntrypointCreationPhase.h
@@ -0,0 +1,54 @@
+/*
+ * Copyright (C) 2013 Apple Inc. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in the
+ * documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. ``AS IS'' AND ANY
+ * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+ * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR
+ * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
+ * EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
+ * PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
+ * PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
+ * OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#ifndef DFGOSREntrypointCreationPhase_h
+#define DFGOSREntrypointCreationPhase_h
+
+#include <wtf/Platform.h>
+
+#if ENABLE(DFG_JIT)
+
+namespace JSC { namespace DFG {
+
+class Graph;
+
+// This phase is only for compilation pipelines that go through SSA, and that
+// do a specialized OSR entry compile. Currently that means FTL.
+//
+// Creates an OSR entrypoint that establishes all argument and local variable
+// values. This entrypoint only works for the Plan::osrEntryBytecodeIndex.
+// Unlike ordinary OSR entry into the DFG, this forces all variables slurped
+// in through OSR entry to appear TOP to the CFA. Hence this phase must be run
+// early - before doing any CFA. But this also does some additional hacks that
+// require this to run before unification.
+
+bool performOSREntrypointCreation(Graph&);
+
+} } // namespace JSC::DFG
+
+#endif // ENABLE(DFG_JIT)
+
+#endif // DFGOSREntrypointCreationPhase_h
+
diff --git a/Source/JavaScriptCore/dfg/DFGOperations.cpp b/Source/JavaScriptCore/dfg/DFGOperations.cpp
index 802c7e6..7a3b0bd 100644
--- a/Source/JavaScriptCore/dfg/DFGOperations.cpp
+++ b/Source/JavaScriptCore/dfg/DFGOperations.cpp
@@ -31,9 +31,15 @@
#include "CodeBlock.h"
#include "CommonSlowPaths.h"
#include "CopiedSpaceInlines.h"
+#include "DFGDriver.h"
#include "DFGOSRExit.h"
#include "DFGRepatch.h"
#include "DFGThunks.h"
+#include "DFGToFTLDeferredCompilationCallback.h"
+#include "DFGToFTLForOSREntryDeferredCompilationCallback.h"
+#include "DFGWorklist.h"
+#include "FTLForOSREntryJITCode.h"
+#include "FTLOSREntry.h"
#include "HostCallReturnValue.h"
#include "GetterSetter.h"
#include "Interpreter.h"
@@ -2015,6 +2021,187 @@
codeBlock->reoptimize();
}
+#if ENABLE(FTL_JIT)
+void DFG_OPERATION triggerTierUpNow(ExecState* exec)
+{
+ VM* vm = &exec->vm();
+ NativeCallFrameTracer tracer(vm, exec);
+ DeferGC deferGC(vm->heap);
+ CodeBlock* codeBlock = exec->codeBlock();
+
+ JITCode* jitCode = codeBlock->jitCode()->dfg();
+
+ if (Options::verboseOSR()) {
+ dataLog(
+ *codeBlock, ": Entered triggerTierUpNow with executeCounter = ",
+ jitCode->tierUpCounter, "\n");
+ }
+
+ if (codeBlock->baselineVersion()->m_didFailFTLCompilation) {
+ if (Options::verboseOSR())
+ dataLog("Deferring FTL-optimization of ", *codeBlock, " indefinitely because there was an FTL failure.\n");
+ jitCode->dontOptimizeAnytimeSoon(codeBlock);
+ return;
+ }
+
+ if (!jitCode->checkIfOptimizationThresholdReached(codeBlock)) {
+ if (Options::verboseOSR())
+ dataLog("Choosing not to FTL-optimize ", *codeBlock, " yet.\n");
+ return;
+ }
+
+ Worklist::State worklistState;
+ if (Worklist* worklist = vm->worklist.get()) {
+ worklistState = worklist->completeAllReadyPlansForVM(
+ *vm, CompilationKey(codeBlock->baselineVersion(), FTLMode));
+ } else
+ worklistState = Worklist::NotKnown;
+
+ if (worklistState == Worklist::Compiling) {
+ jitCode->setOptimizationThresholdBasedOnCompilationResult(
+ codeBlock, CompilationDeferred);
+ return;
+ }
+
+ if (codeBlock->hasOptimizedReplacement()) {
+ // That's great, we've compiled the code - next time we call this function,
+ // we'll enter that replacement.
+ jitCode->optimizeSoon(codeBlock);
+ return;
+ }
+
+ if (worklistState == Worklist::Compiled) {
+ // This means that we finished compiling, but failed somehow; in that case the
+ // thresholds will be set appropriately.
+ if (Options::verboseOSR())
+ dataLog("Code block ", *codeBlock, " was compiled but it doesn't have an optimized replacement.\n");
+ return;
+ }
+
+ // We need to compile the code.
+ compile(
+ *vm, codeBlock->newReplacement().get(), FTLMode, UINT_MAX, Operands<JSValue>(),
+ ToFTLDeferredCompilationCallback::create(codeBlock), vm->ensureWorklist());
+}
+
+char* DFG_OPERATION triggerOSREntryNow(
+ ExecState* exec, int32_t bytecodeIndex, int32_t streamIndex)
+{
+ VM* vm = &exec->vm();
+ NativeCallFrameTracer tracer(vm, exec);
+ DeferGC deferGC(vm->heap);
+ CodeBlock* codeBlock = exec->codeBlock();
+
+ JITCode* jitCode = codeBlock->jitCode()->dfg();
+
+ if (Options::verboseOSR()) {
+ dataLog(
+ *codeBlock, ": Entered triggerTierUpNow with executeCounter = ",
+ jitCode->tierUpCounter, "\n");
+ }
+
+ if (codeBlock->baselineVersion()->m_didFailFTLCompilation) {
+ if (Options::verboseOSR())
+ dataLog("Deferring FTL-optimization of ", *codeBlock, " indefinitely because there was an FTL failure.\n");
+ jitCode->dontOptimizeAnytimeSoon(codeBlock);
+ return 0;
+ }
+
+ if (!jitCode->checkIfOptimizationThresholdReached(codeBlock)) {
+ if (Options::verboseOSR())
+ dataLog("Choosing not to FTL-optimize ", *codeBlock, " yet.\n");
+ return 0;
+ }
+
+ Worklist::State worklistState;
+ if (Worklist* worklist = vm->worklist.get()) {
+ worklistState = worklist->completeAllReadyPlansForVM(
+ *vm, CompilationKey(codeBlock->baselineVersion(), FTLForOSREntryMode));
+ } else
+ worklistState = Worklist::NotKnown;
+
+ if (worklistState == Worklist::Compiling) {
+ ASSERT(!jitCode->osrEntryBlock);
+ jitCode->setOptimizationThresholdBasedOnCompilationResult(
+ codeBlock, CompilationDeferred);
+ return 0;
+ }
+
+ if (CodeBlock* entryBlock = jitCode->osrEntryBlock.get()) {
+ void* address = FTL::prepareOSREntry(
+ exec, codeBlock, entryBlock, bytecodeIndex, streamIndex);
+ if (address) {
+ jitCode->optimizeSoon(codeBlock);
+ return static_cast<char*>(address);
+ }
+
+ FTL::ForOSREntryJITCode* entryCode = entryBlock->jitCode()->ftlForOSREntry();
+ entryCode->countEntryFailure();
+ if (entryCode->entryFailureCount() <
+ Options::ftlOSREntryFailureCountForReoptimization()) {
+
+ jitCode->optimizeSoon(codeBlock);
+ return 0;
+ }
+
+ // OSR entry failed. Oh no! This implies that we need to retry. We retry
+ // without exponential backoff and we only do this for the entry code block.
+ jitCode->osrEntryBlock.clear();
+
+ jitCode->optimizeAfterWarmUp(codeBlock);
+ return 0;
+ }
+
+ if (worklistState == Worklist::Compiled) {
+ // This means that compilation failed and we already set the thresholds.
+ if (Options::verboseOSR())
+ dataLog("Code block ", *codeBlock, " was compiled but it doesn't have an optimized replacement.\n");
+ return 0;
+ }
+
+ // The first order of business is to trigger a for-entry compile.
+ Operands<JSValue> mustHandleValues;
+ jitCode->reconstruct(
+ exec, codeBlock, CodeOrigin(bytecodeIndex), streamIndex, mustHandleValues);
+ CompilationResult forEntryResult = DFG::compile(
+ *vm, codeBlock->newReplacement().get(), FTLForOSREntryMode, bytecodeIndex,
+ mustHandleValues, ToFTLForOSREntryDeferredCompilationCallback::create(codeBlock),
+ vm->ensureWorklist());
+
+ // But we also want to trigger a replacement compile. Of course, we don't want to
+ // trigger it if we don't need to. Note that this is kind of weird because we might
+ // have just finished an FTL compile and that compile failed or was invalidated.
+ // But this seems uncommon enough that we sort of don't care. It's certainly sound
+ // to fire off another compile right now so long as we're not already compiling and
+ // we don't already have an optimized replacement. Note, we don't do this for
+ // obviously bad cases like global code, where we know that there is a slim chance
+ // of this code being invoked ever again.
+ CompilationKey keyForReplacement(codeBlock->baselineVersion(), FTLMode);
+ if (codeBlock->codeType() != GlobalCode
+ && !codeBlock->hasOptimizedReplacement()
+ && (!vm->worklist.get()
+ || vm->worklist->compilationState(keyForReplacement) == Worklist::NotKnown)) {
+ compile(
+ *vm, codeBlock->newReplacement().get(), FTLMode, UINT_MAX, Operands<JSValue>(),
+ ToFTLDeferredCompilationCallback::create(codeBlock), vm->ensureWorklist());
+ }
+
+ if (forEntryResult != CompilationSuccessful)
+ return 0;
+
+ // It's possible that the for-entry compile already succeeded. In that case OSR
+ // entry will succeed unless we ran out of stack. It's not clear what we should do.
+ // We signal to try again after a while if that happens.
+ void* address = FTL::prepareOSREntry(
+ exec, codeBlock, jitCode->osrEntryBlock.get(), bytecodeIndex, streamIndex);
+ if (address)
+ jitCode->optimizeSoon(codeBlock);
+ else
+ jitCode->optimizeAfterWarmUp(codeBlock);
+ return static_cast<char*>(address);
+}
+#endif // ENABLE(FTL_JIT)
+
} // extern "C"
} } // namespace JSC::DFG
diff --git a/Source/JavaScriptCore/dfg/DFGOperations.h b/Source/JavaScriptCore/dfg/DFGOperations.h
index c9d35a3..6c0171b 100644
--- a/Source/JavaScriptCore/dfg/DFGOperations.h
+++ b/Source/JavaScriptCore/dfg/DFGOperations.h
@@ -99,6 +99,7 @@
typedef size_t DFG_OPERATION (*S_DFGOperation_EJ)(ExecState*, EncodedJSValue);
typedef size_t DFG_OPERATION (*S_DFGOperation_EJJ)(ExecState*, EncodedJSValue, EncodedJSValue);
typedef size_t DFG_OPERATION (*S_DFGOperation_J)(EncodedJSValue);
+typedef void DFG_OPERATION (*V_DFGOperation_E)(ExecState*);
typedef void DFG_OPERATION (*V_DFGOperation_EOZD)(ExecState*, JSObject*, int32_t, double);
typedef void DFG_OPERATION (*V_DFGOperation_EOZJ)(ExecState*, JSObject*, int32_t, EncodedJSValue);
typedef void DFG_OPERATION (*V_DFGOperation_EC)(ExecState*, JSCell*);
@@ -126,6 +127,7 @@
typedef char* DFG_OPERATION (*P_DFGOperation_EStPS)(ExecState*, Structure*, void*, size_t);
typedef char* DFG_OPERATION (*P_DFGOperation_EStSS)(ExecState*, Structure*, size_t, size_t);
typedef char* DFG_OPERATION (*P_DFGOperation_EStZ)(ExecState*, Structure*, int32_t);
+typedef char* DFG_OPERATION (*P_DFGOperation_EZZ)(ExecState*, int32_t, int32_t);
typedef StringImpl* DFG_OPERATION (*I_DFGOperation_EJss)(ExecState*, JSString*);
typedef JSString* DFG_OPERATION (*Jss_DFGOperation_EZ)(ExecState*, int32_t);
JSCell* DFG_OPERATION operationStringFromCharCode(ExecState*, int32_t) WTF_INTERNAL;
@@ -302,6 +304,11 @@
void DFG_OPERATION triggerReoptimizationNow(CodeBlock*) WTF_INTERNAL;
+#if ENABLE(FTL_JIT)
+void DFG_OPERATION triggerTierUpNow(ExecState*) WTF_INTERNAL;
+char* DFG_OPERATION triggerOSREntryNow(ExecState*, int32_t bytecodeIndex, int32_t streamIndex) WTF_INTERNAL;
+#endif // ENABLE(FTL_JIT)
+
} // extern "C"
inline P_DFGOperation_EStZ operationNewTypedArrayWithSizeForType(TypedArrayType type)
diff --git a/Source/JavaScriptCore/dfg/DFGPlan.cpp b/Source/JavaScriptCore/dfg/DFGPlan.cpp
index 696dc0e..05e2174 100644
--- a/Source/JavaScriptCore/dfg/DFGPlan.cpp
+++ b/Source/JavaScriptCore/dfg/DFGPlan.cpp
@@ -46,13 +46,16 @@
#include "DFGLivenessAnalysisPhase.h"
#include "DFGLoopPreHeaderCreationPhase.h"
#include "DFGOSRAvailabilityAnalysisPhase.h"
+#include "DFGOSREntrypointCreationPhase.h"
#include "DFGPredictionInjectionPhase.h"
#include "DFGPredictionPropagationPhase.h"
#include "DFGSSAConversionPhase.h"
+#include "DFGTierUpCheckInjectionPhase.h"
#include "DFGTypeCheckHoistingPhase.h"
#include "DFGUnificationPhase.h"
#include "DFGValidate.h"
#include "DFGVirtualRegisterAllocationPhase.h"
+#include "OperandsInlines.h"
#include "Operations.h"
#include <wtf/CurrentTime.h>
@@ -81,13 +84,12 @@
Plan::Plan(
PassRefPtr<CodeBlock> passedCodeBlock, CompilationMode mode,
- unsigned osrEntryBytecodeIndex, unsigned numVarsWithValues)
+ unsigned osrEntryBytecodeIndex, const Operands<JSValue>& mustHandleValues)
: vm(*passedCodeBlock->vm())
, codeBlock(passedCodeBlock)
, mode(mode)
, osrEntryBytecodeIndex(osrEntryBytecodeIndex)
- , numVarsWithValues(numVarsWithValues)
- , mustHandleValues(codeBlock->numParameters(), numVarsWithValues)
+ , mustHandleValues(mustHandleValues)
, compilation(codeBlock->vm()->m_perBytecodeProfiler ? adoptRef(new Profiler::Compilation(codeBlock->vm()->m_perBytecodeProfiler->ensureBytecodesFor(codeBlock.get()), Profiler::DFG)) : 0)
, identifiers(codeBlock.get())
, weakReferences(codeBlock.get())
@@ -109,7 +111,7 @@
CompilationScope compilationScope;
if (logCompilationChanges())
- dataLog("DFG(Plan) compiling ", *codeBlock, ", number of instructions = ", codeBlock->instructionCount(), "\n");
+ dataLog("DFG(Plan) compiling ", *codeBlock, " with ", mode, ", number of instructions = ", codeBlock->instructionCount(), "\n");
CompilationPath path = compileInThreadImpl(longLivedState);
@@ -133,7 +135,7 @@
break;
}
double now = currentTimeMS();
- dataLog("Optimized ", *codeBlock->alternative(), " with ", pathName, " in ", now - before, " ms");
+ dataLog("Optimized ", *codeBlock->alternative(), " using ", mode, " with ", pathName, " in ", now - before, " ms");
if (path == FTLPath)
dataLog(" (DFG: ", beforeFTL - before, ", LLVM: ", now - beforeFTL, ")");
dataLog(".\n");
@@ -142,6 +144,12 @@
Plan::CompilationPath Plan::compileInThreadImpl(LongLivedState& longLivedState)
{
+ if (verboseCompilationEnabled() && osrEntryBytecodeIndex != UINT_MAX) {
+ dataLog("\n");
+ dataLog("Compiler must handle OSR entry from bc#", osrEntryBytecodeIndex, " with values: ", mustHandleValues, "\n");
+ dataLog("\n");
+ }
+
Graph dfg(vm, *this, longLivedState);
if (!parse(dfg)) {
@@ -162,6 +170,15 @@
performUnification(dfg);
performPredictionInjection(dfg);
+ if (mode == FTLForOSREntryMode) {
+ bool result = performOSREntrypointCreation(dfg);
+ if (!result) {
+ finalizer = adoptPtr(new FailedFinalizer(*this));
+ return FailPath;
+ }
+ performCPSRethreading(dfg);
+ }
+
if (validationEnabled())
validate(dfg);
@@ -206,10 +223,19 @@
dfg.m_naturalLoops.computeIfNecessary(dfg);
}
+ switch (mode) {
+ case DFGMode: {
+ performTierUpCheckInjection(dfg);
+ break;
+ }
+
+ case FTLMode:
+ case FTLForOSREntryMode: {
#if ENABLE(FTL_JIT)
- if (Options::useExperimentalFTL()
- && codeBlock->codeType() == FunctionCode
- && FTL::canCompile(dfg)) {
+ if (FTL::canCompile(dfg) == FTL::CannotCompile) {
+ finalizer = adoptPtr(new FailedFinalizer(*this));
+ return FailPath;
+ }
performCriticalEdgeBreaking(dfg);
performLoopPreHeaderCreation(dfg);
@@ -244,10 +270,16 @@
FTL::compile(state);
FTL::link(state);
return FTLPath;
- }
#else
- RELEASE_ASSERT(!Options::useExperimentalFTL());
+ RELEASE_ASSERT_NOT_REACHED();
+ break;
#endif // ENABLE(FTL_JIT)
+ }
+
+ default:
+ RELEASE_ASSERT_NOT_REACHED();
+ break;
+ }
performCPSRethreading(dfg);
performDCE(dfg);
diff --git a/Source/JavaScriptCore/dfg/DFGPlan.h b/Source/JavaScriptCore/dfg/DFGPlan.h
index 01d7c3c..a602697 100644
--- a/Source/JavaScriptCore/dfg/DFGPlan.h
+++ b/Source/JavaScriptCore/dfg/DFGPlan.h
@@ -56,7 +56,7 @@
struct Plan : public ThreadSafeRefCounted<Plan> {
Plan(
PassRefPtr<CodeBlock>, CompilationMode, unsigned osrEntryBytecodeIndex,
- unsigned numVarsWithValues);
+ const Operands<JSValue>& mustHandleValues);
~Plan();
void compileInThread(LongLivedState&);
@@ -72,7 +72,6 @@
RefPtr<CodeBlock> codeBlock;
CompilationMode mode;
const unsigned osrEntryBytecodeIndex;
- const unsigned numVarsWithValues;
Operands<JSValue> mustHandleValues;
RefPtr<Profiler::Compilation> compilation;
diff --git a/Source/JavaScriptCore/dfg/DFGPredictionInjectionPhase.cpp b/Source/JavaScriptCore/dfg/DFGPredictionInjectionPhase.cpp
index c116ef9..a8e403b 100644
--- a/Source/JavaScriptCore/dfg/DFGPredictionInjectionPhase.cpp
+++ b/Source/JavaScriptCore/dfg/DFGPredictionInjectionPhase.cpp
@@ -77,8 +77,8 @@
if (block->bytecodeBegin != m_graph.m_plan.osrEntryBytecodeIndex)
continue;
for (size_t i = 0; i < m_graph.m_plan.mustHandleValues.size(); ++i) {
- Node* node = block->variablesAtHead.operand(
- m_graph.m_plan.mustHandleValues.operandForIndex(i));
+ int operand = m_graph.m_plan.mustHandleValues.operandForIndex(i);
+ Node* node = block->variablesAtHead.operand(operand);
if (!node)
continue;
ASSERT(node->hasLocal(m_graph));
diff --git a/Source/JavaScriptCore/dfg/DFGPredictionPropagationPhase.cpp b/Source/JavaScriptCore/dfg/DFGPredictionPropagationPhase.cpp
index cad4c7a..f682510 100644
--- a/Source/JavaScriptCore/dfg/DFGPredictionPropagationPhase.cpp
+++ b/Source/JavaScriptCore/dfg/DFGPredictionPropagationPhase.cpp
@@ -484,7 +484,10 @@
case ArrayifyToStructure:
case MovHint:
case MovHintAndCheck:
- case ZombieHint: {
+ case ZombieHint:
+ case CheckTierUpInLoop:
+ case CheckTierUpAtReturn:
+ case CheckTierUpAndOSREnter: {
// This node should never be visible at this stage of compilation. It is
// inserted by fixup(), which follows this phase.
RELEASE_ASSERT_NOT_REACHED();
@@ -549,6 +552,11 @@
case PutGlobalVar:
case CheckWatchdogTimer:
case Unreachable:
+ case LoopHint:
+ break;
+
+ // This gets ignored because it already has a prediction.
+ case ExtractOSREntryLocal:
break;
// These gets ignored because it doesn't do anything.
diff --git a/Source/JavaScriptCore/dfg/DFGSafeToExecute.h b/Source/JavaScriptCore/dfg/DFGSafeToExecute.h
index bf99b12..15a6c18 100644
--- a/Source/JavaScriptCore/dfg/DFGSafeToExecute.h
+++ b/Source/JavaScriptCore/dfg/DFGSafeToExecute.h
@@ -233,6 +233,11 @@
case StringFromCharCode:
case NewTypedArray:
case Unreachable:
+ case ExtractOSREntryLocal:
+ case CheckTierUpInLoop:
+ case CheckTierUpAtReturn:
+ case CheckTierUpAndOSREnter:
+ case LoopHint:
return true;
case GetByVal:
diff --git a/Source/JavaScriptCore/dfg/DFGSpeculativeJIT32_64.cpp b/Source/JavaScriptCore/dfg/DFGSpeculativeJIT32_64.cpp
index 8e47021..48057e5 100644
--- a/Source/JavaScriptCore/dfg/DFGSpeculativeJIT32_64.cpp
+++ b/Source/JavaScriptCore/dfg/DFGSpeculativeJIT32_64.cpp
@@ -4823,6 +4823,7 @@
break;
case PhantomLocal:
+ case LoopHint:
// This is a no-op.
noResult(node);
break;
@@ -4835,6 +4836,10 @@
case Phi:
case Upsilon:
case GetArgument:
+ case ExtractOSREntryLocal:
+ case CheckTierUpInLoop:
+ case CheckTierUpAtReturn:
+ case CheckTierUpAndOSREnter:
RELEASE_ASSERT_NOT_REACHED();
break;
}
diff --git a/Source/JavaScriptCore/dfg/DFGSpeculativeJIT64.cpp b/Source/JavaScriptCore/dfg/DFGSpeculativeJIT64.cpp
index d30a7bc..d2c51b7 100644
--- a/Source/JavaScriptCore/dfg/DFGSpeculativeJIT64.cpp
+++ b/Source/JavaScriptCore/dfg/DFGSpeculativeJIT64.cpp
@@ -4673,6 +4673,7 @@
break;
case PhantomLocal:
+ case LoopHint:
// This is a no-op.
noResult(node);
break;
@@ -4680,11 +4681,75 @@
case Unreachable:
RELEASE_ASSERT_NOT_REACHED();
break;
+
+#if ENABLE(FTL_JIT)
+ case CheckTierUpInLoop: {
+ MacroAssembler::Jump done = m_jit.branchAdd32(
+ MacroAssembler::Signed,
+ TrustedImm32(Options::ftlTierUpCounterIncrementForLoop()),
+ MacroAssembler::AbsoluteAddress(&m_jit.jitCode()->tierUpCounter.m_counter));
+
+ silentSpillAllRegisters(InvalidGPRReg);
+ m_jit.setupArgumentsExecState();
+ appendCall(triggerTierUpNow);
+ silentFillAllRegisters(InvalidGPRReg);
+
+ done.link(&m_jit);
+ break;
+ }
+
+ case CheckTierUpAtReturn: {
+ MacroAssembler::Jump done = m_jit.branchAdd32(
+ MacroAssembler::Signed,
+ TrustedImm32(Options::ftlTierUpCounterIncrementForReturn()),
+ MacroAssembler::AbsoluteAddress(&m_jit.jitCode()->tierUpCounter.m_counter));
+
+ silentSpillAllRegisters(InvalidGPRReg);
+ m_jit.setupArgumentsExecState();
+ appendCall(triggerTierUpNow);
+ silentFillAllRegisters(InvalidGPRReg);
+
+ done.link(&m_jit);
+ break;
+ }
+
+ case CheckTierUpAndOSREnter: {
+ ASSERT(!node->codeOrigin.inlineCallFrame);
+
+ GPRTemporary temp(this);
+ GPRReg tempGPR = temp.gpr();
+
+ MacroAssembler::Jump done = m_jit.branchAdd32(
+ MacroAssembler::Signed,
+ TrustedImm32(Options::ftlTierUpCounterIncrementForLoop()),
+ MacroAssembler::AbsoluteAddress(&m_jit.jitCode()->tierUpCounter.m_counter));
+
+ silentSpillAllRegisters(tempGPR);
+ m_jit.setupArgumentsWithExecState(
+ TrustedImm32(node->codeOrigin.bytecodeIndex),
+ TrustedImm32(m_stream->size()));
+ appendCallSetResult(triggerOSREntryNow, tempGPR);
+ MacroAssembler::Jump dontEnter = m_jit.branchTestPtr(MacroAssembler::Zero, tempGPR);
+ m_jit.jump(tempGPR);
+ dontEnter.link(&m_jit);
+ silentFillAllRegisters(tempGPR);
+
+ done.link(&m_jit);
+ break;
+ }
+#else // ENABLE(FTL_JIT)
+ case CheckTierUpInLoop:
+ case CheckTierUpAtReturn:
+ case CheckTierUpAndOSREnter:
+ RELEASE_ASSERT_NOT_REACHED();
+ break;
+#endif // ENABLE(FTL_JIT)
case LastNodeType:
case Phi:
case Upsilon:
case GetArgument:
+ case ExtractOSREntryLocal:
RELEASE_ASSERT_NOT_REACHED();
break;
}
diff --git a/Source/JavaScriptCore/dfg/DFGTierUpCheckInjectionPhase.cpp b/Source/JavaScriptCore/dfg/DFGTierUpCheckInjectionPhase.cpp
new file mode 100644
index 0000000..d51a1f0
--- /dev/null
+++ b/Source/JavaScriptCore/dfg/DFGTierUpCheckInjectionPhase.cpp
@@ -0,0 +1,101 @@
+/*
+ * Copyright (C) 2013 Apple Inc. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in the
+ * documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. ``AS IS'' AND ANY
+ * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+ * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR
+ * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
+ * EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
+ * PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
+ * PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
+ * OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#include "config.h"
+#include "DFGTierUpCheckInjectionPhase.h"
+
+#if ENABLE(DFG_JIT)
+
+#include "DFGGraph.h"
+#include "DFGInsertionSet.h"
+#include "DFGPhase.h"
+#include "FTLCapabilities.h"
+#include "Operations.h"
+
+namespace JSC { namespace DFG {
+
+class TierUpCheckInjectionPhase : public Phase {
+public:
+ TierUpCheckInjectionPhase(Graph& graph)
+ : Phase(graph, "tier-up check injection")
+ {
+ }
+
+ bool run()
+ {
+ RELEASE_ASSERT(m_graph.m_plan.mode == DFGMode);
+
+ if (!Options::useExperimentalFTL())
+ return false;
+
+#if ENABLE(FTL_JIT)
+ FTL::CapabilityLevel level = FTL::canCompile(m_graph);
+ if (level == FTL::CannotCompile)
+ return false;
+
+ InsertionSet insertionSet(m_graph);
+ for (BlockIndex blockIndex = m_graph.numBlocks(); blockIndex--;) {
+ BasicBlock* block = m_graph.block(blockIndex);
+ if (!block)
+ continue;
+
+ if (block->at(0)->op() == LoopHint) {
+ CodeOrigin codeOrigin = block->at(0)->codeOrigin;
+ NodeType nodeType;
+ if (level == FTL::CanCompileAndOSREnter && !codeOrigin.inlineCallFrame) {
+ nodeType = CheckTierUpAndOSREnter;
+ RELEASE_ASSERT(block->bytecodeBegin == codeOrigin.bytecodeIndex);
+ } else
+ nodeType = CheckTierUpInLoop;
+ insertionSet.insertNode(1, SpecNone, nodeType, codeOrigin);
+ }
+
+ if (block->last()->op() == Return) {
+ insertionSet.insertNode(
+ block->size() - 1, SpecNone, CheckTierUpAtReturn, block->last()->codeOrigin);
+ }
+
+ insertionSet.execute(block);
+ }
+
+ return true;
+#else // ENABLE(FTL_JIT)
+ RELEASE_ASSERT_NOT_REACHED();
+ return false;
+#endif // ENABLE(FTL_JIT)
+ }
+};
+
+bool performTierUpCheckInjection(Graph& graph)
+{
+ SamplingRegion samplingRegion("DFG Tier-up Check Injection");
+ return runPhase<TierUpCheckInjectionPhase>(graph);
+}
+
+} } // namespace JSC::DFG
+
+#endif // ENABLE(DFG_JIT)
+
+
diff --git a/Source/JavaScriptCore/dfg/DFGTierUpCheckInjectionPhase.h b/Source/JavaScriptCore/dfg/DFGTierUpCheckInjectionPhase.h
new file mode 100644
index 0000000..f6e799a
--- /dev/null
+++ b/Source/JavaScriptCore/dfg/DFGTierUpCheckInjectionPhase.h
@@ -0,0 +1,47 @@
+/*
+ * Copyright (C) 2013 Apple Inc. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in the
+ * documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. ``AS IS'' AND ANY
+ * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+ * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR
+ * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
+ * EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
+ * PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
+ * PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
+ * OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#ifndef DFGTierUpCheckInjectionPhase_h
+#define DFGTierUpCheckInjectionPhase_h
+
+#include <wtf/Platform.h>
+
+#if ENABLE(DFG_JIT)
+
+namespace JSC { namespace DFG {
+
+class Graph;
+
+// This phase checks if the this code block could be recompiled with the FTL,
+// and if so, it injects tier-up checks.
+
+bool performTierUpCheckInjection(Graph&);
+
+} } // namespace JSC::DFG
+
+#endif // ENABLE(DFG_JIT)
+
+#endif // DFGTierUpCheckInjectionPhase_h
+
diff --git a/Source/JavaScriptCore/dfg/DFGToFTLDeferredCompilationCallback.cpp b/Source/JavaScriptCore/dfg/DFGToFTLDeferredCompilationCallback.cpp
new file mode 100644
index 0000000..1cb32d7
--- /dev/null
+++ b/Source/JavaScriptCore/dfg/DFGToFTLDeferredCompilationCallback.cpp
@@ -0,0 +1,82 @@
+/*
+ * Copyright (C) 2013 Apple Inc. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in the
+ * documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. ``AS IS'' AND ANY
+ * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+ * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR
+ * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
+ * EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
+ * PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
+ * PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
+ * OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#include "config.h"
+#include "DFGToFTLDeferredCompilationCallback.h"
+
+#if ENABLE(FTL_JIT)
+
+#include "CodeBlock.h"
+#include "DFGJITCode.h"
+#include "Executable.h"
+
+namespace JSC { namespace DFG {
+
+ToFTLDeferredCompilationCallback::ToFTLDeferredCompilationCallback(
+ PassRefPtr<CodeBlock> dfgCodeBlock)
+ : m_dfgCodeBlock(dfgCodeBlock)
+{
+}
+
+ToFTLDeferredCompilationCallback::~ToFTLDeferredCompilationCallback() { }
+
+PassRefPtr<ToFTLDeferredCompilationCallback> ToFTLDeferredCompilationCallback::create(
+ PassRefPtr<CodeBlock> dfgCodeBlock)
+{
+ return adoptRef(new ToFTLDeferredCompilationCallback(dfgCodeBlock));
+}
+
+void ToFTLDeferredCompilationCallback::compilationDidBecomeReadyAsynchronously(
+ CodeBlock* codeBlock)
+{
+ if (Options::verboseOSR()) {
+ dataLog(
+ "Optimizing compilation of ", *codeBlock, " (for ", *m_dfgCodeBlock,
+ ") did become ready.\n");
+ }
+
+ m_dfgCodeBlock->jitCode()->dfg()->forceOptimizationSlowPathConcurrently(
+ m_dfgCodeBlock.get());
+}
+
+void ToFTLDeferredCompilationCallback::compilationDidComplete(
+ CodeBlock* codeBlock, CompilationResult result)
+{
+ if (Options::verboseOSR()) {
+ dataLog(
+ "Optimizing compilation of ", *codeBlock, " (for ", *m_dfgCodeBlock,
+ ") result: ", result, "\n");
+ }
+
+ if (result == CompilationSuccessful)
+ codeBlock->install();
+
+ m_dfgCodeBlock->jitCode()->dfg()->setOptimizationThresholdBasedOnCompilationResult(
+ m_dfgCodeBlock.get(), result);
+}
+
+} } // JSC::DFG
+
+#endif // ENABLE(FTL_JIT)
diff --git a/Source/JavaScriptCore/dfg/DFGToFTLDeferredCompilationCallback.h b/Source/JavaScriptCore/dfg/DFGToFTLDeferredCompilationCallback.h
new file mode 100644
index 0000000..a4d840b
--- /dev/null
+++ b/Source/JavaScriptCore/dfg/DFGToFTLDeferredCompilationCallback.h
@@ -0,0 +1,65 @@
+/*
+ * Copyright (C) 2013 Apple Inc. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in the
+ * documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. ``AS IS'' AND ANY
+ * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+ * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR
+ * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
+ * EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
+ * PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
+ * PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
+ * OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#ifndef DFGToFTLDeferredCompilationCallback_h
+#define DFGToFTLDeferredCompilationCallback_h
+
+#include <wtf/Platform.h>
+
+#if ENABLE(FTL_JIT)
+
+#include "DeferredCompilationCallback.h"
+#include <wtf/PassRefPtr.h>
+#include <wtf/RefPtr.h>
+
+namespace JSC {
+
+class ScriptExecutable;
+
+namespace DFG {
+
+class ToFTLDeferredCompilationCallback : public DeferredCompilationCallback {
+protected:
+ ToFTLDeferredCompilationCallback(PassRefPtr<CodeBlock> dfgCodeBlock);
+
+public:
+ virtual ~ToFTLDeferredCompilationCallback();
+
+ static PassRefPtr<ToFTLDeferredCompilationCallback> create(
+ PassRefPtr<CodeBlock> dfgCodeBlock);
+
+ virtual void compilationDidBecomeReadyAsynchronously(CodeBlock*);
+ virtual void compilationDidComplete(CodeBlock*, CompilationResult);
+
+private:
+ RefPtr<CodeBlock> m_dfgCodeBlock;
+};
+
+} } // namespace JSC::DFG
+
+#endif // ENABLE(FTL_JIT)
+
+#endif // DFGToFTLDeferredCompilationCallback_h
+
diff --git a/Source/JavaScriptCore/dfg/DFGToFTLForOSREntryDeferredCompilationCallback.cpp b/Source/JavaScriptCore/dfg/DFGToFTLForOSREntryDeferredCompilationCallback.cpp
new file mode 100644
index 0000000..17b45a3
--- /dev/null
+++ b/Source/JavaScriptCore/dfg/DFGToFTLForOSREntryDeferredCompilationCallback.cpp
@@ -0,0 +1,87 @@
+/*
+ * Copyright (C) 2013 Apple Inc. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in the
+ * documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. ``AS IS'' AND ANY
+ * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+ * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR
+ * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
+ * EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
+ * PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
+ * PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
+ * OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#include "config.h"
+#include "DFGToFTLForOSREntryDeferredCompilationCallback.h"
+
+#if ENABLE(FTL_JIT)
+
+#include "CodeBlock.h"
+#include "DFGJITCode.h"
+#include "Executable.h"
+
+namespace JSC { namespace DFG {
+
+ToFTLForOSREntryDeferredCompilationCallback::ToFTLForOSREntryDeferredCompilationCallback(
+ PassRefPtr<CodeBlock> dfgCodeBlock)
+ : m_dfgCodeBlock(dfgCodeBlock)
+{
+}
+
+ToFTLForOSREntryDeferredCompilationCallback::~ToFTLForOSREntryDeferredCompilationCallback()
+{
+}
+
+PassRefPtr<ToFTLForOSREntryDeferredCompilationCallback>
+ToFTLForOSREntryDeferredCompilationCallback::create(
+ PassRefPtr<CodeBlock> dfgCodeBlock)
+{
+ return adoptRef(new ToFTLForOSREntryDeferredCompilationCallback(dfgCodeBlock));
+}
+
+void ToFTLForOSREntryDeferredCompilationCallback::compilationDidBecomeReadyAsynchronously(
+ CodeBlock* codeBlock)
+{
+ if (Options::verboseOSR()) {
+ dataLog(
+ "Optimizing compilation of ", *codeBlock, " (for ", *m_dfgCodeBlock,
+ ") did become ready.\n");
+ }
+
+ m_dfgCodeBlock->jitCode()->dfg()->forceOptimizationSlowPathConcurrently(
+ m_dfgCodeBlock.get());
+}
+
+void ToFTLForOSREntryDeferredCompilationCallback::compilationDidComplete(
+ CodeBlock* codeBlock, CompilationResult result)
+{
+ if (Options::verboseOSR()) {
+ dataLog(
+ "Optimizing compilation of ", *codeBlock, " (for ", *m_dfgCodeBlock,
+ ") result: ", result, "\n");
+ }
+
+ if (result == CompilationSuccessful)
+ m_dfgCodeBlock->jitCode()->dfg()->osrEntryBlock = codeBlock;
+
+ // FIXME: if we failed, we might want to just turn off OSR entry rather than
+ // totally turning off tier-up.
+ m_dfgCodeBlock->jitCode()->dfg()->setOptimizationThresholdBasedOnCompilationResult(
+ m_dfgCodeBlock.get(), result);
+}
+
+} } // JSC::DFG
+
+#endif // ENABLE(FTL_JIT)
diff --git a/Source/JavaScriptCore/dfg/DFGToFTLForOSREntryDeferredCompilationCallback.h b/Source/JavaScriptCore/dfg/DFGToFTLForOSREntryDeferredCompilationCallback.h
new file mode 100644
index 0000000..af6b97b
--- /dev/null
+++ b/Source/JavaScriptCore/dfg/DFGToFTLForOSREntryDeferredCompilationCallback.h
@@ -0,0 +1,65 @@
+/*
+ * Copyright (C) 2013 Apple Inc. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in the
+ * documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. ``AS IS'' AND ANY
+ * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+ * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR
+ * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
+ * EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
+ * PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
+ * PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
+ * OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#ifndef DFGToFTLForOSREntryDeferredCompilationCallback_h
+#define DFGToFTLForOSREntryDeferredCompilationCallback_h
+
+#include <wtf/Platform.h>
+
+#if ENABLE(FTL_JIT)
+
+#include "DeferredCompilationCallback.h"
+#include <wtf/PassRefPtr.h>
+#include <wtf/RefPtr.h>
+
+namespace JSC {
+
+class ScriptExecutable;
+
+namespace DFG {
+
+class ToFTLForOSREntryDeferredCompilationCallback : public DeferredCompilationCallback {
+protected:
+ ToFTLForOSREntryDeferredCompilationCallback(PassRefPtr<CodeBlock> dfgCodeBlock);
+
+public:
+ virtual ~ToFTLForOSREntryDeferredCompilationCallback();
+
+ static PassRefPtr<ToFTLForOSREntryDeferredCompilationCallback> create(
+ PassRefPtr<CodeBlock> dfgCodeBlock);
+
+ virtual void compilationDidBecomeReadyAsynchronously(CodeBlock*);
+ virtual void compilationDidComplete(CodeBlock*, CompilationResult);
+
+private:
+ RefPtr<CodeBlock> m_dfgCodeBlock;
+};
+
+} } // namespace JSC::DFG
+
+#endif // ENABLE(FTL_JIT)
+
+#endif // DFGToFTLForOSREntryDeferredCompilationCallback_h
+
diff --git a/Source/JavaScriptCore/dfg/DFGWorklist.cpp b/Source/JavaScriptCore/dfg/DFGWorklist.cpp
index 3f434be..98de6fa 100644
--- a/Source/JavaScriptCore/dfg/DFGWorklist.cpp
+++ b/Source/JavaScriptCore/dfg/DFGWorklist.cpp
@@ -277,8 +277,6 @@
Worklist* globalWorklist()
{
- if (!enableConcurrentJIT())
- return 0;
pthread_once(&initializeGlobalWorklistKeyOnce, initializeGlobalWorklistOnce);
return theGlobalWorklist;
}
diff --git a/Source/JavaScriptCore/dfg/DFGWorklist.h b/Source/JavaScriptCore/dfg/DFGWorklist.h
index f8f586d..208f246 100644
--- a/Source/JavaScriptCore/dfg/DFGWorklist.h
+++ b/Source/JavaScriptCore/dfg/DFGWorklist.h
@@ -98,9 +98,6 @@
// For now we use a single global worklist. It's not clear that this
// is the right thing to do, but it is what we do, for now. This function
// will lazily create one when it's needed.
-//
-// This returns null if for any reason we shouldn't be doing concurrent
-// compilation.
Worklist* globalWorklist();
} } // namespace JSC::DFG
