DFG should inline closure calls
https://bugs.webkit.org/show_bug.cgi?id=106067
Reviewed by Gavin Barraclough.
This adds initial support for inlining closure calls to the DFG. A call is considered
to be a closure call when the JSFunction* varies, but always has the same executable.
We already have closure call inline caching in both JITs, which works by checking that
the callee has an expected structure (as a cheap way of detecting that it is in fact
a JSFunction) and an expected executable. Closure call inlining uses profiling data
aggregated by CallLinkStatus to decide when to specialize the call to the particular
structure/executable, and inline the call rather than emitting a call sequence. When
we choose to do a closure inline rather than an ordinary inline, a number of things
change about how inlining is performed:
- The inline is guarded by a CheckStructure/CheckExecutable rather than a
CheckFunction.
- Instead of propagating a constant value for the scope, we emit GetMyScope every time
that the scope is needed, which loads the scope from a local variable. We do similar
things for the callee.
- The prologue of the inlined code includes SetMyScope and SetCallee nodes to eagerly
plant the scope and callee into the "true call frame", i.e. the place on the stack
where the call frame would have been if the call had been actually performed. This
allows GetMyScope/GetCallee to work as they would if the code wasn't inlined. It
also allows for trivial handling of scope and callee for call frame reconstruction
upon stack introspection and during OSR.
- A new node called GetScope is introduced, which just gets the scope of a function.
This node has the expected CSE support. This allows for the
SetMyScope(GetScope(@function)) sequence to set up the scope in the true call frame.
- GetMyScope/GetCallee CSE can match against SetMyScope/SetCallee, which means that
the GetMyScope/GetCallee nodes emitted during parsing are often removed during CSE,
if we can prove that it is safe to do so.
- Inlining heuristics are adjusted to grok the cost of inlining a closure. We are
less likely to inline a closure call than we are to inline a normal call, since we
end up emitting more code for closures due to CheckStructure, CheckExecutable,
GetScope, SetMyScope, and SetCallee.
Additionally, I've fixed the VariableEventStream to ensure that we don't attempt to
plant Undefined into the true call frames. This was previously a harmless oversight,
but it becomes quite bad if OSR is relying on the scope/callee already having been
set and not subsequently clobbered by the OSR itself.
This is a ~60% speed-up on programs that frequently make calls to closures. It's
neutral on V8v7 and other major benchmark suites.
The lack of a definite speed-up is likely due the fact that closure inlining currently
does not do any cardinality [1] optimizations. We don't observe when a closure was
constructed within its caller, and so used the scope from its caller; and furthermore
we have no facility to detect when the scope is single. All scoped variable accesses
are assumed to be multiple instead. A subsequent step will be to ensure that closure
call inlining will be single and loving it.
[1] Single and loving it: Must-alias analysis for higher-order languages. Suresh
Jagannathan, Peter Thiemann, Stephen Weeks, and Andrew Wright. In POPL '98.
* bytecode/CallLinkStatus.cpp:
(JSC::CallLinkStatus::dump):
* bytecode/CallLinkStatus.h:
(JSC::CallLinkStatus::isClosureCall):
(CallLinkStatus):
* bytecode/CodeBlock.cpp:
(JSC::CodeBlock::globalObjectFor):
(JSC):
* bytecode/CodeBlock.h:
(CodeBlock):
* bytecode/CodeOrigin.cpp:
(JSC::InlineCallFrame::dump):
* dfg/DFGAbstractState.cpp:
(JSC::DFG::AbstractState::execute):
* dfg/DFGByteCodeParser.cpp:
(ByteCodeParser):
(JSC::DFG::ByteCodeParser::handleCall):
(JSC::DFG::ByteCodeParser::emitFunctionChecks):
(JSC::DFG::ByteCodeParser::handleInlining):
* dfg/DFGCSEPhase.cpp:
(JSC::DFG::CSEPhase::pureCSE):
(CSEPhase):
(JSC::DFG::CSEPhase::getCalleeLoadElimination):
(JSC::DFG::CSEPhase::checkExecutableElimination):
(JSC::DFG::CSEPhase::getMyScopeLoadElimination):
(JSC::DFG::CSEPhase::performNodeCSE):
* dfg/DFGCapabilities.cpp:
(JSC::DFG::mightInlineFunctionForClosureCall):
* dfg/DFGCapabilities.h:
(DFG):
(JSC::DFG::mightInlineFunctionForClosureCall):
(JSC::DFG::canInlineFunctionForClosureCall):
(JSC::DFG::canInlineFunctionFor):
* dfg/DFGNode.h:
(Node):
(JSC::DFG::Node::hasExecutable):
(JSC::DFG::Node::executable):
* dfg/DFGNodeType.h:
(DFG):
* dfg/DFGPredictionPropagationPhase.cpp:
(JSC::DFG::PredictionPropagationPhase::propagate):
* dfg/DFGSpeculativeJIT32_64.cpp:
(JSC::DFG::SpeculativeJIT::compile):
* dfg/DFGSpeculativeJIT64.cpp:
(JSC::DFG::SpeculativeJIT::compile):
* dfg/DFGVariableEventStream.cpp:
(JSC::DFG::VariableEventStream::reconstruct):
* runtime/Options.h:
(JSC):
git-svn-id: http://svn.webkit.org/repository/webkit/trunk@138921 268f45cc-cd09-0410-ab3c-d52691b4dbfc
diff --git a/Source/JavaScriptCore/dfg/DFGAbstractState.cpp b/Source/JavaScriptCore/dfg/DFGAbstractState.cpp
index 65647a7..9585ae4 100644
--- a/Source/JavaScriptCore/dfg/DFGAbstractState.cpp
+++ b/Source/JavaScriptCore/dfg/DFGAbstractState.cpp
@@ -1382,7 +1382,13 @@
node.setCanExit(false);
forNode(nodeIndex).set(SpecFunction);
break;
+
+ case SetCallee:
+ case SetMyScope:
+ node.setCanExit(false);
+ break;
+ case GetScope: // FIXME: We could get rid of these if we know that the JSFunction is a constant. https://bugs.webkit.org/show_bug.cgi?id=106202
case GetMyScope:
case SkipTopScope:
node.setCanExit(false);
@@ -1455,6 +1461,16 @@
node.setCanExit(true); // Lies, but it's true for the common case of JSArray, so it's good enough.
forNode(nodeIndex).set(SpecInt32);
break;
+
+ case CheckExecutable: {
+ // FIXME: We could track executables in AbstractValue, which would allow us to get rid of these checks
+ // more thoroughly. https://bugs.webkit.org/show_bug.cgi?id=106200
+ // FIXME: We could eliminate these entirely if we know the exact value that flows into this.
+ // https://bugs.webkit.org/show_bug.cgi?id=106201
+ forNode(node.child1()).filter(SpecCell);
+ node.setCanExit(true);
+ break;
+ }
case CheckStructure:
case ForwardCheckStructure: {
diff --git a/Source/JavaScriptCore/dfg/DFGByteCodeParser.cpp b/Source/JavaScriptCore/dfg/DFGByteCodeParser.cpp
index af707e5..27a85c5 100644
--- a/Source/JavaScriptCore/dfg/DFGByteCodeParser.cpp
+++ b/Source/JavaScriptCore/dfg/DFGByteCodeParser.cpp
@@ -159,9 +159,9 @@
// Handle calls. This resolves issues surrounding inlining and intrinsics.
void handleCall(Interpreter*, Instruction* currentInstruction, NodeType op, CodeSpecializationKind);
- void emitFunctionCheck(JSFunction* expectedFunction, NodeIndex callTarget, int registerOffset, CodeSpecializationKind);
+ void emitFunctionChecks(const CallLinkStatus&, NodeIndex callTarget, int registerOffset, CodeSpecializationKind);
// Handle inlining. Return true if it succeeded, false if we need to plant a call.
- bool handleInlining(bool usesResult, NodeIndex callTargetNodeIndex, int resultOperand, bool certainAboutExpectedFunction, JSFunction*, int registerOffset, int argumentCountIncludingThis, unsigned nextOffset, CodeSpecializationKind);
+ bool handleInlining(bool usesResult, NodeIndex callTargetNodeIndex, int resultOperand, const CallLinkStatus&, int registerOffset, int argumentCountIncludingThis, unsigned nextOffset, CodeSpecializationKind);
// Handle setting the result of an intrinsic.
void setIntrinsicResult(bool usesResult, int resultOperand, NodeIndex);
// Handle intrinsic functions. Return true if it succeeded, false if we need to plant a call.
@@ -1329,19 +1329,9 @@
return;
}
- JSFunction* expectedFunction = callLinkStatus.function();
- if (!expectedFunction) {
- // For now we have no way of reasoning about what it means to not have a specific function. This will
- // change soon, though.
-
- addCall(interpreter, currentInstruction, op);
- return;
- }
-
Intrinsic intrinsic = callLinkStatus.intrinsicFor(kind);
if (intrinsic != NoIntrinsic) {
- if (!callLinkStatus.isProved())
- emitFunctionCheck(expectedFunction, callTarget, registerOffset, kind);
+ emitFunctionChecks(callLinkStatus, callTarget, registerOffset, kind);
if (handleIntrinsic(usesResult, resultOperand, intrinsic, registerOffset, argumentCountIncludingThis, prediction)) {
if (!callLinkStatus.isProved()) {
@@ -1350,34 +1340,48 @@
// smart enough to figure that out, since it doesn't understand CheckFunction.
addToGraph(Phantom, callTarget);
}
-
+
return;
}
- } else if (handleInlining(usesResult, callTarget, resultOperand, callLinkStatus.isProved(), expectedFunction, registerOffset, argumentCountIncludingThis, nextOffset, kind))
+ } else if (handleInlining(usesResult, callTarget, resultOperand, callLinkStatus, registerOffset, argumentCountIncludingThis, nextOffset, kind))
return;
addCall(interpreter, currentInstruction, op);
}
-void ByteCodeParser::emitFunctionCheck(JSFunction* expectedFunction, NodeIndex callTarget, int registerOffset, CodeSpecializationKind kind)
+void ByteCodeParser::emitFunctionChecks(const CallLinkStatus& callLinkStatus, NodeIndex callTarget, int registerOffset, CodeSpecializationKind kind)
{
+ if (callLinkStatus.isProved())
+ return;
+
+ ASSERT(callLinkStatus.canOptimize());
+
NodeIndex thisArgument;
if (kind == CodeForCall)
thisArgument = get(registerOffset + argumentToOperand(0));
else
thisArgument = NoNode;
- addToGraph(CheckFunction, OpInfo(expectedFunction), callTarget, thisArgument);
+
+ if (JSFunction* function = callLinkStatus.function())
+ addToGraph(CheckFunction, OpInfo(function), callTarget, thisArgument);
+ else {
+ ASSERT(callLinkStatus.structure());
+ ASSERT(callLinkStatus.executable());
+
+ addToGraph(CheckStructure, OpInfo(m_graph.addStructureSet(callLinkStatus.structure())), callTarget);
+ addToGraph(CheckExecutable, OpInfo(callLinkStatus.executable()), callTarget, thisArgument);
+ }
}
-bool ByteCodeParser::handleInlining(bool usesResult, NodeIndex callTargetNodeIndex, int resultOperand, bool certainAboutExpectedFunction, JSFunction* expectedFunction, int registerOffset, int argumentCountIncludingThis, unsigned nextOffset, CodeSpecializationKind kind)
+bool ByteCodeParser::handleInlining(bool usesResult, NodeIndex callTargetNodeIndex, int resultOperand, const CallLinkStatus& callLinkStatus, int registerOffset, int argumentCountIncludingThis, unsigned nextOffset, CodeSpecializationKind kind)
{
// First, the really simple checks: do we have an actual JS function?
- if (!expectedFunction)
+ if (!callLinkStatus.executable())
return false;
- if (expectedFunction->isHostFunction())
+ if (callLinkStatus.executable()->isHostFunction())
return false;
- FunctionExecutable* executable = expectedFunction->jsExecutable();
+ FunctionExecutable* executable = jsCast<FunctionExecutable*>(callLinkStatus.executable());
// Does the number of arguments we're passing match the arity of the target? We currently
// inline only if the number of arguments passed is greater than or equal to the number
@@ -1406,7 +1410,7 @@
CodeBlock* codeBlock = executable->baselineCodeBlockFor(kind);
if (!codeBlock)
return false;
- if (!canInlineFunctionFor(codeBlock, kind))
+ if (!canInlineFunctionFor(codeBlock, kind, callLinkStatus.isClosureCall()))
return false;
#if DFG_ENABLE(DEBUG_VERBOSE)
@@ -1416,8 +1420,7 @@
// Now we know without a doubt that we are committed to inlining. So begin the process
// by checking the callee (if necessary) and making sure that arguments and the callee
// are flushed.
- if (!certainAboutExpectedFunction)
- emitFunctionCheck(expectedFunction, callTargetNodeIndex, registerOffset, kind);
+ emitFunctionChecks(callLinkStatus, callTargetNodeIndex, registerOffset, kind);
// FIXME: Don't flush constants!
@@ -1439,7 +1442,7 @@
InlineStackEntry inlineStackEntry(
this, codeBlock, codeBlock, m_graph.m_blocks.size() - 1,
- expectedFunction, (VirtualRegister)m_inlineStackTop->remapOperand(
+ callLinkStatus.function(), (VirtualRegister)m_inlineStackTop->remapOperand(
usesResult ? resultOperand : InvalidVirtualRegister),
(VirtualRegister)inlineCallFrameStart, argumentCountIncludingThis, kind);
@@ -1450,6 +1453,10 @@
m_currentProfilingIndex = 0;
addToGraph(InlineStart, OpInfo(argumentPositionStart));
+ if (callLinkStatus.isClosureCall()) {
+ addToGraph(SetCallee, callTargetNodeIndex);
+ addToGraph(SetMyScope, addToGraph(GetScope, callTargetNodeIndex));
+ }
parseCodeBlock();
diff --git a/Source/JavaScriptCore/dfg/DFGCSEPhase.cpp b/Source/JavaScriptCore/dfg/DFGCSEPhase.cpp
index a9673f6..b7efb93 100644
--- a/Source/JavaScriptCore/dfg/DFGCSEPhase.cpp
+++ b/Source/JavaScriptCore/dfg/DFGCSEPhase.cpp
@@ -89,9 +89,7 @@
return result;
}
- enum InlineCallFrameRequirement { RequireSameInlineCallFrame, DoNotCareAboutInlineCallFrame };
- template<InlineCallFrameRequirement inlineCallFrameRequirement>
- NodeIndex genericPureCSE(Node& node)
+ NodeIndex pureCSE(Node& node)
{
NodeIndex child1 = canonicalize(node.child1());
NodeIndex child2 = canonicalize(node.child2());
@@ -112,10 +110,6 @@
if (node.arithNodeFlags() != otherNode.arithNodeFlags())
continue;
- if (inlineCallFrameRequirement == RequireSameInlineCallFrame
- && node.codeOrigin.inlineCallFrame != otherNode.codeOrigin.inlineCallFrame)
- continue;
-
NodeIndex otherChild = canonicalize(otherNode.child1());
if (otherChild == NoNode)
return index;
@@ -139,16 +133,6 @@
return NoNode;
}
- NodeIndex pureCSE(Node& node)
- {
- return genericPureCSE<DoNotCareAboutInlineCallFrame>(node);
- }
-
- NodeIndex pureCSERequiringSameInlineCallFrame(Node& node)
- {
- return genericPureCSE<RequireSameInlineCallFrame>(node);
- }
-
NodeIndex constantCSE(Node& node)
{
for (unsigned i = endIndexForPureCSE(); i--;) {
@@ -181,6 +165,27 @@
return NoNode;
}
+ NodeIndex getCalleeLoadElimination(InlineCallFrame* inlineCallFrame)
+ {
+ for (unsigned i = m_indexInBlock; i--;) {
+ NodeIndex index = m_currentBlock->at(i);
+ Node& node = m_graph[index];
+ if (!node.shouldGenerate())
+ continue;
+ if (node.codeOrigin.inlineCallFrame != inlineCallFrame)
+ continue;
+ switch (node.op()) {
+ case GetCallee:
+ return index;
+ case SetCallee:
+ return node.child1().index();
+ default:
+ break;
+ }
+ }
+ return NoNode;
+ }
+
NodeIndex getArrayLengthElimination(NodeIndex array)
{
for (unsigned i = m_indexInBlock; i--;) {
@@ -416,6 +421,20 @@
}
return false;
}
+
+ bool checkExecutableElimination(ExecutableBase* executable, NodeIndex child1)
+ {
+ for (unsigned i = endIndexForPureCSE(); i--;) {
+ NodeIndex index = m_currentBlock->at(i);
+ if (index == child1)
+ break;
+
+ Node& node = m_graph[index];
+ if (node.op() == CheckExecutable && node.child1() == child1 && node.executable() == executable)
+ return true;
+ }
+ return false;
+ }
bool checkStructureElimination(const StructureSet& structureSet, NodeIndex child1)
{
@@ -830,6 +849,8 @@
return NoNode;
case GetMyScope:
return index;
+ case SetMyScope:
+ return node.child1().index();
default:
break;
}
@@ -1144,11 +1165,12 @@
case SkipTopScope:
case SkipScope:
case GetScopeRegisters:
+ case GetScope:
setReplacement(pureCSE(node));
break;
case GetCallee:
- setReplacement(pureCSERequiringSameInlineCallFrame(node));
+ setReplacement(getCalleeLoadElimination(node.codeOrigin.inlineCallFrame));
break;
case GetLocal: {
@@ -1351,6 +1373,11 @@
eliminate();
break;
+ case CheckExecutable:
+ if (checkExecutableElimination(node.executable(), node.child1().index()))
+ eliminate();
+ break;
+
case CheckArray:
if (checkArrayElimination(node.child1().index(), node.arrayMode()))
eliminate();
diff --git a/Source/JavaScriptCore/dfg/DFGCapabilities.cpp b/Source/JavaScriptCore/dfg/DFGCapabilities.cpp
index 1765619..c5a4a5f 100644
--- a/Source/JavaScriptCore/dfg/DFGCapabilities.cpp
+++ b/Source/JavaScriptCore/dfg/DFGCapabilities.cpp
@@ -55,6 +55,11 @@
return codeBlock->instructionCount() <= Options::maximumFunctionForCallInlineCandidateInstructionCount()
&& !codeBlock->ownerExecutable()->needsActivation();
}
+bool mightInlineFunctionForClosureCall(CodeBlock* codeBlock)
+{
+ return codeBlock->instructionCount() <= Options::maximumFunctionForClosureCallInlineCandidateInstructionCount()
+ && !codeBlock->ownerExecutable()->needsActivation();
+}
bool mightInlineFunctionForConstruct(CodeBlock* codeBlock)
{
return codeBlock->instructionCount() <= Options::maximumFunctionForConstructInlineCandidateInstructionCount()
diff --git a/Source/JavaScriptCore/dfg/DFGCapabilities.h b/Source/JavaScriptCore/dfg/DFGCapabilities.h
index 70b9a5d..7d46102 100644
--- a/Source/JavaScriptCore/dfg/DFGCapabilities.h
+++ b/Source/JavaScriptCore/dfg/DFGCapabilities.h
@@ -44,6 +44,7 @@
bool mightCompileFunctionForCall(CodeBlock*);
bool mightCompileFunctionForConstruct(CodeBlock*);
bool mightInlineFunctionForCall(CodeBlock*);
+bool mightInlineFunctionForClosureCall(CodeBlock*);
bool mightInlineFunctionForConstruct(CodeBlock*);
// Opcode checking.
@@ -272,6 +273,7 @@
inline bool mightCompileFunctionForCall(CodeBlock*) { return false; }
inline bool mightCompileFunctionForConstruct(CodeBlock*) { return false; }
inline bool mightInlineFunctionForCall(CodeBlock*) { return false; }
+inline bool mightInlineFunctionForClosureCall(CodeBlock*) { return false; }
inline bool mightInlineFunctionForConstruct(CodeBlock*) { return false; }
inline CapabilityLevel canCompileOpcode(OpcodeID, CodeBlock*, Instruction*) { return CannotCompile; }
@@ -317,6 +319,11 @@
return mightInlineFunctionForCall(codeBlock) && canInlineOpcodes(codeBlock);
}
+inline bool canInlineFunctionForClosureCall(CodeBlock* codeBlock)
+{
+ return mightInlineFunctionForClosureCall(codeBlock) && canInlineOpcodes(codeBlock);
+}
+
inline bool canInlineFunctionForConstruct(CodeBlock* codeBlock)
{
return mightInlineFunctionForConstruct(codeBlock) && canInlineOpcodes(codeBlock);
@@ -330,8 +337,12 @@
return mightInlineFunctionForConstruct(codeBlock);
}
-inline bool canInlineFunctionFor(CodeBlock* codeBlock, CodeSpecializationKind kind)
+inline bool canInlineFunctionFor(CodeBlock* codeBlock, CodeSpecializationKind kind, bool isClosureCall)
{
+ if (isClosureCall) {
+ ASSERT(kind == CodeForCall);
+ return canInlineFunctionForClosureCall(codeBlock);
+ }
if (kind == CodeForCall)
return canInlineFunctionForCall(codeBlock);
ASSERT(kind == CodeForConstruct);
diff --git a/Source/JavaScriptCore/dfg/DFGNode.h b/Source/JavaScriptCore/dfg/DFGNode.h
index 2111356..e2751b5 100644
--- a/Source/JavaScriptCore/dfg/DFGNode.h
+++ b/Source/JavaScriptCore/dfg/DFGNode.h
@@ -702,6 +702,16 @@
ASSERT(JSValue(result).isFunction());
return result;
}
+
+ bool hasExecutable()
+ {
+ return op() == CheckExecutable;
+ }
+
+ ExecutableBase* executable()
+ {
+ return jsCast<ExecutableBase*>(reinterpret_cast<JSCell*>(m_opInfo));
+ }
bool hasStructureTransitionData()
{
diff --git a/Source/JavaScriptCore/dfg/DFGNodeType.h b/Source/JavaScriptCore/dfg/DFGNodeType.h
index 79dfc5d..1d8e780 100644
--- a/Source/JavaScriptCore/dfg/DFGNodeType.h
+++ b/Source/JavaScriptCore/dfg/DFGNodeType.h
@@ -52,6 +52,7 @@
macro(ConvertThis, NodeResultJS) \
macro(CreateThis, NodeResultJS) /* Note this is not MustGenerate since we're returning it anyway. */ \
macro(GetCallee, NodeResultJS) \
+ macro(SetCallee, NodeMustGenerate) \
\
/* Nodes for local variable access. These nodes are linked together using Phi nodes. */\
/* Any two nodes that are part of the same Phi graph will share the same */\
@@ -124,6 +125,7 @@
macro(PutById, NodeMustGenerate | NodeClobbersWorld) \
macro(PutByIdDirect, NodeMustGenerate | NodeClobbersWorld) \
macro(CheckStructure, NodeMustGenerate) \
+ macro(CheckExecutable, NodeMustGenerate) \
macro(ForwardCheckStructure, NodeMustGenerate) \
/* Transition watchpoints are a contract between the party setting the watchpoint */\
/* and the runtime system, where the party promises that the child object once had */\
@@ -150,7 +152,9 @@
macro(GetByOffset, NodeResultJS) \
macro(PutByOffset, NodeMustGenerate) \
macro(GetArrayLength, NodeResultInt32) \
+ macro(GetScope, NodeResultJS) \
macro(GetMyScope, NodeResultJS) \
+ macro(SetMyScope, NodeMustGenerate) \
macro(SkipTopScope, NodeResultJS) \
macro(SkipScope, NodeResultJS) \
macro(GetScopeRegisters, NodeResultStorage) \
diff --git a/Source/JavaScriptCore/dfg/DFGPredictionPropagationPhase.cpp b/Source/JavaScriptCore/dfg/DFGPredictionPropagationPhase.cpp
index 9c21fcb..2f664f3 100644
--- a/Source/JavaScriptCore/dfg/DFGPredictionPropagationPhase.cpp
+++ b/Source/JavaScriptCore/dfg/DFGPredictionPropagationPhase.cpp
@@ -767,6 +767,16 @@
case Phi:
break;
+ case SetCallee:
+ case SetMyScope:
+ changed |= m_graph[node.child1()].mergeFlags(NodeUsedAsValue);
+ break;
+
+ case GetScope:
+ changed |= m_graph[node.child1()].mergeFlags(NodeUsedAsValue);
+ changed |= setPrediction(SpecCellOther);
+ break;
+
#ifndef NDEBUG
// These get ignored because they don't return anything.
case DFG::Jump:
@@ -777,6 +787,7 @@
case ForceOSRExit:
case SetArgument:
case CheckStructure:
+ case CheckExecutable:
case ForwardCheckStructure:
case StructureTransitionWatchpoint:
case ForwardStructureTransitionWatchpoint:
diff --git a/Source/JavaScriptCore/dfg/DFGSpeculativeJIT32_64.cpp b/Source/JavaScriptCore/dfg/DFGSpeculativeJIT32_64.cpp
index 86ef3f1..b350321 100644
--- a/Source/JavaScriptCore/dfg/DFGSpeculativeJIT32_64.cpp
+++ b/Source/JavaScriptCore/dfg/DFGSpeculativeJIT32_64.cpp
@@ -3969,6 +3969,21 @@
break;
}
+ case SetCallee: {
+ SpeculateCellOperand callee(this, node.child1());
+ m_jit.storePtr(callee.gpr(), JITCompiler::payloadFor(static_cast<VirtualRegister>(node.codeOrigin.stackOffset() + static_cast<int>(JSStack::Callee))));
+ noResult(m_compileIndex);
+ break;
+ }
+
+ case GetScope: {
+ SpeculateCellOperand function(this, node.child1());
+ GPRTemporary result(this, function);
+ m_jit.loadPtr(JITCompiler::Address(function.gpr(), JSFunction::offsetOfScopeChain()), result.gpr());
+ cellResult(result.gpr(), m_compileIndex);
+ break;
+ }
+
case GetMyScope: {
GPRTemporary result(this);
GPRReg resultGPR = result.gpr();
@@ -3978,6 +3993,13 @@
break;
}
+ case SetMyScope: {
+ SpeculateCellOperand callee(this, node.child1());
+ m_jit.storePtr(callee.gpr(), JITCompiler::payloadFor(static_cast<VirtualRegister>(node.codeOrigin.stackOffset() + static_cast<int>(JSStack::ScopeChain))));
+ noResult(m_compileIndex);
+ break;
+ }
+
case SkipTopScope: {
SpeculateCellOperand scope(this, node.child1());
GPRTemporary result(this, scope);
@@ -4142,6 +4164,13 @@
break;
}
+ case CheckExecutable: {
+ SpeculateCellOperand function(this, node.child1());
+ speculationCheck(BadExecutable, JSValueSource::unboxedCell(function.gpr()), node.child1(), m_jit.branchWeakPtr(JITCompiler::NotEqual, JITCompiler::Address(function.gpr(), JSFunction::offsetOfExecutable()), node.executable()));
+ noResult(m_compileIndex);
+ break;
+ }
+
case CheckStructure:
case ForwardCheckStructure: {
AbstractValue& value = m_state.forNode(node.child1());
diff --git a/Source/JavaScriptCore/dfg/DFGSpeculativeJIT64.cpp b/Source/JavaScriptCore/dfg/DFGSpeculativeJIT64.cpp
index 566558d..39c6331 100644
--- a/Source/JavaScriptCore/dfg/DFGSpeculativeJIT64.cpp
+++ b/Source/JavaScriptCore/dfg/DFGSpeculativeJIT64.cpp
@@ -3921,6 +3921,21 @@
break;
}
+ case SetCallee: {
+ SpeculateCellOperand callee(this, node.child1());
+ m_jit.storePtr(callee.gpr(), JITCompiler::addressFor(static_cast<VirtualRegister>(node.codeOrigin.stackOffset() + static_cast<int>(JSStack::Callee))));
+ noResult(m_compileIndex);
+ break;
+ }
+
+ case GetScope: {
+ SpeculateCellOperand function(this, node.child1());
+ GPRTemporary result(this, function);
+ m_jit.loadPtr(JITCompiler::Address(function.gpr(), JSFunction::offsetOfScopeChain()), result.gpr());
+ cellResult(result.gpr(), m_compileIndex);
+ break;
+ }
+
case GetMyScope: {
GPRTemporary result(this);
GPRReg resultGPR = result.gpr();
@@ -3930,6 +3945,13 @@
break;
}
+ case SetMyScope: {
+ SpeculateCellOperand callee(this, node.child1());
+ m_jit.storePtr(callee.gpr(), JITCompiler::addressFor(static_cast<VirtualRegister>(node.codeOrigin.stackOffset() + static_cast<int>(JSStack::ScopeChain))));
+ noResult(m_compileIndex);
+ break;
+ }
+
case SkipTopScope: {
SpeculateCellOperand scope(this, node.child1());
GPRTemporary result(this, scope);
@@ -4080,6 +4102,14 @@
noResult(m_compileIndex);
break;
}
+
+ case CheckExecutable: {
+ SpeculateCellOperand function(this, node.child1());
+ speculationCheck(BadExecutable, JSValueRegs(function.gpr()), node.child1(), m_jit.branchWeakPtr(JITCompiler::NotEqual, JITCompiler::Address(function.gpr(), JSFunction::offsetOfExecutable()), node.executable()));
+ noResult(m_compileIndex);
+ break;
+ }
+
case CheckStructure:
case ForwardCheckStructure: {
AbstractValue& value = m_state.forNode(node.child1());
diff --git a/Source/JavaScriptCore/dfg/DFGVariableEventStream.cpp b/Source/JavaScriptCore/dfg/DFGVariableEventStream.cpp
index 7fa109b..d8652af 100644
--- a/Source/JavaScriptCore/dfg/DFGVariableEventStream.cpp
+++ b/Source/JavaScriptCore/dfg/DFGVariableEventStream.cpp
@@ -282,6 +282,13 @@
valueRecoveries[i] =
ValueRecovery::displacedInJSStack(static_cast<VirtualRegister>(info->u.virtualReg), info->format);
}
+
+ // Step 5: Make sure that for locals that coincide with true call frame headers, the exit compiler knows
+ // that those values don't have to be recovered. Signal this by using ValueRecovery::alreadyInJSStack()
+ for (InlineCallFrame* inlineCallFrame = codeOrigin.inlineCallFrame; inlineCallFrame; inlineCallFrame = inlineCallFrame->caller.inlineCallFrame) {
+ for (unsigned i = JSStack::CallFrameHeaderSize; i--;)
+ valueRecoveries.setLocal(inlineCallFrame->stackOffset - i - 1, ValueRecovery::alreadyInJSStack());
+ }
}
} } // namespace JSC::DFG
