DFG int-to-double conversion should be revealed to CSE
https://bugs.webkit.org/show_bug.cgi?id=82135
Reviewed by Oliver Hunt.
This introduces the notion of an Int32ToDouble node, which is injected
into the graph anytime we know that we have a double use of a node that
was predicted integer. The Int32ToDouble simplifies double speculation
on integers by skipping the path that would unbox doubles, if we know
that the value is already proven to be an integer. It allows integer to
double conversions to be subjected to common subexpression elimination
(CSE) by allowing the CSE phase to see where these conversions are
occurring. Finally, it allows us to see when a constant is being used
as both a double and an integer. This is a bit odd, since it means that
sometimes a double use of a constant will not refer directly to the
constant. This should not cause problems, for now, but it may require
some canonizalization in the future if we want to support strength
reductions of double operations based on constants.
To allow injection of nodes into the graph, this change introduces the
DFG::InsertionSet, which is a way of lazily inserting elements into a
list. This allows the FixupPhase to remain O(N) despite performing
multiple injections in a single basic block. Without the InsertionSet,
each injection would require performing an insertion into a vector,
which is O(N), leading to O(N^2) performance overall. With the
InsertionSet, each injection simply records what insertion would have
been performed, and all insertions are performed at once (via
InsertionSet::execute) after processing of a basic block is completed.
* JavaScriptCore.xcodeproj/project.pbxproj:
* bytecode/PredictedType.h:
(JSC::isActionableIntMutableArrayPrediction):
(JSC):
(JSC::isActionableFloatMutableArrayPrediction):
(JSC::isActionableTypedMutableArrayPrediction):
(JSC::isActionableMutableArrayPrediction):
* dfg/DFGAbstractState.cpp:
(JSC::DFG::AbstractState::execute):
* dfg/DFGCSEPhase.cpp:
(JSC::DFG::CSEPhase::performNodeCSE):
* dfg/DFGCommon.h:
(JSC::DFG::useKindToString):
(DFG):
* dfg/DFGFixupPhase.cpp:
(JSC::DFG::FixupPhase::run):
(JSC::DFG::FixupPhase::fixupBlock):
(FixupPhase):
(JSC::DFG::FixupPhase::fixupNode):
(JSC::DFG::FixupPhase::fixDoubleEdge):
* dfg/DFGGraph.cpp:
(JSC::DFG::Graph::dump):
* dfg/DFGInsertionSet.h: Added.
(DFG):
(Insertion):
(JSC::DFG::Insertion::Insertion):
(JSC::DFG::Insertion::index):
(JSC::DFG::Insertion::element):
(InsertionSet):
(JSC::DFG::InsertionSet::InsertionSet):
(JSC::DFG::InsertionSet::append):
(JSC::DFG::InsertionSet::execute):
* dfg/DFGNodeType.h:
(DFG):
* dfg/DFGPredictionPropagationPhase.cpp:
(JSC::DFG::PredictionPropagationPhase::propagate):
* dfg/DFGSpeculativeJIT.cpp:
(JSC::DFG::SpeculativeJIT::computeValueRecoveryFor):
(JSC::DFG::SpeculativeJIT::compileValueToInt32):
(JSC::DFG::SpeculativeJIT::compileInt32ToDouble):
(DFG):
* dfg/DFGSpeculativeJIT.h:
(SpeculativeJIT):
(JSC::DFG::IntegerOperand::IntegerOperand):
(JSC::DFG::DoubleOperand::DoubleOperand):
(JSC::DFG::JSValueOperand::JSValueOperand):
(JSC::DFG::StorageOperand::StorageOperand):
(JSC::DFG::SpeculateIntegerOperand::SpeculateIntegerOperand):
(JSC::DFG::SpeculateStrictInt32Operand::SpeculateStrictInt32Operand):
(JSC::DFG::SpeculateDoubleOperand::SpeculateDoubleOperand):
(JSC::DFG::SpeculateCellOperand::SpeculateCellOperand):
(JSC::DFG::SpeculateBooleanOperand::SpeculateBooleanOperand):
* dfg/DFGSpeculativeJIT32_64.cpp:
(JSC::DFG::SpeculativeJIT::compile):
* dfg/DFGSpeculativeJIT64.cpp:
(JSC::DFG::SpeculativeJIT::compile):
git-svn-id: http://svn.webkit.org/repository/webkit/trunk@112040 268f45cc-cd09-0410-ab3c-d52691b4dbfc
diff --git a/Source/JavaScriptCore/ChangeLog b/Source/JavaScriptCore/ChangeLog
index b4133f1..a3c8850 100644
--- a/Source/JavaScriptCore/ChangeLog
+++ b/Source/JavaScriptCore/ChangeLog
@@ -1,5 +1,93 @@
2012-03-25 Filip Pizlo <fpizlo@apple.com>
+ DFG int-to-double conversion should be revealed to CSE
+ https://bugs.webkit.org/show_bug.cgi?id=82135
+
+ Reviewed by Oliver Hunt.
+
+ This introduces the notion of an Int32ToDouble node, which is injected
+ into the graph anytime we know that we have a double use of a node that
+ was predicted integer. The Int32ToDouble simplifies double speculation
+ on integers by skipping the path that would unbox doubles, if we know
+ that the value is already proven to be an integer. It allows integer to
+ double conversions to be subjected to common subexpression elimination
+ (CSE) by allowing the CSE phase to see where these conversions are
+ occurring. Finally, it allows us to see when a constant is being used
+ as both a double and an integer. This is a bit odd, since it means that
+ sometimes a double use of a constant will not refer directly to the
+ constant. This should not cause problems, for now, but it may require
+ some canonizalization in the future if we want to support strength
+ reductions of double operations based on constants.
+
+ To allow injection of nodes into the graph, this change introduces the
+ DFG::InsertionSet, which is a way of lazily inserting elements into a
+ list. This allows the FixupPhase to remain O(N) despite performing
+ multiple injections in a single basic block. Without the InsertionSet,
+ each injection would require performing an insertion into a vector,
+ which is O(N), leading to O(N^2) performance overall. With the
+ InsertionSet, each injection simply records what insertion would have
+ been performed, and all insertions are performed at once (via
+ InsertionSet::execute) after processing of a basic block is completed.
+
+ * JavaScriptCore.xcodeproj/project.pbxproj:
+ * bytecode/PredictedType.h:
+ (JSC::isActionableIntMutableArrayPrediction):
+ (JSC):
+ (JSC::isActionableFloatMutableArrayPrediction):
+ (JSC::isActionableTypedMutableArrayPrediction):
+ (JSC::isActionableMutableArrayPrediction):
+ * dfg/DFGAbstractState.cpp:
+ (JSC::DFG::AbstractState::execute):
+ * dfg/DFGCSEPhase.cpp:
+ (JSC::DFG::CSEPhase::performNodeCSE):
+ * dfg/DFGCommon.h:
+ (JSC::DFG::useKindToString):
+ (DFG):
+ * dfg/DFGFixupPhase.cpp:
+ (JSC::DFG::FixupPhase::run):
+ (JSC::DFG::FixupPhase::fixupBlock):
+ (FixupPhase):
+ (JSC::DFG::FixupPhase::fixupNode):
+ (JSC::DFG::FixupPhase::fixDoubleEdge):
+ * dfg/DFGGraph.cpp:
+ (JSC::DFG::Graph::dump):
+ * dfg/DFGInsertionSet.h: Added.
+ (DFG):
+ (Insertion):
+ (JSC::DFG::Insertion::Insertion):
+ (JSC::DFG::Insertion::index):
+ (JSC::DFG::Insertion::element):
+ (InsertionSet):
+ (JSC::DFG::InsertionSet::InsertionSet):
+ (JSC::DFG::InsertionSet::append):
+ (JSC::DFG::InsertionSet::execute):
+ * dfg/DFGNodeType.h:
+ (DFG):
+ * dfg/DFGPredictionPropagationPhase.cpp:
+ (JSC::DFG::PredictionPropagationPhase::propagate):
+ * dfg/DFGSpeculativeJIT.cpp:
+ (JSC::DFG::SpeculativeJIT::computeValueRecoveryFor):
+ (JSC::DFG::SpeculativeJIT::compileValueToInt32):
+ (JSC::DFG::SpeculativeJIT::compileInt32ToDouble):
+ (DFG):
+ * dfg/DFGSpeculativeJIT.h:
+ (SpeculativeJIT):
+ (JSC::DFG::IntegerOperand::IntegerOperand):
+ (JSC::DFG::DoubleOperand::DoubleOperand):
+ (JSC::DFG::JSValueOperand::JSValueOperand):
+ (JSC::DFG::StorageOperand::StorageOperand):
+ (JSC::DFG::SpeculateIntegerOperand::SpeculateIntegerOperand):
+ (JSC::DFG::SpeculateStrictInt32Operand::SpeculateStrictInt32Operand):
+ (JSC::DFG::SpeculateDoubleOperand::SpeculateDoubleOperand):
+ (JSC::DFG::SpeculateCellOperand::SpeculateCellOperand):
+ (JSC::DFG::SpeculateBooleanOperand::SpeculateBooleanOperand):
+ * dfg/DFGSpeculativeJIT32_64.cpp:
+ (JSC::DFG::SpeculativeJIT::compile):
+ * dfg/DFGSpeculativeJIT64.cpp:
+ (JSC::DFG::SpeculativeJIT::compile):
+
+2012-03-25 Filip Pizlo <fpizlo@apple.com>
+
DFGOperands should be moved out of the DFG and into bytecode
https://bugs.webkit.org/show_bug.cgi?id=82151
diff --git a/Source/JavaScriptCore/JavaScriptCore.xcodeproj/project.pbxproj b/Source/JavaScriptCore/JavaScriptCore.xcodeproj/project.pbxproj
index 7ad9743..ba2a994 100644
--- a/Source/JavaScriptCore/JavaScriptCore.xcodeproj/project.pbxproj
+++ b/Source/JavaScriptCore/JavaScriptCore.xcodeproj/project.pbxproj
@@ -74,6 +74,7 @@
0F242DA713F3B1E8007ADD4C /* WeakReferenceHarvester.h in Headers */ = {isa = PBXBuildFile; fileRef = 0F242DA513F3B1BB007ADD4C /* WeakReferenceHarvester.h */; settings = {ATTRIBUTES = (Private, ); }; };
0F2BDC15151C5D4D00CD8910 /* DFGFixupPhase.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 0F2BDC12151C5D4A00CD8910 /* DFGFixupPhase.cpp */; };
0F2BDC16151C5D4F00CD8910 /* DFGFixupPhase.h in Headers */ = {isa = PBXBuildFile; fileRef = 0F2BDC13151C5D4A00CD8910 /* DFGFixupPhase.h */; settings = {ATTRIBUTES = (Private, ); }; };
+ 0F2BDC21151E803B00CD8910 /* DFGInsertionSet.h in Headers */ = {isa = PBXBuildFile; fileRef = 0F2BDC1F151E803800CD8910 /* DFGInsertionSet.h */; settings = {ATTRIBUTES = (Private, ); }; };
0F2BDC2C151FDE9100CD8910 /* Operands.h in Headers */ = {isa = PBXBuildFile; fileRef = 0F2BDC2B151FDE8B00CD8910 /* Operands.h */; settings = {ATTRIBUTES = (Private, ); }; };
0F2C556F14738F3100121E4F /* DFGCodeBlocks.h in Headers */ = {isa = PBXBuildFile; fileRef = 0F2C556E14738F2E00121E4F /* DFGCodeBlocks.h */; settings = {ATTRIBUTES = (Private, ); }; };
0F2C557014738F3500121E4F /* DFGCodeBlocks.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 0F2C556D14738F2E00121E4F /* DFGCodeBlocks.cpp */; };
@@ -724,6 +725,7 @@
0F242DA513F3B1BB007ADD4C /* WeakReferenceHarvester.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = WeakReferenceHarvester.h; sourceTree = "<group>"; };
0F2BDC12151C5D4A00CD8910 /* DFGFixupPhase.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; name = DFGFixupPhase.cpp; path = dfg/DFGFixupPhase.cpp; sourceTree = "<group>"; };
0F2BDC13151C5D4A00CD8910 /* DFGFixupPhase.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; name = DFGFixupPhase.h; path = dfg/DFGFixupPhase.h; sourceTree = "<group>"; };
+ 0F2BDC1F151E803800CD8910 /* DFGInsertionSet.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; name = DFGInsertionSet.h; path = dfg/DFGInsertionSet.h; sourceTree = "<group>"; };
0F2BDC2B151FDE8B00CD8910 /* Operands.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = Operands.h; sourceTree = "<group>"; };
0F2C556D14738F2E00121E4F /* DFGCodeBlocks.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = DFGCodeBlocks.cpp; sourceTree = "<group>"; };
0F2C556E14738F2E00121E4F /* DFGCodeBlocks.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = DFGCodeBlocks.h; sourceTree = "<group>"; };
@@ -1994,6 +1996,7 @@
86AE6C4C136A11E400963012 /* DFGGPRInfo.h */,
86EC9DB71328DF82002B2AD7 /* DFGGraph.cpp */,
86EC9DB81328DF82002B2AD7 /* DFGGraph.h */,
+ 0F2BDC1F151E803800CD8910 /* DFGInsertionSet.h */,
86EC9DBB1328DF82002B2AD7 /* DFGJITCompiler.cpp */,
86EC9DBC1328DF82002B2AD7 /* DFGJITCompiler.h */,
86ECA3E9132DEF1C002B2AD7 /* DFGNode.h */,
@@ -2500,6 +2503,7 @@
0FA581BB150E953000B9A2D9 /* DFGNodeFlags.h in Headers */,
0FA581BC150E953000B9A2D9 /* DFGNodeType.h in Headers */,
0F2BDC16151C5D4F00CD8910 /* DFGFixupPhase.h in Headers */,
+ 0F2BDC21151E803B00CD8910 /* DFGInsertionSet.h in Headers */,
0F2BDC2C151FDE9100CD8910 /* Operands.h in Headers */,
);
runOnlyForDeploymentPostprocessing = 0;
diff --git a/Source/JavaScriptCore/bytecode/PredictedType.h b/Source/JavaScriptCore/bytecode/PredictedType.h
index f99d0ad..dece67c 100644
--- a/Source/JavaScriptCore/bytecode/PredictedType.h
+++ b/Source/JavaScriptCore/bytecode/PredictedType.h
@@ -159,10 +159,9 @@
return value == PredictFloat64Array;
}
-inline bool isActionableMutableArrayPrediction(PredictedType value)
+inline bool isActionableIntMutableArrayPrediction(PredictedType value)
{
- return isArrayPrediction(value)
- || isByteArrayPrediction(value)
+ return isByteArrayPrediction(value)
#if CPU(X86) || CPU(X86_64)
|| isInt8ArrayPrediction(value)
|| isInt16ArrayPrediction(value)
@@ -171,13 +170,30 @@
|| isUint8ArrayPrediction(value)
|| isUint8ClampedArrayPrediction(value)
|| isUint16ArrayPrediction(value)
- || isUint32ArrayPrediction(value)
+ || isUint32ArrayPrediction(value);
+}
+
+inline bool isActionableFloatMutableArrayPrediction(PredictedType value)
+{
+ return false
#if CPU(X86) || CPU(X86_64)
|| isFloat32ArrayPrediction(value)
#endif
|| isFloat64ArrayPrediction(value);
}
+inline bool isActionableTypedMutableArrayPrediction(PredictedType value)
+{
+ return isActionableIntMutableArrayPrediction(value)
+ || isActionableFloatMutableArrayPrediction(value);
+}
+
+inline bool isActionableMutableArrayPrediction(PredictedType value)
+{
+ return isArrayPrediction(value)
+ || isActionableTypedMutableArrayPrediction(value);
+}
+
inline bool isActionableArrayPrediction(PredictedType value)
{
return isStringPrediction(value)
diff --git a/Source/JavaScriptCore/dfg/DFGAbstractState.cpp b/Source/JavaScriptCore/dfg/DFGAbstractState.cpp
index c8d0f2b..2f1e023 100644
--- a/Source/JavaScriptCore/dfg/DFGAbstractState.cpp
+++ b/Source/JavaScriptCore/dfg/DFGAbstractState.cpp
@@ -300,6 +300,11 @@
forNode(nodeIndex).set(PredictInt32);
break;
+
+ case Int32ToDouble:
+ forNode(node.child1()).filter(PredictNumber);
+ forNode(nodeIndex).set(PredictDouble);
+ break;
case ValueAdd:
case ArithAdd: {
diff --git a/Source/JavaScriptCore/dfg/DFGCSEPhase.cpp b/Source/JavaScriptCore/dfg/DFGCSEPhase.cpp
index 72541ed..bf1a82d 100644
--- a/Source/JavaScriptCore/dfg/DFGCSEPhase.cpp
+++ b/Source/JavaScriptCore/dfg/DFGCSEPhase.cpp
@@ -587,6 +587,7 @@
case GetStringLength:
case StringCharAt:
case StringCharCodeAt:
+ case Int32ToDouble:
setReplacement(pureCSE(node));
break;
diff --git a/Source/JavaScriptCore/dfg/DFGCommon.h b/Source/JavaScriptCore/dfg/DFGCommon.h
index c415409..6d7623f 100644
--- a/Source/JavaScriptCore/dfg/DFGCommon.h
+++ b/Source/JavaScriptCore/dfg/DFGCommon.h
@@ -97,9 +97,23 @@
enum UseKind {
UntypedUse,
+ DoubleUse,
LastUseKind // Must always be the last entry in the enum, as it is used to denote the number of enum elements.
};
+inline const char* useKindToString(UseKind useKind)
+{
+ switch (useKind) {
+ case UntypedUse:
+ return "";
+ case DoubleUse:
+ return "d";
+ default:
+ ASSERT_NOT_REACHED();
+ return 0;
+ }
+}
+
} } // namespace JSC::DFG
#endif // ENABLE(DFG_JIT)
diff --git a/Source/JavaScriptCore/dfg/DFGFixupPhase.cpp b/Source/JavaScriptCore/dfg/DFGFixupPhase.cpp
index 6579d82..b704996 100644
--- a/Source/JavaScriptCore/dfg/DFGFixupPhase.cpp
+++ b/Source/JavaScriptCore/dfg/DFGFixupPhase.cpp
@@ -29,6 +29,7 @@
#if ENABLE(DFG_JIT)
#include "DFGGraph.h"
+#include "DFGInsertionSet.h"
#include "DFGPhase.h"
namespace JSC { namespace DFG {
@@ -42,11 +43,20 @@
void run()
{
- for (m_compileIndex = 0; m_compileIndex < m_graph.size(); ++m_compileIndex)
- fixupNode(m_graph[m_compileIndex]);
+ for (BlockIndex blockIndex = 0; blockIndex < m_graph.m_blocks.size(); ++blockIndex)
+ fixupBlock(m_graph.m_blocks[blockIndex].get());
}
private:
+ void fixupBlock(BasicBlock* block)
+ {
+ for (m_indexInBlock = 0; m_indexInBlock < block->size(); ++m_indexInBlock) {
+ m_compileIndex = block->at(m_indexInBlock);
+ fixupNode(m_graph[m_compileIndex]);
+ }
+ m_insertionSet.execute(*block);
+ }
+
void fixupNode(Node& node)
{
if (!node.shouldGenerate())
@@ -152,11 +162,130 @@
break;
}
+ case CompareEq:
+ case CompareLess:
+ case CompareLessEq:
+ case CompareGreater:
+ case CompareGreaterEq:
+ case CompareStrictEq: {
+ if (Node::shouldSpeculateInteger(m_graph[node.child1()], m_graph[node.child2()]))
+ break;
+ if (!Node::shouldSpeculateNumber(m_graph[node.child1()], m_graph[node.child2()]))
+ break;
+ fixDoubleEdge(0);
+ fixDoubleEdge(1);
+ break;
+ }
+
+ case LogicalNot: {
+ if (m_graph[node.child1()].shouldSpeculateInteger())
+ break;
+ if (!m_graph[node.child1()].shouldSpeculateNumber())
+ break;
+ fixDoubleEdge(0);
+ break;
+ }
+
+ case Branch: {
+ if (m_graph[node.child1()].shouldSpeculateInteger())
+ break;
+ if (!m_graph[node.child1()].shouldSpeculateNumber())
+ break;
+ fixDoubleEdge(0);
+ break;
+ }
+
+ case SetLocal: {
+ if (m_graph.isCaptured(node.local()))
+ break;
+ if (!node.variableAccessData()->shouldUseDoubleFormat())
+ break;
+ fixDoubleEdge(0);
+ break;
+ }
+
+ case ArithAdd:
+ case ValueAdd: {
+ if (m_graph.addShouldSpeculateInteger(node))
+ break;
+ if (!Node::shouldSpeculateNumber(m_graph[node.child1()], m_graph[node.child2()]))
+ break;
+ fixDoubleEdge(0);
+ fixDoubleEdge(1);
+ break;
+ }
+
+ case ArithSub: {
+ if (m_graph.addShouldSpeculateInteger(node)
+ && node.canSpeculateInteger())
+ break;
+ fixDoubleEdge(0);
+ fixDoubleEdge(1);
+ break;
+ }
+
+ case ArithNegate: {
+ if (m_graph.negateShouldSpeculateInteger(node))
+ break;
+ fixDoubleEdge(0);
+ break;
+ }
+
+ case ArithMin:
+ case ArithMax:
+ case ArithMul:
+ case ArithDiv:
+ case ArithMod: {
+ if (Node::shouldSpeculateInteger(m_graph[node.child1()], m_graph[node.child2()])
+ && node.canSpeculateInteger())
+ break;
+ fixDoubleEdge(0);
+ fixDoubleEdge(1);
+ break;
+ }
+
+ case ArithAbs: {
+ if (m_graph[node.child1()].shouldSpeculateInteger()
+ && node.canSpeculateInteger())
+ break;
+ fixDoubleEdge(0);
+ break;
+ }
+
+ case ArithSqrt: {
+ fixDoubleEdge(0);
+ break;
+ }
+
+ case PutByVal: {
+ if (!m_graph[node.child1()].prediction() || !m_graph[node.child2()].prediction())
+ break;
+ if (!m_graph[node.child2()].shouldSpeculateInteger())
+ break;
+ if (isActionableIntMutableArrayPrediction(m_graph[node.child1()].prediction())) {
+ if (m_graph[node.child3()].isConstant())
+ break;
+ if (m_graph[node.child3()].shouldSpeculateInteger())
+ break;
+ fixDoubleEdge(2);
+ break;
+ }
+ if (isActionableFloatMutableArrayPrediction(m_graph[node.child1()].prediction())) {
+ fixDoubleEdge(2);
+ break;
+ }
+ break;
+ }
+
default:
break;
}
#if DFG_ENABLE(DEBUG_PROPAGATION_VERBOSE)
+ if (!(node.flags() & NodeHasVarArgs)) {
+ dataLog("new children: ");
+ node.dumpChildren(WTF::dataFile());
+ }
dataLog("\n");
#endif
}
@@ -179,7 +308,39 @@
edge = newEdge;
}
+ void fixDoubleEdge(unsigned childIndex)
+ {
+ Node& source = m_graph[m_compileIndex];
+ Edge& edge = source.children.child(childIndex);
+
+ if (!m_graph[edge].shouldSpeculateInteger()) {
+ edge.setUseKind(DoubleUse);
+ return;
+ }
+
+ NodeIndex resultIndex = (NodeIndex)m_graph.size();
+
+#if DFG_ENABLE(DEBUG_PROPAGATION_VERBOSE)
+ dataLog("(replacing @%u->@%u with @%u->@%u) ",
+ m_compileIndex, edge.index(), m_compileIndex, resultIndex);
+#endif
+
+ // Fix the edge up here because it's a reference that will be clobbered by
+ // the append() below.
+ NodeIndex oldIndex = edge.index();
+ edge = Edge(resultIndex, DoubleUse);
+
+ m_graph.append(Node(Int32ToDouble, source.codeOrigin, oldIndex));
+ m_insertionSet.append(m_indexInBlock, resultIndex);
+
+ Node& int32ToDouble = m_graph[resultIndex];
+ int32ToDouble.predict(PredictDouble);
+ int32ToDouble.ref();
+ }
+
+ unsigned m_indexInBlock;
NodeIndex m_compileIndex;
+ InsertionSet<NodeIndex> m_insertionSet;
};
void performFixup(Graph& graph)
diff --git a/Source/JavaScriptCore/dfg/DFGGraph.cpp b/Source/JavaScriptCore/dfg/DFGGraph.cpp
index af3d96d..86c0a4e 100644
--- a/Source/JavaScriptCore/dfg/DFGGraph.cpp
+++ b/Source/JavaScriptCore/dfg/DFGGraph.cpp
@@ -163,15 +163,30 @@
dataLog(", ");
else
hasPrinted = true;
- dataLog("@%u%s", m_varArgChildren[childIdx].index(), predictionToAbbreviatedString(at(childIdx).prediction()));
+ dataLog("%s@%u%s",
+ useKindToString(m_varArgChildren[childIdx].useKind()),
+ m_varArgChildren[childIdx].index(),
+ predictionToAbbreviatedString(at(childIdx).prediction()));
}
} else {
- if (!!node.child1())
- dataLog("@%u%s", node.child1().index(), predictionToAbbreviatedString(at(node.child1()).prediction()));
- if (!!node.child2())
- dataLog(", @%u%s", node.child2().index(), predictionToAbbreviatedString(at(node.child2()).prediction()));
- if (!!node.child3())
- dataLog(", @%u%s", node.child3().index(), predictionToAbbreviatedString(at(node.child3()).prediction()));
+ if (!!node.child1()) {
+ dataLog("%s@%u%s",
+ useKindToString(node.child1().useKind()),
+ node.child1().index(),
+ predictionToAbbreviatedString(at(node.child1()).prediction()));
+ }
+ if (!!node.child2()) {
+ dataLog(", %s@%u%s",
+ useKindToString(node.child2().useKind()),
+ node.child2().index(),
+ predictionToAbbreviatedString(at(node.child2()).prediction()));
+ }
+ if (!!node.child3()) {
+ dataLog(", %s@%u%s",
+ useKindToString(node.child3().useKind()),
+ node.child3().index(),
+ predictionToAbbreviatedString(at(node.child3()).prediction()));
+ }
hasPrinted = !!node.child1();
}
diff --git a/Source/JavaScriptCore/dfg/DFGInsertionSet.h b/Source/JavaScriptCore/dfg/DFGInsertionSet.h
new file mode 100644
index 0000000..82a6a6f
--- /dev/null
+++ b/Source/JavaScriptCore/dfg/DFGInsertionSet.h
@@ -0,0 +1,98 @@
+/*
+ * Copyright (C) 2012 Apple Inc. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in the
+ * documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. ``AS IS'' AND ANY
+ * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+ * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR
+ * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
+ * EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
+ * PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
+ * PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
+ * OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#ifndef DFGInsertionSet_h
+#define DFGInsectionSet_h
+
+#include <wtf/Platform.h>
+
+#if ENABLE(DFG_JIT)
+
+#include <wtf/Vector.h>
+
+namespace JSC { namespace DFG {
+
+template<typename ElementType>
+class Insertion {
+public:
+ Insertion() { }
+
+ Insertion(size_t index, const ElementType& element)
+ : m_index(index)
+ , m_element(element)
+ {
+ }
+
+ size_t index() const { return m_index; }
+ const ElementType& element() const { return m_element; }
+private:
+ size_t m_index;
+ ElementType m_element;
+};
+
+template<typename ElementType>
+class InsertionSet {
+public:
+ InsertionSet() { }
+
+ void append(const Insertion<ElementType>& insertion)
+ {
+ ASSERT(!m_insertions.size() || m_insertions.last().index() <= insertion.index());
+ m_insertions.append(insertion);
+ }
+
+ void append(size_t index, const ElementType& element)
+ {
+ append(Insertion<ElementType>(index, element));
+ }
+
+ template<typename CollectionType>
+ void execute(CollectionType& collection)
+ {
+ if (!m_insertions.size())
+ return;
+ collection.grow(collection.size() + m_insertions.size());
+ size_t lastIndex = collection.size();
+ for (size_t indexInInsertions = m_insertions.size(); indexInInsertions--;) {
+ Insertion<ElementType>& insertion = m_insertions[indexInInsertions];
+ size_t firstIndex = insertion.index() + indexInInsertions;
+ size_t indexOffset = indexInInsertions + 1;
+ for (size_t i = lastIndex; i-- > firstIndex;)
+ collection[i] = collection[i - indexOffset];
+ collection[firstIndex] = insertion.element();
+ lastIndex = firstIndex;
+ }
+ m_insertions.resize(0);
+ }
+private:
+ Vector<Insertion<ElementType>, 8> m_insertions;
+};
+
+} } // namespace JSC::DFG
+
+#endif // ENABLE(DFG_JIT)
+
+#endif // DFGInsertionSet_h
+
diff --git a/Source/JavaScriptCore/dfg/DFGNodeType.h b/Source/JavaScriptCore/dfg/DFGNodeType.h
index c028bbe..11685e1 100644
--- a/Source/JavaScriptCore/dfg/DFGNodeType.h
+++ b/Source/JavaScriptCore/dfg/DFGNodeType.h
@@ -75,6 +75,9 @@
macro(ValueToInt32, NodeResultInt32 | NodeMustGenerate) \
/* Used to box the result of URShift nodes (result has range 0..2^32-1). */\
macro(UInt32ToNumber, NodeResultNumber) \
+ /* Used to cast known integers to doubles, so as to separate the double form */\
+ /* of the value from the integer form. */\
+ macro(Int32ToDouble, NodeResultNumber) \
\
/* Nodes for arithmetic operations. */\
macro(ArithAdd, NodeResultNumber) \
diff --git a/Source/JavaScriptCore/dfg/DFGPredictionPropagationPhase.cpp b/Source/JavaScriptCore/dfg/DFGPredictionPropagationPhase.cpp
index f54ea75..d8c8cc8 100644
--- a/Source/JavaScriptCore/dfg/DFGPredictionPropagationPhase.cpp
+++ b/Source/JavaScriptCore/dfg/DFGPredictionPropagationPhase.cpp
@@ -569,7 +569,8 @@
case GetUint32ArrayLength:
case GetFloat32ArrayLength:
case GetFloat64ArrayLength:
- case GetStringLength: {
+ case GetStringLength:
+ case Int32ToDouble: {
// This node should never be visible at this stage of compilation. It is
// inserted by fixup(), which follows this phase.
ASSERT_NOT_REACHED();
diff --git a/Source/JavaScriptCore/dfg/DFGSpeculativeJIT.cpp b/Source/JavaScriptCore/dfg/DFGSpeculativeJIT.cpp
index fa70f5c..cd6fcb3 100644
--- a/Source/JavaScriptCore/dfg/DFGSpeculativeJIT.cpp
+++ b/Source/JavaScriptCore/dfg/DFGSpeculativeJIT.cpp
@@ -1284,6 +1284,10 @@
// Try to see if there is an alternate node that would contain the value we want.
// There are four possibilities:
//
+ // Int32ToDouble: We can use this in place of the original node, but
+ // we'd rather not; so we use it only if it is the only remaining
+ // live version.
+ //
// ValueToInt32: If the only remaining live version of the value is
// ValueToInt32, then we can use it.
//
@@ -1306,6 +1310,7 @@
}
if (!found) {
+ NodeIndex int32ToDoubleIndex = NoNode;
NodeIndex valueToInt32Index = NoNode;
NodeIndex uint32ToNumberIndex = NoNode;
@@ -1319,6 +1324,9 @@
if (node.child1Unchecked() != valueSource.nodeIndex())
continue;
switch (node.op()) {
+ case Int32ToDouble:
+ int32ToDoubleIndex = info.nodeIndex();
+ break;
case ValueToInt32:
valueToInt32Index = info.nodeIndex();
break;
@@ -1331,7 +1339,9 @@
}
NodeIndex nodeIndexToUse;
- if (valueToInt32Index != NoNode)
+ if (int32ToDoubleIndex != NoNode)
+ nodeIndexToUse = int32ToDoubleIndex;
+ else if (valueToInt32Index != NoNode)
nodeIndexToUse = valueToInt32Index;
else if (uint32ToNumberIndex != NoNode)
nodeIndexToUse = uint32ToNumberIndex;
@@ -1537,7 +1547,7 @@
}
case GeneratedOperandDouble: {
GPRTemporary result(this);
- SpeculateDoubleOperand op1(this, node.child1());
+ DoubleOperand op1(this, node.child1());
FPRReg fpr = op1.fpr();
GPRReg gpr = result.gpr();
JITCompiler::Jump truncatedToInteger = m_jit.branchTruncateDoubleToInt32(fpr, gpr, JITCompiler::BranchIfTruncateSuccessful);
@@ -1674,6 +1684,82 @@
integerResult(result.gpr(), m_compileIndex, op1.format());
}
+void SpeculativeJIT::compileInt32ToDouble(Node& node)
+{
+#if USE(JSVALUE64)
+ // On JSVALUE64 we have a way of loading double constants in a more direct manner
+ // than a int->double conversion. On 32_64, unfortunately, we currently don't have
+ // any such mechanism - though we could have it, if we just provisioned some memory
+ // in CodeBlock for the double form of integer constants.
+ if (at(node.child1()).hasConstant()) {
+ ASSERT(isInt32Constant(node.child1().index()));
+ FPRTemporary result(this);
+ GPRTemporary temp(this);
+ m_jit.move(MacroAssembler::ImmPtr(reinterpret_cast<void*>(reinterpretDoubleToIntptr(valueOfNumberConstant(node.child1().index())))), temp.gpr());
+ m_jit.movePtrToDouble(temp.gpr(), result.fpr());
+ doubleResult(result.fpr(), m_compileIndex);
+ return;
+ }
+#endif
+
+ if (isInt32Prediction(m_state.forNode(node.child1()).m_type)) {
+ SpeculateIntegerOperand op1(this, node.child1());
+ FPRTemporary result(this);
+ m_jit.convertInt32ToDouble(op1.gpr(), result.fpr());
+ doubleResult(result.fpr(), m_compileIndex);
+ return;
+ }
+
+ JSValueOperand op1(this, node.child1());
+ FPRTemporary result(this);
+
+#if USE(JSVALUE64)
+ GPRTemporary temp(this);
+
+ GPRReg op1GPR = op1.gpr();
+ GPRReg tempGPR = temp.gpr();
+ FPRReg resultFPR = result.fpr();
+
+ JITCompiler::Jump isInteger = m_jit.branchPtr(
+ MacroAssembler::AboveOrEqual, op1GPR, GPRInfo::tagTypeNumberRegister);
+
+ speculationCheck(
+ BadType, JSValueRegs(op1GPR), node.child1(),
+ m_jit.branchTestPtr(MacroAssembler::Zero, op1GPR, GPRInfo::tagTypeNumberRegister));
+
+ m_jit.move(op1GPR, tempGPR);
+ unboxDouble(tempGPR, resultFPR);
+ JITCompiler::Jump done = m_jit.jump();
+
+ isInteger.link(&m_jit);
+ m_jit.convertInt32ToDouble(op1GPR, resultFPR);
+ done.link(&m_jit);
+#else
+ FPRTemporary temp(this);
+
+ GPRReg op1TagGPR = op1.tagGPR();
+ GPRReg op1PayloadGPR = op1.payloadGPR();
+ FPRReg tempFPR = temp.fpr();
+ FPRReg resultFPR = result.fpr();
+
+ JITCompiler::Jump isInteger = m_jit.branch32(
+ MacroAssembler::Equal, op1TagGPR, TrustedImm32(JSValue::Int32Tag));
+
+ speculationCheck(
+ BadType, JSValueRegs(op1TagGPR, op1PayloadGPR), node.child1(),
+ m_jit.branch32(MacroAssembler::AboveOrEqual, op1TagGPR, TrustedImm32(JSValue::LowestTag)));
+
+ unboxDouble(op1TagGPR, op1PayloadGPR, resultFPR, tempFPR);
+ JITCompiler::Jump done = m_jit.jump();
+
+ isInteger.link(&m_jit);
+ m_jit.convertInt32ToDouble(op1PayloadGPR, resultFPR);
+ done.link(&m_jit);
+#endif
+
+ doubleResult(resultFPR, m_compileIndex);
+}
+
static double clampDoubleToByte(double d)
{
d += 0.5;
diff --git a/Source/JavaScriptCore/dfg/DFGSpeculativeJIT.h b/Source/JavaScriptCore/dfg/DFGSpeculativeJIT.h
index 4f79ea0..a717ea6 100644
--- a/Source/JavaScriptCore/dfg/DFGSpeculativeJIT.h
+++ b/Source/JavaScriptCore/dfg/DFGSpeculativeJIT.h
@@ -1732,6 +1732,7 @@
void compileGetByValOnString(Node&);
void compileValueToInt32(Node&);
void compileUInt32ToNumber(Node&);
+ void compileInt32ToDouble(Node&);
void compileGetByValOnByteArray(Node&);
void compilePutByValForByteArray(GPRReg base, GPRReg property, Node&);
void compileAdd(Node&);
@@ -1984,6 +1985,7 @@
#endif
{
ASSERT(m_jit);
+ ASSERT(use.useKind() != DoubleUse);
if (jit->isFilled(m_index))
gpr();
}
@@ -2033,6 +2035,15 @@
, m_fprOrInvalid(InvalidFPRReg)
{
ASSERT(m_jit);
+
+ // This is counter-intuitive but correct. DoubleOperand is intended to
+ // be used only when you're a node that is happy to accept an untyped
+ // value, but will special-case for doubles (using DoubleOperand) if the
+ // value happened to already be represented as a double. The implication
+ // is that you will not try to force the value to become a double if it
+ // is not one already.
+ ASSERT(use.useKind() != DoubleUse);
+
if (jit->isFilledDouble(m_index))
fpr();
}
@@ -2078,6 +2089,7 @@
#endif
{
ASSERT(m_jit);
+ ASSERT(use.useKind() != DoubleUse);
#if USE(JSVALUE64)
if (jit->isFilled(m_index))
gpr();
@@ -2188,6 +2200,7 @@
, m_gprOrInvalid(InvalidGPRReg)
{
ASSERT(m_jit);
+ ASSERT(use.useKind() != DoubleUse);
if (jit->isFilled(m_index))
gpr();
}
@@ -2360,6 +2373,7 @@
#endif
{
ASSERT(m_jit);
+ ASSERT(use.useKind() != DoubleUse);
if (jit->isFilled(m_index))
gpr();
}
@@ -2404,6 +2418,7 @@
, m_gprOrInvalid(InvalidGPRReg)
{
ASSERT(m_jit);
+ ASSERT(use.useKind() != DoubleUse);
if (jit->isFilled(m_index))
gpr();
}
@@ -2445,6 +2460,7 @@
, m_fprOrInvalid(InvalidFPRReg)
{
ASSERT(m_jit);
+ ASSERT(use.useKind() == DoubleUse);
if (jit->isFilled(m_index))
fpr();
}
@@ -2481,6 +2497,7 @@
, m_gprOrInvalid(InvalidGPRReg)
{
ASSERT(m_jit);
+ ASSERT(use.useKind() != DoubleUse);
if (jit->isFilled(m_index))
gpr();
}
@@ -2522,6 +2539,7 @@
, m_gprOrInvalid(InvalidGPRReg)
{
ASSERT(m_jit);
+ ASSERT(use.useKind() != DoubleUse);
if (jit->isFilled(m_index))
gpr();
}
diff --git a/Source/JavaScriptCore/dfg/DFGSpeculativeJIT32_64.cpp b/Source/JavaScriptCore/dfg/DFGSpeculativeJIT32_64.cpp
index 4b53fcb..7e0b092 100644
--- a/Source/JavaScriptCore/dfg/DFGSpeculativeJIT32_64.cpp
+++ b/Source/JavaScriptCore/dfg/DFGSpeculativeJIT32_64.cpp
@@ -1878,6 +1878,11 @@
compileValueToInt32(node);
break;
}
+
+ case Int32ToDouble: {
+ compileInt32ToDouble(node);
+ break;
+ }
case ValueAdd:
case ArithAdd:
diff --git a/Source/JavaScriptCore/dfg/DFGSpeculativeJIT64.cpp b/Source/JavaScriptCore/dfg/DFGSpeculativeJIT64.cpp
index 5a09eec..9c65551 100644
--- a/Source/JavaScriptCore/dfg/DFGSpeculativeJIT64.cpp
+++ b/Source/JavaScriptCore/dfg/DFGSpeculativeJIT64.cpp
@@ -1965,6 +1965,11 @@
compileValueToInt32(node);
break;
}
+
+ case Int32ToDouble: {
+ compileInt32ToDouble(node);
+ break;
+ }
case ValueAdd:
case ArithAdd:
