We should have a more concise way of determining when we're varargs calling a function using rest parameters
https://bugs.webkit.org/show_bug.cgi?id=164258
Reviewed by Yusuke Suzuki.
JSTests:
* microbenchmarks/call-using-spread.js: Added.
(bar):
(foo):
* microbenchmarks/spread-large-array.js: Added.
(foo):
(arrays.push):
* microbenchmarks/spread-small-array.js: Added.
(foo):
* stress/spread-array-iterator-watchpoint-2.js: Added.
(foo):
(arrayIterator.next):
* stress/spread-array-iterator-watchpoint.js: Added.
(foo):
(Array.prototype.Symbol.iterator):
* stress/spread-non-array.js: Added.
(assert):
(foo):
(let.customIterator.Symbol.iterator):
(bar):
Source/JavaScriptCore:
This patch adds two new bytecodes and DFG nodes for the following code patterns:
```
foo(a, b, ...c)
let x = [a, b, ...c];
```
To do this, I've introduced two new bytecode operations (and their
corresponding DFG nodes):
op_spread and op_new_array_with_spread.
op_spread takes a single input and performs the ES6 iteration protocol on it.
It returns the result of doing the spread inside a new class I've
made called JSFixedArray. JSFixedArray is a cell with a single 'size'
field and a buffer of values allocated inline in the cell. Abstracting
the protocol into a single node is good because it will make IR analysis
in the future much simpler. For now, it's also good because it allows
us to create fast paths for array iteration (which is quite common).
This fast path allows us to emit really good code for array iteration
inside the DFG/FTL.
op_new_array_with_spread is a variable argument bytecode that also
has a bit vector associated with it. The bit vector indicates if
any particular argument is to be spread or not. Arguments that
are spread are known to be JSFixedArray because we must emit an
op_spread before op_new_array_with_spread consumes the value.
For example, for this array:
[a, b, ...c, d, ...e]
we will have this bit vector:
[0, 0, 1, 0, 1]
The reason I've chosen this IR is that it will make eliminating
a rest allocation for this type of code much easier:
```
function foo(...args) {
return bar(a, b, ...args);
}
```
It will be easier to analyze the IR now that the operations
will be described at a high level.
This patch is an ~8% speedup on ES6SampleBench on my MBP.
* CMakeLists.txt:
* DerivedSources.make:
* JavaScriptCore.xcodeproj/project.pbxproj:
* builtins/IteratorHelpers.js: Added.
(performIteration):
* bytecode/BytecodeList.json:
* bytecode/BytecodeUseDef.h:
(JSC::computeUsesForBytecodeOffset):
(JSC::computeDefsForBytecodeOffset):
* bytecode/CodeBlock.cpp:
(JSC::CodeBlock::dumpBytecode):
* bytecode/ObjectPropertyConditionSet.cpp:
(JSC::generateConditionForSelfEquivalence):
* bytecode/ObjectPropertyConditionSet.h:
* bytecode/TrackedReferences.cpp:
(JSC::TrackedReferences::check):
* bytecode/UnlinkedCodeBlock.h:
(JSC::UnlinkedCodeBlock::bitVectors):
(JSC::UnlinkedCodeBlock::bitVector):
(JSC::UnlinkedCodeBlock::addBitVector):
(JSC::UnlinkedCodeBlock::shrinkToFit):
* bytecompiler/BytecodeGenerator.cpp:
(JSC::BytecodeGenerator::emitNewArrayWithSpread):
* bytecompiler/BytecodeGenerator.h:
* bytecompiler/NodesCodegen.cpp:
(JSC::ArrayNode::emitBytecode):
* dfg/DFGAbstractInterpreterInlines.h:
(JSC::DFG::AbstractInterpreter<AbstractStateType>::executeEffects):
* dfg/DFGByteCodeParser.cpp:
(JSC::DFG::ByteCodeParser::addToGraph):
(JSC::DFG::ByteCodeParser::parseBlock):
* dfg/DFGCapabilities.cpp:
(JSC::DFG::capabilityLevel):
* dfg/DFGClobberize.h:
(JSC::DFG::clobberize):
* dfg/DFGDoesGC.cpp:
(JSC::DFG::doesGC):
* dfg/DFGFixupPhase.cpp:
(JSC::DFG::FixupPhase::fixupNode):
(JSC::DFG::FixupPhase::watchHavingABadTime):
* dfg/DFGGraph.h:
(JSC::DFG::Graph::isWatchingArrayIteratorProtocolWatchpoint):
* dfg/DFGNode.h:
(JSC::DFG::Node::bitVector):
* dfg/DFGNodeType.h:
* dfg/DFGOperations.cpp:
* dfg/DFGOperations.h:
* dfg/DFGPredictionPropagationPhase.cpp:
* dfg/DFGSafeToExecute.h:
(JSC::DFG::safeToExecute):
* dfg/DFGSpeculativeJIT.cpp:
(JSC::DFG::SpeculativeJIT::compileSpread):
(JSC::DFG::SpeculativeJIT::compileNewArrayWithSpread):
* dfg/DFGSpeculativeJIT.h:
(JSC::DFG::SpeculativeJIT::callOperation):
* dfg/DFGSpeculativeJIT32_64.cpp:
(JSC::DFG::SpeculativeJIT::compile):
* dfg/DFGSpeculativeJIT64.cpp:
(JSC::DFG::SpeculativeJIT::compile):
* dfg/DFGStructureRegistrationPhase.cpp:
(JSC::DFG::StructureRegistrationPhase::run):
* ftl/FTLAbstractHeapRepository.h:
* ftl/FTLCapabilities.cpp:
(JSC::FTL::canCompile):
* ftl/FTLLowerDFGToB3.cpp:
(JSC::FTL::DFG::LowerDFGToB3::compileNode):
(JSC::FTL::DFG::LowerDFGToB3::compileNewArrayWithSpread):
(JSC::FTL::DFG::LowerDFGToB3::compileSpread):
(JSC::FTL::DFG::LowerDFGToB3::allocateVariableSizedCell):
* jit/AssemblyHelpers.h:
(JSC::AssemblyHelpers::emitAllocateVariableSizedCell):
(JSC::AssemblyHelpers::emitAllocateVariableSizedJSObject):
* jit/JIT.cpp:
(JSC::JIT::privateCompileMainPass):
* jit/JIT.h:
* jit/JITOpcodes.cpp:
(JSC::JIT::emit_op_new_array_with_spread):
(JSC::JIT::emit_op_spread):
* jit/JITOperations.h:
* llint/LLIntData.cpp:
(JSC::LLInt::Data::performAssertions):
* llint/LLIntSlowPaths.cpp:
* llint/LowLevelInterpreter.asm:
* runtime/ArrayIteratorAdaptiveWatchpoint.cpp: Added.
(JSC::ArrayIteratorAdaptiveWatchpoint::ArrayIteratorAdaptiveWatchpoint):
(JSC::ArrayIteratorAdaptiveWatchpoint::handleFire):
* runtime/ArrayIteratorAdaptiveWatchpoint.h: Added.
* runtime/CommonSlowPaths.cpp:
(JSC::SLOW_PATH_DECL):
* runtime/CommonSlowPaths.h:
* runtime/IteratorOperations.h:
(JSC::forEachInIterable):
* runtime/JSCInlines.h:
* runtime/JSFixedArray.cpp: Added.
(JSC::JSFixedArray::visitChildren):
* runtime/JSFixedArray.h: Added.
(JSC::JSFixedArray::createStructure):
(JSC::JSFixedArray::createFromArray):
(JSC::JSFixedArray::get):
(JSC::JSFixedArray::buffer):
(JSC::JSFixedArray::size):
(JSC::JSFixedArray::offsetOfSize):
(JSC::JSFixedArray::offsetOfData):
(JSC::JSFixedArray::create):
(JSC::JSFixedArray::JSFixedArray):
(JSC::JSFixedArray::allocationSize):
* runtime/JSGlobalObject.cpp:
(JSC::JSGlobalObject::JSGlobalObject):
(JSC::JSGlobalObject::init):
(JSC::JSGlobalObject::visitChildren):
(JSC::JSGlobalObject::objectPrototypeIsSane): Deleted.
(JSC::JSGlobalObject::arrayPrototypeChainIsSane): Deleted.
(JSC::JSGlobalObject::stringPrototypeChainIsSane): Deleted.
* runtime/JSGlobalObject.h:
(JSC::JSGlobalObject::arrayIteratorProtocolWatchpoint):
(JSC::JSGlobalObject::iteratorProtocolFunction):
* runtime/JSGlobalObjectInlines.h: Added.
(JSC::JSGlobalObject::objectPrototypeIsSane):
(JSC::JSGlobalObject::arrayPrototypeChainIsSane):
(JSC::JSGlobalObject::stringPrototypeChainIsSane):
(JSC::JSGlobalObject::isArrayIteratorProtocolFastAndNonObservable):
* runtime/JSType.h:
* runtime/VM.cpp:
(JSC::VM::VM):
* runtime/VM.h:
git-svn-id: http://svn.webkit.org/repository/webkit/trunk@208637 268f45cc-cd09-0410-ab3c-d52691b4dbfc
diff --git a/Source/JavaScriptCore/ftl/FTLLowerDFGToB3.cpp b/Source/JavaScriptCore/ftl/FTLLowerDFGToB3.cpp
index 60e04a0..fd8a90b 100644
--- a/Source/JavaScriptCore/ftl/FTLLowerDFGToB3.cpp
+++ b/Source/JavaScriptCore/ftl/FTLLowerDFGToB3.cpp
@@ -727,6 +727,12 @@
case NewArray:
compileNewArray();
break;
+ case NewArrayWithSpread:
+ compileNewArrayWithSpread();
+ break;
+ case Spread:
+ compileSpread();
+ break;
case NewArrayBuffer:
compileNewArrayBuffer();
break;
@@ -4296,6 +4302,195 @@
setJSValue(result);
}
+
+ void compileNewArrayWithSpread()
+ {
+ if (m_graph.isWatchingHavingABadTimeWatchpoint(m_node)) {
+ unsigned startLength = 0;
+ BitVector* bitVector = m_node->bitVector();
+ for (unsigned i = 0; i < m_node->numChildren(); ++i) {
+ if (!bitVector->get(i))
+ ++startLength;
+ }
+
+ LValue length = m_out.constInt32(startLength);
+
+ for (unsigned i = 0; i < m_node->numChildren(); ++i) {
+ if (bitVector->get(i)) {
+ Edge use = m_graph.varArgChild(m_node, i);
+ LValue fixedArray = lowCell(use);
+ length = m_out.add(length, m_out.load32(fixedArray, m_heaps.JSFixedArray_size));
+ }
+ }
+
+ Structure* structure = m_graph.globalObjectFor(m_node->origin.semantic)->originalArrayStructureForIndexingType(ArrayWithContiguous);
+ ArrayValues arrayValues = allocateUninitializedContiguousJSArray(length, structure);
+ LValue result = arrayValues.array;
+ LValue storage = arrayValues.butterfly;
+ LValue index = m_out.constIntPtr(0);
+
+ for (unsigned i = 0; i < m_node->numChildren(); ++i) {
+ Edge use = m_graph.varArgChild(m_node, i);
+ if (bitVector->get(i)) {
+ LBasicBlock loopStart = m_out.newBlock();
+ LBasicBlock continuation = m_out.newBlock();
+
+ LValue fixedArray = lowCell(use);
+
+ ValueFromBlock fixedIndexStart = m_out.anchor(m_out.constIntPtr(0));
+ ValueFromBlock arrayIndexStart = m_out.anchor(index);
+ ValueFromBlock arrayIndexStartForFinish = m_out.anchor(index);
+
+ LValue fixedArraySize = m_out.zeroExtPtr(m_out.load32(fixedArray, m_heaps.JSFixedArray_size));
+
+ m_out.branch(
+ m_out.isZero64(fixedArraySize),
+ unsure(continuation), unsure(loopStart));
+
+ LBasicBlock lastNext = m_out.appendTo(loopStart, continuation);
+
+ LValue arrayIndex = m_out.phi(pointerType(), arrayIndexStart);
+ LValue fixedArrayIndex = m_out.phi(pointerType(), fixedIndexStart);
+
+ LValue item = m_out.load64(m_out.baseIndex(m_heaps.JSFixedArray_buffer, fixedArray, fixedArrayIndex));
+ m_out.store64(item, m_out.baseIndex(m_heaps.indexedContiguousProperties, storage, arrayIndex));
+
+ LValue nextArrayIndex = m_out.add(arrayIndex, m_out.constIntPtr(1));
+ LValue nextFixedArrayIndex = m_out.add(fixedArrayIndex, m_out.constIntPtr(1));
+ ValueFromBlock arrayIndexLoopForFinish = m_out.anchor(nextArrayIndex);
+
+ m_out.addIncomingToPhi(fixedArrayIndex, m_out.anchor(nextFixedArrayIndex));
+ m_out.addIncomingToPhi(arrayIndex, m_out.anchor(nextArrayIndex));
+
+ m_out.branch(
+ m_out.below(nextFixedArrayIndex, fixedArraySize),
+ unsure(loopStart), unsure(continuation));
+
+ m_out.appendTo(continuation, lastNext);
+ index = m_out.phi(pointerType(), arrayIndexStartForFinish, arrayIndexLoopForFinish);
+ } else {
+ IndexedAbstractHeap& heap = m_heaps.indexedContiguousProperties;
+ LValue item = lowJSValue(use);
+ m_out.store64(item, m_out.baseIndex(heap, storage, index));
+ index = m_out.add(index, m_out.constIntPtr(1));
+ }
+ }
+
+ setJSValue(result);
+ return;
+ }
+
+ ASSERT(m_node->numChildren());
+ size_t scratchSize = sizeof(EncodedJSValue) * m_node->numChildren();
+ ScratchBuffer* scratchBuffer = vm().scratchBufferForSize(scratchSize);
+ EncodedJSValue* buffer = static_cast<EncodedJSValue*>(scratchBuffer->dataBuffer());
+ BitVector* bitVector = m_node->bitVector();
+ for (unsigned i = 0; i < m_node->numChildren(); ++i) {
+ Edge use = m_graph.m_varArgChildren[m_node->firstChild() + i];
+ LValue value;
+ if (bitVector->get(i))
+ value = lowCell(use);
+ else
+ value = lowJSValue(use);
+ m_out.store64(value, m_out.absolute(&buffer[i]));
+ }
+
+ m_out.storePtr(m_out.constIntPtr(scratchSize), m_out.absolute(scratchBuffer->activeLengthPtr()));
+ LValue result = vmCall(Int64, m_out.operation(operationNewArrayWithSpreadSlow), m_callFrame, m_out.constIntPtr(buffer), m_out.constInt32(m_node->numChildren()));
+ m_out.storePtr(m_out.constIntPtr(0), m_out.absolute(scratchBuffer->activeLengthPtr()));
+
+ setJSValue(result);
+ }
+
+ void compileSpread()
+ {
+ LValue argument = lowCell(m_node->child1());
+
+ LValue result;
+ if (m_node->child1().useKind() == ArrayUse) {
+ speculateArray(m_node->child1());
+
+ LBasicBlock preLoop = m_out.newBlock();
+ LBasicBlock loopSelection = m_out.newBlock();
+ LBasicBlock contiguousLoopStart = m_out.newBlock();
+ LBasicBlock doubleLoopStart = m_out.newBlock();
+ LBasicBlock slowPath = m_out.newBlock();
+ LBasicBlock continuation = m_out.newBlock();
+
+ LValue indexingShape = m_out.load8ZeroExt32(argument, m_heaps.JSCell_indexingType);
+ indexingShape = m_out.bitAnd(indexingShape, m_out.constInt32(IndexingShapeMask));
+ LValue isOKIndexingType = m_out.belowOrEqual(
+ m_out.sub(indexingShape, m_out.constInt32(Int32Shape)),
+ m_out.constInt32(ContiguousShape - Int32Shape));
+
+ m_out.branch(isOKIndexingType, unsure(preLoop), unsure(slowPath));
+ LBasicBlock lastNext = m_out.appendTo(preLoop, loopSelection);
+
+ LValue butterfly = m_out.loadPtr(argument, m_heaps.JSObject_butterfly);
+ LValue length = m_out.load32NonNegative(butterfly, m_heaps.Butterfly_publicLength);
+ static_assert(sizeof(JSValue) == 8 && 1 << 3 == 8, "Assumed in the code below.");
+ LValue size = m_out.add(
+ m_out.shl(m_out.zeroExtPtr(length), m_out.constInt32(3)),
+ m_out.constIntPtr(JSFixedArray::offsetOfData()));
+
+ LValue fastAllocation = allocateVariableSizedCell<JSFixedArray>(size, m_graph.m_vm.fixedArrayStructure.get(), slowPath);
+ ValueFromBlock fastResult = m_out.anchor(fastAllocation);
+ m_out.store32(length, fastAllocation, m_heaps.JSFixedArray_size);
+
+ ValueFromBlock startIndexForContiguous = m_out.anchor(m_out.constIntPtr(0));
+ ValueFromBlock startIndexForDouble = m_out.anchor(m_out.constIntPtr(0));
+
+ m_out.branch(m_out.isZero32(length), unsure(continuation), unsure(loopSelection));
+
+ m_out.appendTo(loopSelection, contiguousLoopStart);
+ m_out.branch(m_out.equal(indexingShape, m_out.constInt32(DoubleShape)),
+ unsure(doubleLoopStart), unsure(contiguousLoopStart));
+
+ {
+ m_out.appendTo(contiguousLoopStart, doubleLoopStart);
+ LValue index = m_out.phi(pointerType(), startIndexForContiguous);
+
+ TypedPointer loadSite = m_out.baseIndex(m_heaps.root, butterfly, index, ScaleEight); // We read TOP here since we can be reading either int32 or contiguous properties.
+ LValue value = m_out.load64(loadSite);
+ value = m_out.select(m_out.isZero64(value), m_out.constInt64(JSValue::encode(jsUndefined())), value);
+ m_out.store64(value, m_out.baseIndex(m_heaps.JSFixedArray_buffer, fastAllocation, index));
+
+ LValue nextIndex = m_out.add(index, m_out.constIntPtr(1));
+ m_out.addIncomingToPhi(index, m_out.anchor(nextIndex));
+
+ m_out.branch(m_out.below(nextIndex, m_out.zeroExtPtr(length)),
+ unsure(contiguousLoopStart), unsure(continuation));
+ }
+
+ {
+ m_out.appendTo(doubleLoopStart, slowPath);
+ LValue index = m_out.phi(pointerType(), startIndexForDouble);
+
+ LValue value = m_out.loadDouble(m_out.baseIndex(m_heaps.indexedDoubleProperties, butterfly, index));
+ LValue isNaN = m_out.doubleNotEqualOrUnordered(value, value);
+ LValue holeResult = m_out.constInt64(JSValue::encode(jsUndefined()));
+ LValue normalResult = boxDouble(value);
+ value = m_out.select(isNaN, holeResult, normalResult);
+ m_out.store64(value, m_out.baseIndex(m_heaps.JSFixedArray_buffer, fastAllocation, index));
+
+ LValue nextIndex = m_out.add(index, m_out.constIntPtr(1));
+ m_out.addIncomingToPhi(index, m_out.anchor(nextIndex));
+
+ m_out.branch(m_out.below(nextIndex, m_out.zeroExtPtr(length)),
+ unsure(doubleLoopStart), unsure(continuation));
+ }
+
+ m_out.appendTo(slowPath, continuation);
+ ValueFromBlock slowResult = m_out.anchor(vmCall(Int64, m_out.operation(operationSpreadFastArray), m_callFrame, argument));
+ m_out.jump(continuation);
+
+ m_out.appendTo(continuation, lastNext);
+ result = m_out.phi(Int64, fastResult, slowResult);
+ } else
+ result = vmCall(Int64, m_out.operation(operationSpreadGeneric), m_callFrame, argument);
+
+ setJSValue(result);
+ }
void compileNewArrayBuffer()
{
@@ -9741,6 +9936,15 @@
vm().heap.subspaceForObjectOfType<ClassType>(), size, slowPath);
return allocateObject(allocator, structure, butterfly, slowPath);
}
+
+ template<typename ClassType>
+ LValue allocateVariableSizedCell(
+ LValue size, Structure* structure, LBasicBlock slowPath)
+ {
+ LValue allocator = allocatorForSize(
+ vm().heap.subspaceForObjectOfType<ClassType>(), size, slowPath);
+ return allocateCell(allocator, structure, slowPath);
+ }
LValue allocateObject(Structure* structure)
{
