Object cycles should not prevent allocation elimination/sinking
https://bugs.webkit.org/show_bug.cgi?id=143073
Reviewed by Filip Pizlo.
Source/JavaScriptCore:
This patch introduces a new allocation sinking phase that is able to
sink cycles, in DFGAllocationCycleSinkingPhase.cpp. This phase
supersedes the old allocation sinking phase in
DFGObjectAllocationSinkingPhase.cpp, as that previous phase was never
able to sink allocation cycles while the new phase sometimes can; see
DFGAllocationCycleSinkingPhase.cpp for details.
For now, the new sinking phase is kept behind a
JSC_enableAllocationCycleSinking flag that reverts to the old sinking
phase when false (i.e., by default). This also removes the old
JSC_enableObjectAllocationSinking flag. run-javascriptcore-tests
defaults to using the new sinking phase.
* dfg/DFGGraph.h:
(JSC::DFG::Graph::addStructureSet): Allow empty structure sets
* dfg/DFGLazyNode.cpp:
(JSC::DFG::LazyNode::dump): Prettier dump
* dfg/DFGNode.h:
(JSC::DFG::Node::cellOperand): Move to opInfo for MaterializeCreateActivation
(JSC::DFG::Node::hasStructureSet): Add MaterializeNewObject
(JSC::DFG::Node::objectMaterializationData): Move to opInfo2
* dfg/DFGOSRAvailabilityAnalysisPhase.cpp: Remove unused header
* dfg/DFGObjectAllocationSinkingPhase.cpp:
(JSC::DFG::ObjectAllocationSinkingPhase::ObjectAllocationSinkingPhase): Deleted.
(JSC::DFG::ObjectAllocationSinkingPhase::run): Deleted.
(JSC::DFG::ObjectAllocationSinkingPhase::performSinking): Deleted.
(JSC::DFG::ObjectAllocationSinkingPhase::determineMaterializationPoints): Deleted.
(JSC::DFG::ObjectAllocationSinkingPhase::placeMaterializationPoints): Deleted.
(JSC::DFG::ObjectAllocationSinkingPhase::lowerNonReadingOperationsOnPhantomAllocations): Deleted.
(JSC::DFG::ObjectAllocationSinkingPhase::promoteSunkenFields): Deleted.
(JSC::DFG::ObjectAllocationSinkingPhase::resolve): Deleted.
(JSC::DFG::ObjectAllocationSinkingPhase::handleNode): Deleted.
(JSC::DFG::ObjectAllocationSinkingPhase::createMaterialize): Deleted.
(JSC::DFG::ObjectAllocationSinkingPhase::populateMaterialize): Deleted.
* dfg/DFGObjectAllocationSinkingPhase.h:
* dfg/DFGPromotedHeapLocation.h: Add a hash and a helper function to PromotedLocationDescriptor
(JSC::DFG::PromotedLocationDescriptor::PromotedLocationDescriptor):
(JSC::DFG::PromotedLocationDescriptor::operator bool):
(JSC::DFG::PromotedLocationDescriptor::neededForMaterialization):
(JSC::DFG::PromotedLocationDescriptorHash::hash):
(JSC::DFG::PromotedLocationDescriptorHash::equal):
* dfg/DFGValidate.cpp:
(JSC::DFG::Validate::validateSSA): Assert that most nodes never see a phantom allocation
* ftl/FTLLowerDFGToLLVM.cpp:
(JSC::FTL::DFG::LowerDFGToLLVM::compileMaterializeNewObject): Use the new structureSet() operand
(JSC::FTL::DFG::LowerDFGToLLVM::compileMaterializeCreateActivation): Node has a new child
* ftl/FTLOSRExitCompiler.cpp: Handle materialization cycles
(JSC::FTL::compileStub):
* ftl/FTLOperations.cpp: Handle materialization cycles
(JSC::FTL::operationPopulateObjectInOSR):
(JSC::FTL::operationMaterializeObjectInOSR):
* ftl/FTLOperations.h: Handle materialization cycles
* tests/stress/correctly-sink-object-even-though-it-dies.js: Added.
(clobber):
(foo):
* tests/stress/eliminate-object-read-over-call.js: Added.
(clobber):
(foo):
* tests/stress/materialize-object-on-edge.js: Added.
(call):
(foo):
* tests/stress/object-sinking-stress.js: Added.
(foo):
* tests/stress/sink-object-cycle.js: Added.
(clobber):
(foo):
* tests/stress/sink-object-past-put.js: Added.
(clobber):
(foo):
* tests/stress/sinkable-new-object-in-loop.js: Added.
(foo):
LayoutTests:
Add a few microbenchmarks that show performance improvement when
sinking or elimininating object cycles.
* js/regress/elidable-new-object-cycle-expected.txt: Added.
* js/regress/elidable-new-object-cycle.html: Added.
* js/regress/script-tests/elidable-new-object-cycle.js: Added.
(sumOfArithSeries):
(foo):
* js/regress/script-tests/sinkable-closure-cycle.js: Added.
(factorial.f):
(factorial):
* js/regress/script-tests/sinkable-new-object-cycle.js: Added.
(sumOfArithSeries):
(verify):
(foo):
* js/regress/sinkable-closure-cycle-expected.txt: Added.
* js/regress/sinkable-closure-cycle.html: Added.
* js/regress/sinkable-new-object-cycle-expected.txt: Added.
* js/regress/sinkable-new-object-cycle.html: Added.
git-svn-id: http://svn.webkit.org/repository/webkit/trunk@186795 268f45cc-cd09-0410-ab3c-d52691b4dbfc
diff --git a/Source/JavaScriptCore/ftl/FTLOSRExitCompiler.cpp b/Source/JavaScriptCore/ftl/FTLOSRExitCompiler.cpp
index 27788e7..1650143 100644
--- a/Source/JavaScriptCore/ftl/FTLOSRExitCompiler.cpp
+++ b/Source/JavaScriptCore/ftl/FTLOSRExitCompiler.cpp
@@ -220,24 +220,24 @@
jit.or32(GPRInfo::regT2, MacroAssembler::AbsoluteAddress(arrayProfile->addressOfArrayModes()));
}
}
-
+
if (!!exit.m_valueProfile)
jit.store64(GPRInfo::regT0, exit.m_valueProfile.getSpecFailBucket(0));
}
-
- // Materialize all objects. Don't materialize an object until all of the objects it needs
- // have been materialized. Curiously, this is the only place that we have an algorithm that prevents
- // OSR exit from handling cyclic object materializations. Of course, object allocation sinking
- // currently wouldn't recognize a cycle as being sinkable - but if it did then the only thing that
- // would ahve to change is this fixpoint. Instead we would allocate the objects first and populate
- // them with data later.
+
+ // Materialize all objects. Don't materialize an object until all
+ // of the objects it needs have been materialized. We break cycles
+ // by populating objects late - we only consider an object as
+ // needing another object if the later is needed for the
+ // allocation of the former.
+
HashSet<ExitTimeObjectMaterialization*> toMaterialize;
for (ExitTimeObjectMaterialization* materialization : exit.m_materializations)
toMaterialize.add(materialization);
-
+
while (!toMaterialize.isEmpty()) {
unsigned previousToMaterializeSize = toMaterialize.size();
-
+
Vector<ExitTimeObjectMaterialization*> worklist;
worklist.appendRange(toMaterialize.begin(), toMaterialize.end());
for (ExitTimeObjectMaterialization* materialization : worklist) {
@@ -246,20 +246,28 @@
for (ExitPropertyValue value : materialization->properties()) {
if (!value.value().isObjectMaterialization())
continue;
+ if (!value.location().neededForMaterialization())
+ continue;
if (toMaterialize.contains(value.value().objectMaterialization())) {
- // Gotta skip this one, since one of its fields points to a materialization
- // that hasn't been materialized.
+ // Gotta skip this one, since it needs a
+ // materialization that hasn't been materialized.
allGood = false;
break;
}
}
if (!allGood)
continue;
-
- // All systems go for materializing the object. First we recover the values of all of
- // its fields and then we call a function to actually allocate the beast.
+
+ // All systems go for materializing the object. First we
+ // recover the values of all of its fields and then we
+ // call a function to actually allocate the beast.
+ // We only recover the fields that are needed for the allocation.
for (unsigned propertyIndex = materialization->properties().size(); propertyIndex--;) {
- const ExitValue& value = materialization->properties()[propertyIndex].value();
+ const ExitPropertyValue& property = materialization->properties()[propertyIndex];
+ const ExitValue& value = property.value();
+ if (!property.location().neededForMaterialization())
+ continue;
+
compileRecovery(
jit, value, record, jitCode->stackmaps, registerScratch,
materializationToPointer);
@@ -273,7 +281,7 @@
jit.move(CCallHelpers::TrustedImmPtr(bitwise_cast<void*>(operationMaterializeObjectInOSR)), GPRInfo::nonArgGPR0);
jit.call(GPRInfo::nonArgGPR0);
jit.storePtr(GPRInfo::returnValueGPR, materializationToPointer.get(materialization));
-
+
// Let everyone know that we're done.
toMaterialize.remove(materialization);
}
@@ -284,6 +292,27 @@
RELEASE_ASSERT(toMaterialize.size() < previousToMaterializeSize);
}
+ // Now that all the objects have been allocated, we populate them
+ // with the correct values. This time we can recover all the
+ // fields, including those that are only needed for the allocation.
+ for (ExitTimeObjectMaterialization* materialization : exit.m_materializations) {
+ for (unsigned propertyIndex = materialization->properties().size(); propertyIndex--;) {
+ const ExitValue& value = materialization->properties()[propertyIndex].value();
+ compileRecovery(
+ jit, value, record, jitCode->stackmaps, registerScratch,
+ materializationToPointer);
+ jit.storePtr(GPRInfo::regT0, materializationArguments + propertyIndex);
+ }
+
+ // This call assumes that we don't pass arguments on the stack
+ jit.setupArgumentsWithExecState(
+ CCallHelpers::TrustedImmPtr(materialization),
+ CCallHelpers::TrustedImmPtr(materializationToPointer.get(materialization)),
+ CCallHelpers::TrustedImmPtr(materializationArguments));
+ jit.move(CCallHelpers::TrustedImmPtr(bitwise_cast<void*>(operationPopulateObjectInOSR)), GPRInfo::nonArgGPR0);
+ jit.call(GPRInfo::nonArgGPR0);
+ }
+
// Save all state from wherever the exit data tells us it was, into the appropriate place in
// the scratch buffer. This also does the reboxing.
