FTL should sink object allocations
https://bugs.webkit.org/show_bug.cgi?id=136330
Reviewed by Oliver Hunt.
Source/JavaScriptCore:
This adds a comprehensive infrastructure for sinking object allocations in DFG SSA form. The
ultimate goal of sinking is to sink an allocation "past the points of its death" - i.e. to
eliminate it completely. The way sinking reasons about the CFG means that it resembles a
partial escape analysis: we create paths through a function where some allocation(s) don't
have to be done at all even if there are other paths along which those allocations still have
to happen. But it also produces other side benefits. Even if an allocation isn't eliminated
along any path, the act of sinking reduces the number of barriers that have to execute.
Because this was a fairly ambituous SSA analysis and transformation, I added a bunch of C++11
sugar to the DFG's internal APIs to allow for easier iteration over blocks, nodes, and
successors; and to add more functor goodness to allow for more lambdas.
This is just the beginning. The bug has a bunch of other bugs that depend on it. So far this
is a spectacular speed-up on microbenchmarks but it's still too limited to affect big
benchmarks. For example, doing o == p makes the sinking phase think that o and p escape.
That's just an omission and there are likely others; we can easily fix them. I think it's
best to land it in its current form and then to worry about the big benchmarks in subsequent
work (see bug 137126).
* CMakeLists.txt:
* JavaScriptCore.vcxproj/JavaScriptCore.vcxproj:
* JavaScriptCore.xcodeproj/project.pbxproj:
* bytecode/StructureSet.h:
(JSC::StructureSet::iterator::iterator):
(JSC::StructureSet::iterator::operator*):
(JSC::StructureSet::iterator::operator++):
(JSC::StructureSet::iterator::operator==):
(JSC::StructureSet::iterator::operator!=):
(JSC::StructureSet::begin):
(JSC::StructureSet::end):
* dfg/DFGAbstractInterpreter.h:
(JSC::DFG::AbstractInterpreter::phiChildren):
* dfg/DFGAbstractInterpreterInlines.h:
(JSC::DFG::AbstractInterpreter<AbstractStateType>::AbstractInterpreter):
(JSC::DFG::AbstractInterpreter<AbstractStateType>::startExecuting):
(JSC::DFG::AbstractInterpreter<AbstractStateType>::executeEffects):
(JSC::DFG::AbstractInterpreter<AbstractStateType>::execute):
* dfg/DFGAvailability.h:
(JSC::DFG::Availability::shouldUseNode):
(JSC::DFG::Availability::isFlushUseful):
(JSC::DFG::Availability::isDead):
(JSC::DFG::Availability::operator!=):
* dfg/DFGAvailabilityMap.cpp: Added.
(JSC::DFG::AvailabilityMap::prune):
(JSC::DFG::AvailabilityMap::clear):
(JSC::DFG::AvailabilityMap::dump):
(JSC::DFG::AvailabilityMap::operator==):
(JSC::DFG::AvailabilityMap::merge):
* dfg/DFGAvailabilityMap.h: Added.
(JSC::DFG::AvailabilityMap::forEachAvailability):
* dfg/DFGBasicBlock.cpp:
(JSC::DFG::BasicBlock::SSAData::SSAData):
* dfg/DFGBasicBlock.h:
(JSC::DFG::BasicBlock::begin):
(JSC::DFG::BasicBlock::end):
(JSC::DFG::BasicBlock::SuccessorsIterable::SuccessorsIterable):
(JSC::DFG::BasicBlock::SuccessorsIterable::iterator::iterator):
(JSC::DFG::BasicBlock::SuccessorsIterable::iterator::operator*):
(JSC::DFG::BasicBlock::SuccessorsIterable::iterator::operator++):
(JSC::DFG::BasicBlock::SuccessorsIterable::iterator::operator==):
(JSC::DFG::BasicBlock::SuccessorsIterable::iterator::operator!=):
(JSC::DFG::BasicBlock::SuccessorsIterable::begin):
(JSC::DFG::BasicBlock::SuccessorsIterable::end):
(JSC::DFG::BasicBlock::successors):
* dfg/DFGClobberize.h:
(JSC::DFG::clobberize):
* dfg/DFGConstantFoldingPhase.cpp:
(JSC::DFG::ConstantFoldingPhase::foldConstants):
* dfg/DFGDoesGC.cpp:
(JSC::DFG::doesGC):
* dfg/DFGFixupPhase.cpp:
(JSC::DFG::FixupPhase::fixupNode):
* dfg/DFGFlushedAt.cpp:
(JSC::DFG::FlushedAt::dump):
* dfg/DFGFlushedAt.h:
(JSC::DFG::FlushedAt::FlushedAt):
* dfg/DFGGraph.cpp:
(JSC::DFG::Graph::dump):
(JSC::DFG::Graph::dumpBlockHeader):
(JSC::DFG::Graph::mergeRelevantToOSR):
(JSC::DFG::Graph::invalidateCFG):
* dfg/DFGGraph.h:
(JSC::DFG::Graph::NaturalBlockIterable::NaturalBlockIterable):
(JSC::DFG::Graph::NaturalBlockIterable::iterator::iterator):
(JSC::DFG::Graph::NaturalBlockIterable::iterator::operator*):
(JSC::DFG::Graph::NaturalBlockIterable::iterator::operator++):
(JSC::DFG::Graph::NaturalBlockIterable::iterator::operator==):
(JSC::DFG::Graph::NaturalBlockIterable::iterator::operator!=):
(JSC::DFG::Graph::NaturalBlockIterable::iterator::findNext):
(JSC::DFG::Graph::NaturalBlockIterable::begin):
(JSC::DFG::Graph::NaturalBlockIterable::end):
(JSC::DFG::Graph::blocksInNaturalOrder):
(JSC::DFG::Graph::doToChildrenWithNode):
(JSC::DFG::Graph::doToChildren):
* dfg/DFGHeapLocation.cpp:
(WTF::printInternal):
* dfg/DFGHeapLocation.h:
* dfg/DFGInsertOSRHintsForUpdate.cpp: Added.
(JSC::DFG::insertOSRHintsForUpdate):
* dfg/DFGInsertOSRHintsForUpdate.h: Added.
* dfg/DFGInsertionSet.h:
(JSC::DFG::InsertionSet::graph):
* dfg/DFGMayExit.cpp:
(JSC::DFG::mayExit):
* dfg/DFGNode.h:
(JSC::DFG::Node::convertToPutByOffsetHint):
(JSC::DFG::Node::convertToPutStructureHint):
(JSC::DFG::Node::convertToPhantomNewObject):
(JSC::DFG::Node::isCellConstant):
(JSC::DFG::Node::castConstant):
(JSC::DFG::Node::hasIdentifier):
(JSC::DFG::Node::hasStorageAccessData):
(JSC::DFG::Node::hasObjectMaterializationData):
(JSC::DFG::Node::objectMaterializationData):
(JSC::DFG::Node::isPhantomObjectAllocation):
* dfg/DFGNodeType.h:
* dfg/DFGOSRAvailabilityAnalysisPhase.cpp:
(JSC::DFG::OSRAvailabilityAnalysisPhase::run):
(JSC::DFG::LocalOSRAvailabilityCalculator::endBlock):
(JSC::DFG::LocalOSRAvailabilityCalculator::executeNode):
* dfg/DFGOSRAvailabilityAnalysisPhase.h:
* dfg/DFGObjectAllocationSinkingPhase.cpp: Added.
(JSC::DFG::ObjectAllocationSinkingPhase::ObjectAllocationSinkingPhase):
(JSC::DFG::ObjectAllocationSinkingPhase::run):
(JSC::DFG::ObjectAllocationSinkingPhase::performSinking):
(JSC::DFG::ObjectAllocationSinkingPhase::determineMaterializationPoints):
(JSC::DFG::ObjectAllocationSinkingPhase::placeMaterializationPoints):
(JSC::DFG::ObjectAllocationSinkingPhase::lowerNonReadingOperationsOnPhantomAllocations):
(JSC::DFG::ObjectAllocationSinkingPhase::promoteSunkenFields):
(JSC::DFG::ObjectAllocationSinkingPhase::resolve):
(JSC::DFG::ObjectAllocationSinkingPhase::handleNode):
(JSC::DFG::ObjectAllocationSinkingPhase::createMaterialize):
(JSC::DFG::ObjectAllocationSinkingPhase::populateMaterialize):
(JSC::DFG::performObjectAllocationSinking):
* dfg/DFGObjectAllocationSinkingPhase.h: Added.
* dfg/DFGObjectMaterializationData.cpp: Added.
(JSC::DFG::PhantomPropertyValue::dump):
(JSC::DFG::ObjectMaterializationData::dump):
(JSC::DFG::ObjectMaterializationData::oneWaySimilarityScore):
(JSC::DFG::ObjectMaterializationData::similarityScore):
* dfg/DFGObjectMaterializationData.h: Added.
(JSC::DFG::PhantomPropertyValue::PhantomPropertyValue):
(JSC::DFG::PhantomPropertyValue::operator==):
* dfg/DFGPhantomCanonicalizationPhase.cpp:
(JSC::DFG::PhantomCanonicalizationPhase::run):
* dfg/DFGPhantomRemovalPhase.cpp:
(JSC::DFG::PhantomRemovalPhase::run):
* dfg/DFGPhiChildren.cpp: Added.
(JSC::DFG::PhiChildren::PhiChildren):
(JSC::DFG::PhiChildren::~PhiChildren):
(JSC::DFG::PhiChildren::upsilonsOf):
* dfg/DFGPhiChildren.h: Added.
(JSC::DFG::PhiChildren::forAllIncomingValues):
(JSC::DFG::PhiChildren::forAllTransitiveIncomingValues):
* dfg/DFGPlan.cpp:
(JSC::DFG::Plan::compileInThreadImpl):
* dfg/DFGPrePostNumbering.cpp: Added.
(JSC::DFG::PrePostNumbering::PrePostNumbering):
(JSC::DFG::PrePostNumbering::~PrePostNumbering):
(JSC::DFG::PrePostNumbering::compute):
(WTF::printInternal):
* dfg/DFGPrePostNumbering.h: Added.
(JSC::DFG::PrePostNumbering::preNumber):
(JSC::DFG::PrePostNumbering::postNumber):
(JSC::DFG::PrePostNumbering::isStrictAncestorOf):
(JSC::DFG::PrePostNumbering::isAncestorOf):
(JSC::DFG::PrePostNumbering::isStrictDescendantOf):
(JSC::DFG::PrePostNumbering::isDescendantOf):
(JSC::DFG::PrePostNumbering::edgeKind):
* dfg/DFGPredictionPropagationPhase.cpp:
(JSC::DFG::PredictionPropagationPhase::propagate):
* dfg/DFGPromoteHeapAccess.h: Added.
(JSC::DFG::promoteHeapAccess):
* dfg/DFGPromotedHeapLocation.cpp: Added.
(JSC::DFG::PromotedLocationDescriptor::dump):
(JSC::DFG::PromotedHeapLocation::createHint):
(JSC::DFG::PromotedHeapLocation::dump):
(WTF::printInternal):
* dfg/DFGPromotedHeapLocation.h: Added.
(JSC::DFG::PromotedLocationDescriptor::PromotedLocationDescriptor):
(JSC::DFG::PromotedLocationDescriptor::operator!):
(JSC::DFG::PromotedLocationDescriptor::kind):
(JSC::DFG::PromotedLocationDescriptor::info):
(JSC::DFG::PromotedLocationDescriptor::hash):
(JSC::DFG::PromotedLocationDescriptor::operator==):
(JSC::DFG::PromotedLocationDescriptor::operator!=):
(JSC::DFG::PromotedLocationDescriptor::isHashTableDeletedValue):
(JSC::DFG::PromotedHeapLocation::PromotedHeapLocation):
(JSC::DFG::PromotedHeapLocation::operator!):
(JSC::DFG::PromotedHeapLocation::kind):
(JSC::DFG::PromotedHeapLocation::base):
(JSC::DFG::PromotedHeapLocation::info):
(JSC::DFG::PromotedHeapLocation::descriptor):
(JSC::DFG::PromotedHeapLocation::hash):
(JSC::DFG::PromotedHeapLocation::operator==):
(JSC::DFG::PromotedHeapLocation::isHashTableDeletedValue):
(JSC::DFG::PromotedHeapLocationHash::hash):
(JSC::DFG::PromotedHeapLocationHash::equal):
* dfg/DFGSSACalculator.cpp:
(JSC::DFG::SSACalculator::reset):
* dfg/DFGSSACalculator.h:
* dfg/DFGSafeToExecute.h:
(JSC::DFG::safeToExecute):
* dfg/DFGSpeculativeJIT.cpp:
(JSC::DFG::SpeculativeJIT::compileCurrentBlock):
* dfg/DFGSpeculativeJIT32_64.cpp:
(JSC::DFG::SpeculativeJIT::compile):
* dfg/DFGSpeculativeJIT64.cpp:
(JSC::DFG::SpeculativeJIT::compile):
* dfg/DFGStructureRegistrationPhase.cpp:
(JSC::DFG::StructureRegistrationPhase::run):
* dfg/DFGValidate.cpp:
(JSC::DFG::Validate::validate):
* ftl/FTLCapabilities.cpp:
(JSC::FTL::canCompile):
* ftl/FTLExitPropertyValue.cpp: Added.
(JSC::FTL::ExitPropertyValue::dump):
* ftl/FTLExitPropertyValue.h: Added.
(JSC::FTL::ExitPropertyValue::ExitPropertyValue):
(JSC::FTL::ExitPropertyValue::operator!):
(JSC::FTL::ExitPropertyValue::location):
(JSC::FTL::ExitPropertyValue::value):
* ftl/FTLExitTimeObjectMaterialization.cpp: Added.
(JSC::FTL::ExitTimeObjectMaterialization::ExitTimeObjectMaterialization):
(JSC::FTL::ExitTimeObjectMaterialization::~ExitTimeObjectMaterialization):
(JSC::FTL::ExitTimeObjectMaterialization::add):
(JSC::FTL::ExitTimeObjectMaterialization::get):
(JSC::FTL::ExitTimeObjectMaterialization::dump):
* ftl/FTLExitTimeObjectMaterialization.h: Added.
(JSC::FTL::ExitTimeObjectMaterialization::type):
(JSC::FTL::ExitTimeObjectMaterialization::properties):
* ftl/FTLExitValue.cpp:
(JSC::FTL::ExitValue::materializeNewObject):
(JSC::FTL::ExitValue::dumpInContext):
* ftl/FTLExitValue.h:
(JSC::FTL::ExitValue::isObjectMaterialization):
(JSC::FTL::ExitValue::objectMaterialization):
(JSC::FTL::ExitValue::withVirtualRegister):
(JSC::FTL::ExitValue::valueFormat):
* ftl/FTLLowerDFGToLLVM.cpp:
(JSC::FTL::LowerDFGToLLVM::compileNode):
(JSC::FTL::LowerDFGToLLVM::compileCheckStructure):
(JSC::FTL::LowerDFGToLLVM::compileArrayifyToStructure):
(JSC::FTL::LowerDFGToLLVM::compilePutStructure):
(JSC::FTL::LowerDFGToLLVM::compileNewObject):
(JSC::FTL::LowerDFGToLLVM::compileMultiGetByOffset):
(JSC::FTL::LowerDFGToLLVM::compileMultiPutByOffset):
(JSC::FTL::LowerDFGToLLVM::compileInvalidationPoint):
(JSC::FTL::LowerDFGToLLVM::compileCheckStructureImmediate):
(JSC::FTL::LowerDFGToLLVM::compileMaterializeNewObject):
(JSC::FTL::LowerDFGToLLVM::checkStructure):
(JSC::FTL::LowerDFGToLLVM::allocateCell):
(JSC::FTL::LowerDFGToLLVM::storeStructure):
(JSC::FTL::LowerDFGToLLVM::allocateObject):
(JSC::FTL::LowerDFGToLLVM::speculateStringObjectForStructureID):
(JSC::FTL::LowerDFGToLLVM::appendOSRExit):
(JSC::FTL::LowerDFGToLLVM::buildExitArguments):
(JSC::FTL::LowerDFGToLLVM::exitValueForAvailability):
(JSC::FTL::LowerDFGToLLVM::exitValueForNode):
(JSC::FTL::LowerDFGToLLVM::weakStructureID):
(JSC::FTL::LowerDFGToLLVM::weakStructure):
(JSC::FTL::LowerDFGToLLVM::availabilityMap):
(JSC::FTL::LowerDFGToLLVM::availability): Deleted.
* ftl/FTLOSRExit.h:
* ftl/FTLOSRExitCompiler.cpp:
(JSC::FTL::compileRecovery):
(JSC::FTL::compileStub):
* ftl/FTLOperations.cpp: Added.
(JSC::FTL::operationNewObjectWithButterfly):
(JSC::FTL::operationMaterializeObjectInOSR):
* ftl/FTLOperations.h: Added.
* ftl/FTLSwitchCase.h:
(JSC::FTL::SwitchCase::SwitchCase):
* runtime/JSObject.h:
(JSC::JSObject::finishCreation):
(JSC::JSFinalObject::JSFinalObject):
(JSC::JSFinalObject::create):
* runtime/Structure.cpp:
(JSC::Structure::canUseForAllocationsOf):
* runtime/Structure.h:
* tests/stress/elidable-new-object-roflcopter-then-exit.js: Added.
(sumOfArithSeries):
(foo):
* tests/stress/elide-new-object-dag-then-exit.js: Added.
(sumOfArithSeries):
(bar):
(verify):
(foo):
* tests/stress/obviously-elidable-new-object-then-exit.js: Added.
(sumOfArithSeries):
(foo):
Source/WTF:
Make it possible to reset a Bag.
* wtf/Bag.h:
(WTF::Bag::Bag):
(WTF::Bag::~Bag):
(WTF::Bag::clear):
LayoutTests:
* js/math-denorm.html: Added.
* js/regress/elidable-new-object-dag-expected.txt: Added.
* js/regress/elidable-new-object-dag.html: Added.
* js/regress/elidable-new-object-roflcopter-expected.txt: Added.
* js/regress/elidable-new-object-roflcopter.html: Added.
* js/regress/elidable-new-object-tree-expected.txt: Added.
* js/regress/elidable-new-object-tree.html: Added.
* js/regress/obvious-sink-pathology-expected.txt: Added.
* js/regress/obvious-sink-pathology-taken-expected.txt: Added.
* js/regress/obvious-sink-pathology-taken.html: Added.
* js/regress/obvious-sink-pathology.html: Added.
* js/regress/obviously-elidable-new-object-expected.txt: Added.
* js/regress/obviously-elidable-new-object.html: Added.
* js/regress/script-tests/elidable-new-object-dag.js: Added.
(sumOfArithSeries):
(foo):
* js/regress/script-tests/elidable-new-object-roflcopter.js: Added.
(sumOfArithSeries):
(foo):
* js/regress/script-tests/elidable-new-object-tree.js: Added.
(sumOfArithSeries):
(foo):
* js/regress/script-tests/obvious-sink-pathology-taken.js: Added.
(sumOfArithSeries):
(bar):
(foo):
* js/regress/script-tests/obvious-sink-pathology.js: Added.
(sumOfArithSeries):
(bar):
(foo):
* js/regress/script-tests/obviously-elidable-new-object.js: Added.
(sumOfArithSeries):
(foo):
* js/regress/script-tests/sinkable-new-object-dag.js: Added.
(sumOfArithSeries):
(verify):
(foo):
* js/regress/script-tests/sinkable-new-object-taken.js: Added.
(sumOfArithSeries):
(bar):
(foo):
* js/regress/script-tests/sinkable-new-object.js: Added.
(sumOfArithSeries):
(bar):
(foo):
* js/regress/sinkable-new-object-dag-expected.txt: Added.
* js/regress/sinkable-new-object-dag.html: Added.
* js/regress/sinkable-new-object-expected.txt: Added.
* js/regress/sinkable-new-object-taken-expected.txt: Added.
* js/regress/sinkable-new-object-taken.html: Added.
* js/regress/sinkable-new-object.html: Added.
git-svn-id: http://svn.webkit.org/repository/webkit/trunk@173993 268f45cc-cd09-0410-ab3c-d52691b4dbfc
diff --git a/Source/JavaScriptCore/ftl/FTLOperations.h b/Source/JavaScriptCore/ftl/FTLOperations.h
new file mode 100644
index 0000000..d15f18e
--- /dev/null
+++ b/Source/JavaScriptCore/ftl/FTLOperations.h
@@ -0,0 +1,50 @@
+/*
+ * Copyright (C) 2014 Apple Inc. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in the
+ * documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. ``AS IS'' AND ANY
+ * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+ * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR
+ * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
+ * EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
+ * PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
+ * PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
+ * OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#ifndef FTLOperations_h
+#define FTLOperations_h
+
+#if ENABLE(FTL_JIT)
+
+#include "DFGOperations.h"
+#include "FTLExitTimeObjectMaterialization.h"
+
+namespace JSC { namespace FTL {
+
+extern "C" {
+
+JSCell* JIT_OPERATION operationNewObjectWithButterfly(ExecState*, Structure*) WTF_INTERNAL;
+
+JSCell* JIT_OPERATION operationMaterializeObjectInOSR(
+ ExecState*, ExitTimeObjectMaterialization*, EncodedJSValue*) WTF_INTERNAL;
+
+} // extern "C"
+
+} } // namespace JSC::DFG
+
+#endif // ENABLE(FTL_JIT)
+
+#endif // FTLOperations_h
+
