get_by_id ICs should have a structure history used to indicate when we should skip generating an IC
https://bugs.webkit.org/show_bug.cgi?id=204904
<rdar://problem/57631437>
Reviewed by Yusuke Suzuki and Tadeu Zagallo.
I implemented a similar policy for get_by_val for the number of unique seen
identifiers. This allows us to create a heuristic to directly call the slow
path when profiling information tells us if inline caching might not be
profitable. This patch implements a similar policy for get_by_id where we
profile the seen base value structures. If the LLInt observes enough
unique structures, we omit emitting the inline cache in the upper
tiers.
The goal here was to try to speed up Speedometer2. Local testing showed
this patch to repeatedly be 0.5% faster, but all the P values I got were
insignificant. So it appears it's either neutral or slightly faster.
This patch also adjusts the policy of seeing a non-identifier inside
the PointerHistory data structure. Instead of increasing it to reach the
limit when we see a non-identifier, we just treat each execution with
a non-identifier to increment the count by 1.
* JavaScriptCore.xcodeproj/project.pbxproj:
* bytecode/BytecodeList.rb:
* bytecode/GetByValHistory.h: Removed.
* bytecode/PointerHistory.h: Copied from Source/JavaScriptCore/bytecode/GetByValHistory.h.
(JSC::PointerHistory::observe):
(JSC::PointerHistory::observeNull):
(JSC::GetByValHistory::observeNonUID): Deleted.
(JSC::GetByValHistory::observe): Deleted.
(JSC::GetByValHistory::count const): Deleted.
(JSC::GetByValHistory::filter const): Deleted.
(JSC::GetByValHistory::update): Deleted.
* dfg/DFGByteCodeParser.cpp:
(JSC::DFG::ByteCodeParser::parseGetById):
(JSC::DFG::ByteCodeParser::parseBlock):
* dfg/DFGGraph.h:
* dfg/DFGSpeculativeJIT.cpp:
(JSC::DFG::SpeculativeJIT::compileGetById):
* dfg/DFGSpeculativeJIT32_64.cpp:
(JSC::DFG::SpeculativeJIT::compile):
* dfg/DFGSpeculativeJIT64.cpp:
(JSC::DFG::SpeculativeJIT::compile):
* ftl/FTLLowerDFGToB3.cpp:
(JSC::FTL::DFG::LowerDFGToB3::compileGetById):
(JSC::FTL::DFG::LowerDFGToB3::compileGetByVal):
* generator/DSL.rb:
* jit/JITPropertyAccess.cpp:
(JSC::JIT::emit_op_try_get_by_id):
(JSC::JIT::emitSlow_op_try_get_by_id):
(JSC::JIT::emit_op_get_by_id_direct):
(JSC::JIT::emitSlow_op_get_by_id_direct):
(JSC::JIT::emit_op_get_by_id):
(JSC::JIT::emitSlow_op_get_by_id):
* llint/LLIntSlowPaths.cpp:
(JSC::LLInt::LLINT_SLOW_PATH_DECL):
* runtime/OptionsList.h:
git-svn-id: http://svn.webkit.org/repository/webkit/trunk@253201 268f45cc-cd09-0410-ab3c-d52691b4dbfc
diff --git a/Source/JavaScriptCore/ChangeLog b/Source/JavaScriptCore/ChangeLog
index 5ca44b1..1b70267 100644
--- a/Source/JavaScriptCore/ChangeLog
+++ b/Source/JavaScriptCore/ChangeLog
@@ -1,3 +1,64 @@
+2019-12-05 Saam Barati <sbarati@apple.com>
+
+ get_by_id ICs should have a structure history used to indicate when we should skip generating an IC
+ https://bugs.webkit.org/show_bug.cgi?id=204904
+ <rdar://problem/57631437>
+
+ Reviewed by Yusuke Suzuki and Tadeu Zagallo.
+
+ I implemented a similar policy for get_by_val for the number of unique seen
+ identifiers. This allows us to create a heuristic to directly call the slow
+ path when profiling information tells us if inline caching might not be
+ profitable. This patch implements a similar policy for get_by_id where we
+ profile the seen base value structures. If the LLInt observes enough
+ unique structures, we omit emitting the inline cache in the upper
+ tiers.
+
+ The goal here was to try to speed up Speedometer2. Local testing showed
+ this patch to repeatedly be 0.5% faster, but all the P values I got were
+ insignificant. So it appears it's either neutral or slightly faster.
+
+ This patch also adjusts the policy of seeing a non-identifier inside
+ the PointerHistory data structure. Instead of increasing it to reach the
+ limit when we see a non-identifier, we just treat each execution with
+ a non-identifier to increment the count by 1.
+
+ * JavaScriptCore.xcodeproj/project.pbxproj:
+ * bytecode/BytecodeList.rb:
+ * bytecode/GetByValHistory.h: Removed.
+ * bytecode/PointerHistory.h: Copied from Source/JavaScriptCore/bytecode/GetByValHistory.h.
+ (JSC::PointerHistory::observe):
+ (JSC::PointerHistory::observeNull):
+ (JSC::GetByValHistory::observeNonUID): Deleted.
+ (JSC::GetByValHistory::observe): Deleted.
+ (JSC::GetByValHistory::count const): Deleted.
+ (JSC::GetByValHistory::filter const): Deleted.
+ (JSC::GetByValHistory::update): Deleted.
+ * dfg/DFGByteCodeParser.cpp:
+ (JSC::DFG::ByteCodeParser::parseGetById):
+ (JSC::DFG::ByteCodeParser::parseBlock):
+ * dfg/DFGGraph.h:
+ * dfg/DFGSpeculativeJIT.cpp:
+ (JSC::DFG::SpeculativeJIT::compileGetById):
+ * dfg/DFGSpeculativeJIT32_64.cpp:
+ (JSC::DFG::SpeculativeJIT::compile):
+ * dfg/DFGSpeculativeJIT64.cpp:
+ (JSC::DFG::SpeculativeJIT::compile):
+ * ftl/FTLLowerDFGToB3.cpp:
+ (JSC::FTL::DFG::LowerDFGToB3::compileGetById):
+ (JSC::FTL::DFG::LowerDFGToB3::compileGetByVal):
+ * generator/DSL.rb:
+ * jit/JITPropertyAccess.cpp:
+ (JSC::JIT::emit_op_try_get_by_id):
+ (JSC::JIT::emitSlow_op_try_get_by_id):
+ (JSC::JIT::emit_op_get_by_id_direct):
+ (JSC::JIT::emitSlow_op_get_by_id_direct):
+ (JSC::JIT::emit_op_get_by_id):
+ (JSC::JIT::emitSlow_op_get_by_id):
+ * llint/LLIntSlowPaths.cpp:
+ (JSC::LLInt::LLINT_SLOW_PATH_DECL):
+ * runtime/OptionsList.h:
+
2019-12-05 Tadeu Zagallo <tzagallo@apple.com>
[WebAssembly] Fix LLIntCallee's ownership
diff --git a/Source/JavaScriptCore/JavaScriptCore.xcodeproj/project.pbxproj b/Source/JavaScriptCore/JavaScriptCore.xcodeproj/project.pbxproj
index 54ea706..3933ade 100644
--- a/Source/JavaScriptCore/JavaScriptCore.xcodeproj/project.pbxproj
+++ b/Source/JavaScriptCore/JavaScriptCore.xcodeproj/project.pbxproj
@@ -889,7 +889,6 @@
451539B912DC994500EF7AC4 /* Yarr.h in Headers */ = {isa = PBXBuildFile; fileRef = 451539B812DC994500EF7AC4 /* Yarr.h */; settings = {ATTRIBUTES = (Private, ); }; };
473DA4A4764C45FE871B0485 /* DefinePropertyAttributes.h in Headers */ = {isa = PBXBuildFile; fileRef = 169948EDE68D4054B01EF797 /* DefinePropertyAttributes.h */; settings = {ATTRIBUTES = (Private, ); }; };
4BAA07CEB81F49A296E02203 /* WasmSignatureInlines.h in Headers */ = {isa = PBXBuildFile; fileRef = 30A5F403F11C4F599CD596D5 /* WasmSignatureInlines.h */; settings = {ATTRIBUTES = (Private, ); }; };
- 520D99F12388CC81000509A3 /* GetByValHistory.h in Headers */ = {isa = PBXBuildFile; fileRef = 520D99F02388CC78000509A3 /* GetByValHistory.h */; };
521131F71F82BF14007CCEEE /* PolyProtoAccessChain.h in Headers */ = {isa = PBXBuildFile; fileRef = 521131F61F82BF11007CCEEE /* PolyProtoAccessChain.h */; settings = {ATTRIBUTES = (Private, ); }; };
521322461ECBCE8200F65615 /* WebAssemblyFunctionBase.h in Headers */ = {isa = PBXBuildFile; fileRef = 521322441ECBCE8200F65615 /* WebAssemblyFunctionBase.h */; };
522927D5235FD0B9005CB169 /* GCMemoryOperations.h in Headers */ = {isa = PBXBuildFile; fileRef = 5272987B235FC8BA005C982C /* GCMemoryOperations.h */; settings = {ATTRIBUTES = (Private, ); }; };
@@ -902,6 +901,7 @@
526AC4B71E977C5D003500E1 /* WasmCodeBlock.h in Headers */ = {isa = PBXBuildFile; fileRef = 526AC4B51E977C5D003500E1 /* WasmCodeBlock.h */; settings = {ATTRIBUTES = (Private, ); }; };
527CE35422555FE500C6F382 /* JSToWasmICCallee.h in Headers */ = {isa = PBXBuildFile; fileRef = 527CE35322555FDD00C6F382 /* JSToWasmICCallee.h */; };
52847ADC21FFB8690061A9DB /* WasmAirIRGenerator.h in Headers */ = {isa = PBXBuildFile; fileRef = 52847ADA21FFB8630061A9DB /* WasmAirIRGenerator.h */; };
+ 52AD08232399A1FB00B4101A /* PointerHistory.h in Headers */ = {isa = PBXBuildFile; fileRef = 52AD08222399A1F200B4101A /* PointerHistory.h */; };
52B310FB1974AE610080857C /* FunctionHasExecutedCache.h in Headers */ = {isa = PBXBuildFile; fileRef = 52B310FA1974AE610080857C /* FunctionHasExecutedCache.h */; settings = {ATTRIBUTES = (Private, ); }; };
52B311011975B4670080857C /* TypeLocationCache.h in Headers */ = {isa = PBXBuildFile; fileRef = 52B311001975B4670080857C /* TypeLocationCache.h */; settings = {ATTRIBUTES = (Private, ); }; };
52C0611F1AA51E1C00B4ADBA /* RuntimeType.h in Headers */ = {isa = PBXBuildFile; fileRef = 52C0611D1AA51E1B00B4ADBA /* RuntimeType.h */; settings = {ATTRIBUTES = (Private, ); }; };
@@ -3538,7 +3538,6 @@
4CE978E385A8498199052153 /* ModuleNamespaceAccessCase.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = ModuleNamespaceAccessCase.h; sourceTree = "<group>"; };
51F0EB6105C86C6B00E6DF1B /* Foundation.framework */ = {isa = PBXFileReference; lastKnownFileType = wrapper.framework; name = Foundation.framework; path = /System/Library/Frameworks/Foundation.framework; sourceTree = "<absolute>"; };
51F0EC0705C86C9A00E6DF1B /* libobjc.dylib */ = {isa = PBXFileReference; lastKnownFileType = "compiled.mach-o.dylib"; name = libobjc.dylib; path = /usr/lib/libobjc.dylib; sourceTree = "<absolute>"; };
- 520D99F02388CC78000509A3 /* GetByValHistory.h */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.c.h; path = GetByValHistory.h; sourceTree = "<group>"; };
521131F51F82BF11007CCEEE /* PolyProtoAccessChain.cpp */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.cpp.cpp; path = PolyProtoAccessChain.cpp; sourceTree = "<group>"; };
521131F61F82BF11007CCEEE /* PolyProtoAccessChain.h */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.c.h; path = PolyProtoAccessChain.h; sourceTree = "<group>"; };
521322431ECBCE8200F65615 /* WebAssemblyFunctionBase.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; name = WebAssemblyFunctionBase.cpp; path = js/WebAssemblyFunctionBase.cpp; sourceTree = "<group>"; };
@@ -3561,6 +3560,7 @@
527CE35322555FDD00C6F382 /* JSToWasmICCallee.h */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.c.h; name = JSToWasmICCallee.h; path = js/JSToWasmICCallee.h; sourceTree = "<group>"; };
52847AD921FFB8630061A9DB /* WasmAirIRGenerator.cpp */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.cpp.cpp; path = WasmAirIRGenerator.cpp; sourceTree = "<group>"; };
52847ADA21FFB8630061A9DB /* WasmAirIRGenerator.h */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.c.h; path = WasmAirIRGenerator.h; sourceTree = "<group>"; };
+ 52AD08222399A1F200B4101A /* PointerHistory.h */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.c.h; path = PointerHistory.h; sourceTree = "<group>"; };
52B310FA1974AE610080857C /* FunctionHasExecutedCache.h */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.c.h; path = FunctionHasExecutedCache.h; sourceTree = "<group>"; };
52B310FC1974AE870080857C /* FunctionHasExecutedCache.cpp */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.cpp.cpp; path = FunctionHasExecutedCache.cpp; sourceTree = "<group>"; };
52B310FE1975B4240080857C /* TypeLocationCache.cpp */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.cpp.cpp; path = TypeLocationCache.cpp; sourceTree = "<group>"; };
@@ -8241,7 +8241,6 @@
0F0332C218B01763005F979A /* GetByIdVariant.h */,
0F93329514CA7DC10085F3C6 /* GetByStatus.cpp */,
0F93329614CA7DC10085F3C6 /* GetByStatus.h */,
- 520D99F02388CC78000509A3 /* GetByValHistory.h */,
14AD91081DCA92940014F9FE /* GlobalCodeBlock.h */,
0F0B83A814BCF55E00885B4F /* HandlerInfo.h */,
0F44A7AB20BF685E0022B171 /* ICStatusMap.cpp */,
@@ -8297,6 +8296,7 @@
A70447E917A0BD4600F5898E /* OperandsInlines.h */,
E34E657420668E8E00FB81AC /* ParseHash.cpp */,
E34E657320668E8D00FB81AC /* ParseHash.h */,
+ 52AD08222399A1F200B4101A /* PointerHistory.h */,
0FF9CE711B9CD6D0004EDCA6 /* PolymorphicAccess.cpp */,
0FF9CE721B9CD6D0004EDCA6 /* PolymorphicAccess.h */,
521131F51F82BF11007CCEEE /* PolyProtoAccessChain.cpp */,
@@ -9526,7 +9526,6 @@
1498CAD6214BF36D00710879 /* GetByIdMetadata.h in Headers */,
0F0332C418B01763005F979A /* GetByIdVariant.h in Headers */,
0F9332A014CA7DCD0085F3C6 /* GetByStatus.h in Headers */,
- 520D99F12388CC81000509A3 /* GetByValHistory.h in Headers */,
7964656A1B952FF0003059EE /* GetPutInfo.h in Headers */,
534E03581E53BF2F00213F64 /* GetterSetterAccessCase.h in Headers */,
FE1D6D6F236258FE007A5C26 /* GetVM.h in Headers */,
@@ -10303,6 +10302,7 @@
FED94F2F171E3E2300BE77A4 /* Watchdog.h in Headers */,
0F919D2615853CE3004A4E7D /* Watchpoint.h in Headers */,
142E313C134FF0A600AFADB5 /* Weak.h in Headers */,
+ 52AD08232399A1FB00B4101A /* PointerHistory.h in Headers */,
14E84F9F14EE1ACC00D6D5D4 /* WeakBlock.h in Headers */,
14BFCE6910CDB1FC00364CCE /* WeakGCMap.h in Headers */,
AD86A93E1AA4D88D002FE77F /* WeakGCMapInlines.h in Headers */,
diff --git a/Source/JavaScriptCore/bytecode/BytecodeList.rb b/Source/JavaScriptCore/bytecode/BytecodeList.rb
index 71bf6b0..257a3ee 100644
--- a/Source/JavaScriptCore/bytecode/BytecodeList.rb
+++ b/Source/JavaScriptCore/bytecode/BytecodeList.rb
@@ -30,7 +30,7 @@
:ErrorType,
:GetByIdMode,
:GetByIdModeMetadata,
- :GetByValHistory,
+ :PointerHistory,
:GetPutInfo,
:IndexingType,
:JSCell,
@@ -451,6 +451,7 @@
metadata: {
modeMetadata: GetByIdModeMetadata,
profile: ValueProfile,
+ seenStructures: PointerHistory,
}
op :get_by_id_with_this,
@@ -462,6 +463,7 @@
},
metadata: {
profile: ValueProfile,
+ seenStructures: PointerHistory,
}
op :get_by_val_with_this,
@@ -485,6 +487,7 @@
profile: ValueProfile, # not used in llint
structureID: StructureID,
offset: unsigned,
+ seenStructures: PointerHistory,
}
op :try_get_by_id,
@@ -495,6 +498,7 @@
},
metadata: {
profile: ValueProfile,
+ seenStructures: PointerHistory,
}
op :put_by_id,
@@ -535,7 +539,7 @@
metadata: {
profile: ValueProfile,
arrayProfile: ArrayProfile,
- seenIdentifiers: GetByValHistory,
+ seenIdentifiers: PointerHistory,
}
op :put_by_val,
diff --git a/Source/JavaScriptCore/bytecode/GetByValHistory.h b/Source/JavaScriptCore/bytecode/PointerHistory.h
similarity index 80%
rename from Source/JavaScriptCore/bytecode/GetByValHistory.h
rename to Source/JavaScriptCore/bytecode/PointerHistory.h
index bdc0d33..1819364 100644
--- a/Source/JavaScriptCore/bytecode/GetByValHistory.h
+++ b/Source/JavaScriptCore/bytecode/PointerHistory.h
@@ -30,17 +30,11 @@
namespace JSC {
-struct GetByValHistory {
- void observeNonUID()
+struct PointerHistory {
+ void observe(const void* pointer)
{
- uintptr_t count = Options::getByValICMaxNumberOfIdentifiers() + 1;
- update(count, filter());
- }
-
- void observe(const UniquedStringImpl* impl)
- {
- if (!impl) {
- observeNonUID();
+ if (!pointer) {
+ observeNull();
return;
}
@@ -48,10 +42,10 @@
uintptr_t filter = this->filter();
TinyBloomFilter bloomFilter(filter);
- uintptr_t implBits = bitwise_cast<uintptr_t>(impl);
- ASSERT(((static_cast<uint64_t>(implBits) << 8) >> 8) == static_cast<uint64_t>(implBits));
- if (bloomFilter.ruleOut(implBits)) {
- bloomFilter.add(implBits);
+ uintptr_t pointerBits = bitwise_cast<uintptr_t>(pointer);
+ ASSERT(((static_cast<uint64_t>(pointerBits) << 8) >> 8) == static_cast<uint64_t>(pointerBits));
+ if (bloomFilter.ruleOut(pointerBits)) {
+ bloomFilter.add(pointerBits);
++count;
update(count, bloomFilter.bits());
}
@@ -60,6 +54,11 @@
uintptr_t count() const { return static_cast<uintptr_t>(m_payload >> 56); }
private:
+ void observeNull()
+ {
+ update(count() + 1, filter());
+ }
+
uintptr_t filter() const { return static_cast<uintptr_t>((m_payload << 8) >> 8); }
void update(uint64_t count, uint64_t filter)
diff --git a/Source/JavaScriptCore/dfg/DFGByteCodeParser.cpp b/Source/JavaScriptCore/dfg/DFGByteCodeParser.cpp
index 79cd4ad..dcc03ae 100644
--- a/Source/JavaScriptCore/dfg/DFGByteCodeParser.cpp
+++ b/Source/JavaScriptCore/dfg/DFGByteCodeParser.cpp
@@ -4712,11 +4712,6 @@
Node* base = get(bytecode.m_base);
unsigned identifierNumber = m_inlineStackTop->m_identifierRemap[bytecode.m_property];
-
- GetByStatus getByStatus = GetByStatus::computeFor(
- m_inlineStackTop->m_profiledBlock,
- m_inlineStackTop->m_baselineMap, m_icContextStack,
- currentCodeOrigin(), GetByStatus::TrackIdentifiers::No);
AccessType type = AccessType::GetById;
unsigned opcodeLength = currentInstruction->size();
@@ -4724,10 +4719,29 @@
type = AccessType::TryGetById;
else if (Op::opcodeID == op_get_by_id_direct)
type = AccessType::GetByIdDirect;
+
+ if (bytecode.metadata(m_inlineStackTop->m_codeBlock).m_seenStructures.count() > Options::getByIdICMaxNumberOfStructures()) {
+ NodeType getById;
+ if (type == AccessType::GetById)
+ getById = GetById;
+ else if (type == AccessType::TryGetById)
+ getById = TryGetById;
+ else
+ getById = GetByIdDirect;
+
+ Node* node = addToGraph(getById, OpInfo(identifierNumber), OpInfo(prediction), base);
+ set(bytecode.m_dst, node);
+ m_graph.m_shouldSkipIC.add(node);
+ return;
+ }
+
+ GetByStatus getByStatus = GetByStatus::computeFor(
+ m_inlineStackTop->m_profiledBlock,
+ m_inlineStackTop->m_baselineMap, m_icContextStack,
+ currentCodeOrigin(), GetByStatus::TrackIdentifiers::No);
handleGetById(
bytecode.m_dst, prediction, base, identifierNumber, getByStatus, type, opcodeLength);
-
}
static uint64_t makeDynamicVarOpInfo(unsigned identifierNumber, unsigned getPutInfo)
@@ -5656,7 +5670,7 @@
m_exitOK = false; // GetByVal must be treated as if it clobbers exit state, since FixupPhase may make it generic.
set(bytecode.m_dst, getByVal);
if (getByStatus.observedStructureStubInfoSlowPath() || bytecode.metadata(codeBlock).m_seenIdentifiers.count() > Options::getByValICMaxNumberOfIdentifiers())
- m_graph.m_slowGetByVal.add(getByVal);
+ m_graph.m_shouldSkipIC.add(getByVal);
}
NEXT_OPCODE(op_get_by_val);
diff --git a/Source/JavaScriptCore/dfg/DFGGraph.h b/Source/JavaScriptCore/dfg/DFGGraph.h
index fab9cf0..69173a6 100644
--- a/Source/JavaScriptCore/dfg/DFGGraph.h
+++ b/Source/JavaScriptCore/dfg/DFGGraph.h
@@ -1153,7 +1153,7 @@
RegisteredStructure stringStructure;
RegisteredStructure symbolStructure;
- HashSet<Node*> m_slowGetByVal;
+ HashSet<Node*> m_shouldSkipIC;
private:
bool isStringPrototypeMethodSane(JSGlobalObject*, UniquedStringImpl*);
diff --git a/Source/JavaScriptCore/dfg/DFGSpeculativeJIT.cpp b/Source/JavaScriptCore/dfg/DFGSpeculativeJIT.cpp
index 77e9b77..9f9ec182 100644
--- a/Source/JavaScriptCore/dfg/DFGSpeculativeJIT.cpp
+++ b/Source/JavaScriptCore/dfg/DFGSpeculativeJIT.cpp
@@ -986,6 +986,20 @@
{
ASSERT(accessType == AccessType::GetById || accessType == AccessType::GetByIdDirect || accessType == AccessType::TryGetById);
+ if (m_graph.m_shouldSkipIC.contains(node)) {
+ JSValueOperand base(this, node->child1(), ManualOperandSpeculation);
+ speculate(node, node->child1());
+ JSValueRegs baseRegs = base.jsValueRegs();
+
+ flushRegisters();
+ JSValueRegsFlushedCallResult result(this);
+ JSValueRegs resultRegs = result.regs();
+ callOperation(appropriateGenericGetByIdFunction(accessType), resultRegs, TrustedImmPtr::weakPointer(m_graph, m_graph.globalObjectFor(node->origin.semantic)), baseRegs, TrustedImmPtr(identifierUID(node->identifierNumber())));
+ m_jit.exceptionCheck();
+ jsValueResult(resultRegs, node, DataFormatJS);
+ return;
+ }
+
switch (node->child1().useKind()) {
case CellUse: {
SpeculateCellOperand base(this, node->child1());
diff --git a/Source/JavaScriptCore/dfg/DFGSpeculativeJIT32_64.cpp b/Source/JavaScriptCore/dfg/DFGSpeculativeJIT32_64.cpp
index 67b0e0b..72b4b45 100644
--- a/Source/JavaScriptCore/dfg/DFGSpeculativeJIT32_64.cpp
+++ b/Source/JavaScriptCore/dfg/DFGSpeculativeJIT32_64.cpp
@@ -2259,7 +2259,7 @@
break;
}
case Array::Generic: {
- if (m_graph.m_slowGetByVal.contains(node)) {
+ if (m_graph.m_shouldSkipIC.contains(node)) {
if (m_graph.varArgChild(node, 0).useKind() == ObjectUse) {
if (m_graph.varArgChild(node, 1).useKind() == StringUse) {
compileGetByValForObjectWithString(node);
diff --git a/Source/JavaScriptCore/dfg/DFGSpeculativeJIT64.cpp b/Source/JavaScriptCore/dfg/DFGSpeculativeJIT64.cpp
index 3e638c9..d685770 100644
--- a/Source/JavaScriptCore/dfg/DFGSpeculativeJIT64.cpp
+++ b/Source/JavaScriptCore/dfg/DFGSpeculativeJIT64.cpp
@@ -2408,7 +2408,7 @@
break;
}
case Array::Generic: {
- if (m_graph.m_slowGetByVal.contains(node)) {
+ if (m_graph.m_shouldSkipIC.contains(node)) {
if (m_graph.varArgChild(node, 0).useKind() == ObjectUse) {
if (m_graph.varArgChild(node, 1).useKind() == StringUse) {
compileGetByValForObjectWithString(node);
diff --git a/Source/JavaScriptCore/ftl/FTLLowerDFGToB3.cpp b/Source/JavaScriptCore/ftl/FTLLowerDFGToB3.cpp
index bed3b84..bf3a4d7 100644
--- a/Source/JavaScriptCore/ftl/FTLLowerDFGToB3.cpp
+++ b/Source/JavaScriptCore/ftl/FTLLowerDFGToB3.cpp
@@ -3546,6 +3546,16 @@
{
ASSERT(type == AccessType::GetById || type == AccessType::TryGetById || type == AccessType::GetByIdDirect);
JSGlobalObject* globalObject = m_graph.globalObjectFor(m_node->origin.semantic);
+
+ if (m_graph.m_shouldSkipIC.contains(m_node)) {
+ speculate(m_node->child1());
+ LValue base = lowJSValue(m_node->child1(), ManualOperandSpeculation);
+
+ setJSValue(vmCall(Int64, appropriateGenericGetByIdFunction(type),
+ weakPointer(globalObject), base, m_out.constIntPtr(m_graph.identifiers()[m_node->identifierNumber()])));
+ return;
+ }
+
switch (m_node->child1().useKind()) {
case CellUse: {
setJSValue(getById(lowCell(m_node->child1()), type));
@@ -4453,7 +4463,7 @@
}
case Array::Generic: {
- if (m_graph.m_slowGetByVal.contains(m_node)) {
+ if (m_graph.m_shouldSkipIC.contains(m_node)) {
if (m_graph.varArgChild(m_node, 0).useKind() == ObjectUse) {
if (m_graph.varArgChild(m_node, 1).useKind() == StringUse) {
setJSValue(vmCall(
diff --git a/Source/JavaScriptCore/generator/DSL.rb b/Source/JavaScriptCore/generator/DSL.rb
index a78b19d..d7ed3aa 100644
--- a/Source/JavaScriptCore/generator/DSL.rb
+++ b/Source/JavaScriptCore/generator/DSL.rb
@@ -136,7 +136,7 @@
#include "BytecodeDumper.h"
#include "Fits.h"
#include "GetByIdMetadata.h"
-#include "GetByValHistory.h"
+#include "PointerHistory.h"
#include "Instruction.h"
#include "Opcode.h"
#include "PutByIdStatus.h"
diff --git a/Source/JavaScriptCore/jit/JITPropertyAccess.cpp b/Source/JavaScriptCore/jit/JITPropertyAccess.cpp
index ab99fd9..9dafb56 100644
--- a/Source/JavaScriptCore/jit/JITPropertyAccess.cpp
+++ b/Source/JavaScriptCore/jit/JITPropertyAccess.cpp
@@ -409,37 +409,44 @@
int resultVReg = bytecode.m_dst.offset();
int baseVReg = bytecode.m_base.offset();
const Identifier* ident = &(m_codeBlock->identifier(bytecode.m_property));
+ auto& metadata = bytecode.metadata(m_codeBlock);
emitGetVirtualRegister(baseVReg, regT0);
- emitJumpSlowCaseIfNotJSCell(regT0, baseVReg);
+ if (metadata.m_seenStructures.count() > Options::getByIdICMaxNumberOfStructures())
+ callOperationWithProfile(bytecode.metadata(m_codeBlock), operationTryGetByIdGeneric, resultVReg, TrustedImmPtr(m_codeBlock->globalObject()), regT0, TrustedImmPtr(ident->impl()));
+ else {
+ emitJumpSlowCaseIfNotJSCell(regT0, baseVReg);
- JITGetByIdGenerator gen(
- m_codeBlock, CodeOrigin(m_bytecodeIndex), CallSiteIndex(m_bytecodeIndex), RegisterSet::stubUnavailableRegisters(),
- ident->impl(), JSValueRegs(regT0), JSValueRegs(regT0), AccessType::TryGetById);
- gen.generateFastPath(*this);
- addSlowCase(gen.slowPathJump());
- m_getByIds.append(gen);
-
- emitValueProfilingSite(bytecode.metadata(m_codeBlock));
- emitPutVirtualRegister(resultVReg);
+ JITGetByIdGenerator gen(
+ m_codeBlock, CodeOrigin(m_bytecodeIndex), CallSiteIndex(m_bytecodeIndex), RegisterSet::stubUnavailableRegisters(),
+ ident->impl(), JSValueRegs(regT0), JSValueRegs(regT0), AccessType::TryGetById);
+ gen.generateFastPath(*this);
+ addSlowCase(gen.slowPathJump());
+ m_getByIds.append(gen);
+
+ emitValueProfilingSite(bytecode.metadata(m_codeBlock));
+ emitPutVirtualRegister(resultVReg);
+ }
}
void JIT::emitSlow_op_try_get_by_id(const Instruction* currentInstruction, Vector<SlowCaseEntry>::iterator& iter)
{
- linkAllSlowCases(iter);
+ if (hasAnySlowCases(iter)) {
+ linkAllSlowCases(iter);
- auto bytecode = currentInstruction->as<OpTryGetById>();
- int resultVReg = bytecode.m_dst.offset();
- const Identifier* ident = &(m_codeBlock->identifier(bytecode.m_property));
+ auto bytecode = currentInstruction->as<OpTryGetById>();
+ int resultVReg = bytecode.m_dst.offset();
+ const Identifier* ident = &(m_codeBlock->identifier(bytecode.m_property));
- JITGetByIdGenerator& gen = m_getByIds[m_getByIdIndex++];
+ JITGetByIdGenerator& gen = m_getByIds[m_getByIdIndex++];
- Label coldPathBegin = label();
+ Label coldPathBegin = label();
- Call call = callOperation(operationTryGetByIdOptimize, resultVReg, TrustedImmPtr(m_codeBlock->globalObject()), gen.stubInfo(), regT0, ident->impl());
-
- gen.reportSlowPathCall(coldPathBegin, call);
+ Call call = callOperation(operationTryGetByIdOptimize, resultVReg, TrustedImmPtr(m_codeBlock->globalObject()), gen.stubInfo(), regT0, ident->impl());
+
+ gen.reportSlowPathCall(coldPathBegin, call);
+ }
}
void JIT::emit_op_get_by_id_direct(const Instruction* currentInstruction)
@@ -448,37 +455,44 @@
int resultVReg = bytecode.m_dst.offset();
int baseVReg = bytecode.m_base.offset();
const Identifier* ident = &(m_codeBlock->identifier(bytecode.m_property));
+ auto& metadata = bytecode.metadata(m_codeBlock);
emitGetVirtualRegister(baseVReg, regT0);
- emitJumpSlowCaseIfNotJSCell(regT0, baseVReg);
+ if (metadata.m_seenStructures.count() > Options::getByIdICMaxNumberOfStructures())
+ callOperationWithProfile(bytecode.metadata(m_codeBlock), operationGetByIdDirectGeneric, resultVReg, TrustedImmPtr(m_codeBlock->globalObject()), regT0, TrustedImmPtr(ident->impl()));
+ else {
+ emitJumpSlowCaseIfNotJSCell(regT0, baseVReg);
- JITGetByIdGenerator gen(
- m_codeBlock, CodeOrigin(m_bytecodeIndex), CallSiteIndex(m_bytecodeIndex), RegisterSet::stubUnavailableRegisters(),
- ident->impl(), JSValueRegs(regT0), JSValueRegs(regT0), AccessType::GetByIdDirect);
- gen.generateFastPath(*this);
- addSlowCase(gen.slowPathJump());
- m_getByIds.append(gen);
+ JITGetByIdGenerator gen(
+ m_codeBlock, CodeOrigin(m_bytecodeIndex), CallSiteIndex(m_bytecodeIndex), RegisterSet::stubUnavailableRegisters(),
+ ident->impl(), JSValueRegs(regT0), JSValueRegs(regT0), AccessType::GetByIdDirect);
+ gen.generateFastPath(*this);
+ addSlowCase(gen.slowPathJump());
+ m_getByIds.append(gen);
- emitValueProfilingSite(bytecode.metadata(m_codeBlock));
- emitPutVirtualRegister(resultVReg);
+ emitValueProfilingSite(bytecode.metadata(m_codeBlock));
+ emitPutVirtualRegister(resultVReg);
+ }
}
void JIT::emitSlow_op_get_by_id_direct(const Instruction* currentInstruction, Vector<SlowCaseEntry>::iterator& iter)
{
- linkAllSlowCases(iter);
+ if (hasAnySlowCases(iter)) {
+ linkAllSlowCases(iter);
- auto bytecode = currentInstruction->as<OpGetByIdDirect>();
- int resultVReg = bytecode.m_dst.offset();
- const Identifier* ident = &(m_codeBlock->identifier(bytecode.m_property));
+ auto bytecode = currentInstruction->as<OpGetByIdDirect>();
+ int resultVReg = bytecode.m_dst.offset();
+ const Identifier* ident = &(m_codeBlock->identifier(bytecode.m_property));
- JITGetByIdGenerator& gen = m_getByIds[m_getByIdIndex++];
+ JITGetByIdGenerator& gen = m_getByIds[m_getByIdIndex++];
- Label coldPathBegin = label();
+ Label coldPathBegin = label();
- Call call = callOperationWithProfile(bytecode.metadata(m_codeBlock), operationGetByIdDirectOptimize, resultVReg, TrustedImmPtr(m_codeBlock->globalObject()), gen.stubInfo(), regT0, ident->impl());
+ Call call = callOperationWithProfile(bytecode.metadata(m_codeBlock), operationGetByIdDirectOptimize, resultVReg, TrustedImmPtr(m_codeBlock->globalObject()), gen.stubInfo(), regT0, ident->impl());
- gen.reportSlowPathCall(coldPathBegin, call);
+ gen.reportSlowPathCall(coldPathBegin, call);
+ }
}
void JIT::emit_op_get_by_id(const Instruction* currentInstruction)
@@ -490,24 +504,54 @@
const Identifier* ident = &(m_codeBlock->identifier(bytecode.m_property));
emitGetVirtualRegister(baseVReg, regT0);
-
- emitJumpSlowCaseIfNotJSCell(regT0, baseVReg);
-
- if (*ident == m_vm->propertyNames->length && shouldEmitProfiling()) {
- Jump notArrayLengthMode = branch8(NotEqual, AbsoluteAddress(&metadata.m_modeMetadata.mode), TrustedImm32(static_cast<uint8_t>(GetByIdMode::ArrayLength)));
- emitArrayProfilingSiteWithCell(regT0, regT1, &metadata.m_modeMetadata.arrayLengthMode.arrayProfile);
- notArrayLengthMode.link(this);
+
+ if (metadata.m_seenStructures.count() > Options::getByIdICMaxNumberOfStructures()) {
+ if (*ident == m_vm->propertyNames->length && shouldEmitProfiling()) {
+ auto notCell = branchIfNotCell(regT0);
+ Jump notArrayLengthMode = branch8(NotEqual, AbsoluteAddress(&metadata.m_modeMetadata.mode), TrustedImm32(static_cast<uint8_t>(GetByIdMode::ArrayLength)));
+ emitArrayProfilingSiteWithCell(regT0, regT1, &metadata.m_modeMetadata.arrayLengthMode.arrayProfile);
+ notArrayLengthMode.link(this);
+ notCell.link(this);
+ }
+ callOperationWithProfile(bytecode.metadata(m_codeBlock), operationGetByIdGeneric, resultVReg, TrustedImmPtr(m_codeBlock->globalObject()), regT0, TrustedImmPtr(ident->impl()));
+ } else {
+ emitJumpSlowCaseIfNotJSCell(regT0, baseVReg);
+
+ if (*ident == m_vm->propertyNames->length && shouldEmitProfiling()) {
+ Jump notArrayLengthMode = branch8(NotEqual, AbsoluteAddress(&metadata.m_modeMetadata.mode), TrustedImm32(static_cast<uint8_t>(GetByIdMode::ArrayLength)));
+ emitArrayProfilingSiteWithCell(regT0, regT1, &metadata.m_modeMetadata.arrayLengthMode.arrayProfile);
+ notArrayLengthMode.link(this);
+ }
+
+ JITGetByIdGenerator gen(
+ m_codeBlock, CodeOrigin(m_bytecodeIndex), CallSiteIndex(m_bytecodeIndex), RegisterSet::stubUnavailableRegisters(),
+ ident->impl(), JSValueRegs(regT0), JSValueRegs(regT0), AccessType::GetById);
+ gen.generateFastPath(*this);
+ addSlowCase(gen.slowPathJump());
+ m_getByIds.append(gen);
+
+ emitValueProfilingSite(bytecode.metadata(m_codeBlock));
+ emitPutVirtualRegister(resultVReg);
}
+}
- JITGetByIdGenerator gen(
- m_codeBlock, CodeOrigin(m_bytecodeIndex), CallSiteIndex(m_bytecodeIndex), RegisterSet::stubUnavailableRegisters(),
- ident->impl(), JSValueRegs(regT0), JSValueRegs(regT0), AccessType::GetById);
- gen.generateFastPath(*this);
- addSlowCase(gen.slowPathJump());
- m_getByIds.append(gen);
+void JIT::emitSlow_op_get_by_id(const Instruction* currentInstruction, Vector<SlowCaseEntry>::iterator& iter)
+{
+ if (hasAnySlowCases(iter)) {
+ linkAllSlowCases(iter);
- emitValueProfilingSite(bytecode.metadata(m_codeBlock));
- emitPutVirtualRegister(resultVReg);
+ auto bytecode = currentInstruction->as<OpGetById>();
+ int resultVReg = bytecode.m_dst.offset();
+ const Identifier* ident = &(m_codeBlock->identifier(bytecode.m_property));
+
+ JITGetByIdGenerator& gen = m_getByIds[m_getByIdIndex++];
+
+ Label coldPathBegin = label();
+
+ Call call = callOperationWithProfile(bytecode.metadata(m_codeBlock), operationGetByIdOptimize, resultVReg, TrustedImmPtr(m_codeBlock->globalObject()), gen.stubInfo(), regT0, ident->impl());
+
+ gen.reportSlowPathCall(coldPathBegin, call);
+ }
}
void JIT::emit_op_get_by_id_with_this(const Instruction* currentInstruction)
@@ -534,23 +578,6 @@
emitPutVirtualRegister(resultVReg);
}
-void JIT::emitSlow_op_get_by_id(const Instruction* currentInstruction, Vector<SlowCaseEntry>::iterator& iter)
-{
- linkAllSlowCases(iter);
-
- auto bytecode = currentInstruction->as<OpGetById>();
- int resultVReg = bytecode.m_dst.offset();
- const Identifier* ident = &(m_codeBlock->identifier(bytecode.m_property));
-
- JITGetByIdGenerator& gen = m_getByIds[m_getByIdIndex++];
-
- Label coldPathBegin = label();
-
- Call call = callOperationWithProfile(bytecode.metadata(m_codeBlock), operationGetByIdOptimize, resultVReg, TrustedImmPtr(m_codeBlock->globalObject()), gen.stubInfo(), regT0, ident->impl());
-
- gen.reportSlowPathCall(coldPathBegin, call);
-}
-
void JIT::emitSlow_op_get_by_id_with_this(const Instruction* currentInstruction, Vector<SlowCaseEntry>::iterator& iter)
{
linkAllSlowCases(iter);
diff --git a/Source/JavaScriptCore/llint/LLIntSlowPaths.cpp b/Source/JavaScriptCore/llint/LLIntSlowPaths.cpp
index 357a46e..e9bfeab 100644
--- a/Source/JavaScriptCore/llint/LLIntSlowPaths.cpp
+++ b/Source/JavaScriptCore/llint/LLIntSlowPaths.cpp
@@ -640,8 +640,12 @@
{
LLINT_BEGIN();
auto bytecode = pc->as<OpTryGetById>();
+ auto& metadata = bytecode.metadata(codeBlock);
+
const Identifier& ident = codeBlock->identifier(bytecode.m_property);
JSValue baseValue = getOperand(callFrame, bytecode.m_base);
+ if (metadata.m_seenStructures.count() <= Options::getByIdICMaxNumberOfStructures())
+ metadata.m_seenStructures.observe(baseValue.structureOrNull());
PropertySlot slot(baseValue, PropertySlot::PropertySlot::InternalMethodType::VMInquiry);
baseValue.getPropertySlot(globalObject, ident, slot);
@@ -654,8 +658,11 @@
{
LLINT_BEGIN();
auto bytecode = pc->as<OpGetByIdDirect>();
+ auto& metadata = bytecode.metadata(codeBlock);
const Identifier& ident = codeBlock->identifier(bytecode.m_property);
JSValue baseValue = getOperand(callFrame, bytecode.m_base);
+ if (metadata.m_seenStructures.count() <= Options::getByIdICMaxNumberOfStructures())
+ metadata.m_seenStructures.observe(baseValue.structureOrNull());
PropertySlot slot(baseValue, PropertySlot::PropertySlot::InternalMethodType::GetOwnProperty);
bool found = baseValue.getOwnPropertySlot(globalObject, ident, slot);
@@ -664,7 +671,6 @@
LLINT_CHECK_EXCEPTION();
if (!LLINT_ALWAYS_ACCESS_SLOW && slot.isCacheable() && !slot.isUnset()) {
- auto& metadata = bytecode.metadata(codeBlock);
{
StructureID oldStructureID = metadata.m_structureID;
if (oldStructureID) {
@@ -765,6 +771,10 @@
auto& metadata = bytecode.metadata(codeBlock);
const Identifier& ident = codeBlock->identifier(bytecode.m_property);
JSValue baseValue = getOperand(callFrame, bytecode.m_base);
+
+ if (metadata.m_seenStructures.count() <= Options::getByIdICMaxNumberOfStructures())
+ metadata.m_seenStructures.observe(baseValue.structureOrNull());
+
PropertySlot slot(baseValue, PropertySlot::PropertySlot::InternalMethodType::Get);
JSValue result = baseValue.get(globalObject, ident, slot);
diff --git a/Source/JavaScriptCore/runtime/OptionsList.h b/Source/JavaScriptCore/runtime/OptionsList.h
index 03f2852..1dc5262 100644
--- a/Source/JavaScriptCore/runtime/OptionsList.h
+++ b/Source/JavaScriptCore/runtime/OptionsList.h
@@ -494,6 +494,7 @@
v(Bool, useUnlinkedCodeBlockJettisoning, false, Normal, "If true, UnlinkedCodeBlock can be jettisoned.") \
v(Bool, forceOSRExitToLLInt, false, Normal, "If true, we always exit to the LLInt. If false, we exit to whatever is most convenient.") \
v(Unsigned, getByValICMaxNumberOfIdentifiers, 4, Normal, "Number of identifiers we see in the LLInt that could cause us to bail on generating an IC for get_by_val.") \
+ v(Unsigned, getByIdICMaxNumberOfStructures, 5, Normal, "Number of structures we see in the LLInt that could cause us to bail on generating an IC for various get_by_id ICs.") \
enum OptionEquivalence {
SameOption,
