Remove Gigacage from arm64 and use PAC for arm64e instead
https://bugs.webkit.org/show_bug.cgi?id=197110
Reviewed by Saam Barati.
Source/bmalloc:
Stop using gigacage on arm64 and add a new cage function cagedMayBeNull that is the same as
cage but returns a nullptr if the incoming pointer is already null.
* bmalloc/Gigacage.h:
(Gigacage::cagedMayBeNull):
Source/JavaScriptCore:
This patch makes a bunch of changes. I'll start with global changes then go over changes to each tier and finish with bug fixes.
Global Changes:
Change CagedBarrierPtr to work with PAC so constructors and accessors now expect to receive a length.
Update assembler helper methods to use do PAC when caging.
LLInt:
Add arm64e.rb backend as we missed that when originally open sourcing our arm64e code.
Add a new optional t6 temporary, which is only used currently on arm64e for GetByVal on a TypedArray.
Refactor caging into two helper macros for Primitive/JSValue cages.
Baseline/DFG:
Add authentication where needed for GetByVal and inline object construction.
FTL:
Add a new ValueRep that allows for a late register use. We want this for the authentication patchpoint since we use the length register at the same time as we are defing the authenticated pointer.
Wasm:
Use the TaggedArrayStoragePtr class for the memory base pointer. In theory we should be caging those pointers but I don't want to risk introducing a performance regression with the rest of this change. I've filed https://bugs.webkit.org/show_bug.cgi?id=197620 to do this later.
As we no longer have the Gigacage using most of our VA memory, we can enable fast memories on iOS.
Using fast memories leads to roughly a 2% JetStream2 speedup.
* assembler/MacroAssemblerARM64E.h:
(JSC::MacroAssemblerARM64E::tagArrayPtr):
(JSC::MacroAssemblerARM64E::untagArrayPtr):
(JSC::MacroAssemblerARM64E::removeArrayPtrTag):
* b3/B3LowerToAir.cpp:
* b3/B3PatchpointSpecial.cpp:
(JSC::B3::PatchpointSpecial::admitsStack):
* b3/B3StackmapSpecial.cpp:
(JSC::B3::StackmapSpecial::forEachArgImpl):
(JSC::B3::StackmapSpecial::isArgValidForRep):
* b3/B3Validate.cpp:
* b3/B3ValueRep.cpp:
(JSC::B3::ValueRep::addUsedRegistersTo const):
(JSC::B3::ValueRep::dump const):
(WTF::printInternal):
* b3/B3ValueRep.h:
(JSC::B3::ValueRep::ValueRep):
(JSC::B3::ValueRep::isReg const):
* dfg/DFGOperations.cpp:
(JSC::DFG::newTypedArrayWithSize):
* dfg/DFGSpeculativeJIT.cpp:
(JSC::DFG::SpeculativeJIT::jumpForTypedArrayIsNeuteredIfOutOfBounds):
(JSC::DFG::SpeculativeJIT::cageTypedArrayStorage):
(JSC::DFG::SpeculativeJIT::compileGetIndexedPropertyStorage):
(JSC::DFG::SpeculativeJIT::compileGetTypedArrayByteOffset):
(JSC::DFG::SpeculativeJIT::compileNewTypedArrayWithSize):
* dfg/DFGSpeculativeJIT.h:
* dfg/DFGSpeculativeJIT64.cpp:
(JSC::DFG::SpeculativeJIT::compile):
* ftl/FTLLowerDFGToB3.cpp:
(JSC::FTL::DFG::LowerDFGToB3::compileGetIndexedPropertyStorage):
(JSC::FTL::DFG::LowerDFGToB3::compileGetTypedArrayByteOffset):
(JSC::FTL::DFG::LowerDFGToB3::compileNewTypedArray):
(JSC::FTL::DFG::LowerDFGToB3::compileDataViewGet):
(JSC::FTL::DFG::LowerDFGToB3::compileDataViewSet):
(JSC::FTL::DFG::LowerDFGToB3::untagArrayPtr):
(JSC::FTL::DFG::LowerDFGToB3::caged):
(JSC::FTL::DFG::LowerDFGToB3::speculateTypedArrayIsNotNeutered):
* jit/AssemblyHelpers.h:
(JSC::AssemblyHelpers::cageConditionally):
* jit/IntrinsicEmitter.cpp:
(JSC::IntrinsicGetterAccessCase::emitIntrinsicGetter):
* jit/JITPropertyAccess.cpp:
(JSC::JIT::emitDirectArgumentsGetByVal):
(JSC::JIT::emitIntTypedArrayGetByVal):
(JSC::JIT::emitFloatTypedArrayGetByVal):
(JSC::JIT::emitIntTypedArrayPutByVal):
(JSC::JIT::emitFloatTypedArrayPutByVal):
* jit/PolymorphicCallStubRoutine.cpp:
(JSC::PolymorphicCallNode::clearCallLinkInfo):
* llint/LowLevelInterpreter64.asm:
* offlineasm/arm64.rb:
* offlineasm/arm64e.rb: Added.
* offlineasm/ast.rb:
* offlineasm/instructions.rb:
* offlineasm/registers.rb:
* offlineasm/x86.rb:
* runtime/ArrayBuffer.cpp:
(JSC::SharedArrayBufferContents::SharedArrayBufferContents):
(JSC::SharedArrayBufferContents::~SharedArrayBufferContents):
(JSC::ArrayBufferContents::ArrayBufferContents):
(JSC::ArrayBufferContents::destroy):
(JSC::ArrayBufferContents::tryAllocate):
(JSC::ArrayBufferContents::makeShared):
(JSC::ArrayBufferContents::copyTo):
* runtime/ArrayBuffer.h:
(JSC::SharedArrayBufferContents::data const):
(JSC::ArrayBufferContents::data const):
(JSC::ArrayBuffer::data):
(JSC::ArrayBuffer::data const):
(JSC::ArrayBuffer::byteLength const):
* runtime/ArrayBufferView.cpp:
(JSC::ArrayBufferView::ArrayBufferView):
* runtime/ArrayBufferView.h:
(JSC::ArrayBufferView::baseAddress const):
(JSC::ArrayBufferView::byteLength const):
(JSC::ArrayBufferView::setRangeImpl):
(JSC::ArrayBufferView::getRangeImpl):
* runtime/CachedTypes.cpp:
(JSC::CachedScopedArgumentsTable::encode):
(JSC::CachedScopedArgumentsTable::decode const):
* runtime/CagedBarrierPtr.h:
(JSC::CagedBarrierPtr::CagedBarrierPtr):
(JSC::CagedBarrierPtr::set):
(JSC::CagedBarrierPtr::get const):
(JSC::CagedBarrierPtr::getMayBeNull const):
(JSC::CagedBarrierPtr::getUnsafe const):
(JSC::CagedBarrierPtr::at const):
(JSC::CagedBarrierPtr::operator== const):
(JSC::CagedBarrierPtr::operator bool const):
(JSC::CagedBarrierPtr::setWithoutBarrier):
(JSC::CagedBarrierPtr::operator* const): Deleted.
(JSC::CagedBarrierPtr::operator-> const): Deleted.
(JSC::CagedBarrierPtr::operator[] const): Deleted.
(): Deleted.
* runtime/DataView.cpp:
(JSC::DataView::DataView):
* runtime/DataView.h:
(JSC::DataView::get):
(JSC::DataView::set):
* runtime/DirectArguments.cpp:
(JSC::DirectArguments::visitChildren):
(JSC::DirectArguments::overrideThings):
(JSC::DirectArguments::unmapArgument):
* runtime/DirectArguments.h:
* runtime/GenericArguments.h:
* runtime/GenericArgumentsInlines.h:
(JSC::GenericArguments<Type>::visitChildren):
(JSC::GenericArguments<Type>::initModifiedArgumentsDescriptor):
(JSC::GenericArguments<Type>::setModifiedArgumentDescriptor):
(JSC::GenericArguments<Type>::isModifiedArgumentDescriptor):
* runtime/GenericTypedArrayView.h:
* runtime/GenericTypedArrayViewInlines.h:
(JSC::GenericTypedArrayView<Adaptor>::GenericTypedArrayView):
* runtime/JSArrayBufferView.cpp:
(JSC::JSArrayBufferView::ConstructionContext::ConstructionContext):
(JSC::JSArrayBufferView::JSArrayBufferView):
(JSC::JSArrayBufferView::finalize):
(JSC::JSArrayBufferView::slowDownAndWasteMemory):
* runtime/JSArrayBufferView.h:
(JSC::JSArrayBufferView::ConstructionContext::vector const):
(JSC::JSArrayBufferView::isNeutered):
(JSC::JSArrayBufferView::hasVector const):
(JSC::JSArrayBufferView::vector const):
* runtime/JSGenericTypedArrayViewInlines.h:
(JSC::JSGenericTypedArrayView<Adaptor>::createUninitialized):
(JSC::JSGenericTypedArrayView<Adaptor>::estimatedSize):
(JSC::JSGenericTypedArrayView<Adaptor>::visitChildren):
* runtime/Options.h:
* runtime/ScopedArgumentsTable.cpp:
(JSC::ScopedArgumentsTable::clone):
(JSC::ScopedArgumentsTable::setLength):
* runtime/ScopedArgumentsTable.h:
* runtime/SymbolTable.h:
* wasm/WasmAirIRGenerator.cpp:
(JSC::Wasm::AirIRGenerator::restoreWebAssemblyGlobalState):
(JSC::Wasm::AirIRGenerator::addCallIndirect):
* wasm/WasmB3IRGenerator.cpp:
(JSC::Wasm::B3IRGenerator::restoreWebAssemblyGlobalState):
(JSC::Wasm::B3IRGenerator::addCallIndirect):
* wasm/WasmBBQPlan.cpp:
(JSC::Wasm::BBQPlan::complete):
* wasm/WasmBinding.cpp:
(JSC::Wasm::wasmToWasm):
* wasm/WasmInstance.h:
(JSC::Wasm::Instance::cachedMemory const):
(JSC::Wasm::Instance::updateCachedMemory):
* wasm/WasmMemory.cpp:
(JSC::Wasm::Memory::Memory):
(JSC::Wasm::Memory::~Memory):
(JSC::Wasm::Memory::grow):
(JSC::Wasm::Memory::dump const):
* wasm/WasmMemory.h:
(JSC::Wasm::Memory::memory const):
* wasm/js/JSToWasm.cpp:
(JSC::Wasm::createJSToWasmWrapper):
* wasm/js/WebAssemblyFunction.cpp:
(JSC::WebAssemblyFunction::jsCallEntrypointSlow):
Source/WTF:
This patch changes the Gigacage to use PAC on arm64e. As part of
this process all platforms must provide their length when
materializing the caged pointer. Since it would be somewhat
confusing to have two parameters for an operator [] those methods
have been removed. Lastly, this patch removes the specializations
for void* caged pointers, instead opting to use enable_if on the
methods that would normally fail on void* e.g. anything that
returns a T&.
* WTF.xcodeproj/project.pbxproj:
* wtf/CMakeLists.txt:
* wtf/CagedPtr.h:
(WTF::CagedPtr::CagedPtr):
(WTF::CagedPtr::get const):
(WTF::CagedPtr::getMayBeNull const):
(WTF::CagedPtr::getUnsafe const):
(WTF::CagedPtr::at const):
(WTF::CagedPtr::reauthenticate):
(WTF::CagedPtr::operator=):
(WTF::CagedPtr::operator== const):
(WTF::CagedPtr::operator bool const):
(WTF::CagedPtr::operator* const): Deleted.
(WTF::CagedPtr::operator-> const): Deleted.
(WTF::CagedPtr::operator[] const): Deleted.
(): Deleted.
* wtf/CagedUniquePtr.h:
(WTF::CagedUniquePtr::CagedUniquePtr):
(WTF::CagedUniquePtr::create):
(WTF::CagedUniquePtr::~CagedUniquePtr):
(WTF::CagedUniquePtr::destroy):
(): Deleted.
* wtf/Gigacage.h:
(Gigacage::cagedMayBeNull):
* wtf/PtrTag.h:
(WTF::tagArrayPtr):
(WTF::untagArrayPtr):
(WTF::removeArrayPtrTag):
(WTF::retagArrayPtr):
* wtf/TaggedArrayStoragePtr.h: Copied from Source/JavaScriptCore/runtime/ArrayBufferView.cpp.
(WTF::TaggedArrayStoragePtr::TaggedArrayStoragePtr):
(WTF::TaggedArrayStoragePtr::get const):
(WTF::TaggedArrayStoragePtr::getUnsafe const):
(WTF::TaggedArrayStoragePtr::resize):
(WTF::TaggedArrayStoragePtr::operator bool const):
git-svn-id: http://svn.webkit.org/repository/webkit/trunk@245064 268f45cc-cd09-0410-ab3c-d52691b4dbfc
diff --git a/Source/JavaScriptCore/ChangeLog b/Source/JavaScriptCore/ChangeLog
index d62b6f8..5193c8d 100644
--- a/Source/JavaScriptCore/ChangeLog
+++ b/Source/JavaScriptCore/ChangeLog
@@ -1,3 +1,191 @@
+2019-05-08 Keith Miller <keith_miller@apple.com>
+
+ Remove Gigacage from arm64 and use PAC for arm64e instead
+ https://bugs.webkit.org/show_bug.cgi?id=197110
+
+ Reviewed by Saam Barati.
+
+ This patch makes a bunch of changes. I'll start with global changes then go over changes to each tier and finish with bug fixes.
+
+ Global Changes:
+ Change CagedBarrierPtr to work with PAC so constructors and accessors now expect to receive a length.
+ Update assembler helper methods to use do PAC when caging.
+
+ LLInt:
+ Add arm64e.rb backend as we missed that when originally open sourcing our arm64e code.
+ Add a new optional t6 temporary, which is only used currently on arm64e for GetByVal on a TypedArray.
+ Refactor caging into two helper macros for Primitive/JSValue cages.
+
+ Baseline/DFG:
+ Add authentication where needed for GetByVal and inline object construction.
+
+ FTL:
+ Add a new ValueRep that allows for a late register use. We want this for the authentication patchpoint since we use the length register at the same time as we are defing the authenticated pointer.
+
+ Wasm:
+ Use the TaggedArrayStoragePtr class for the memory base pointer. In theory we should be caging those pointers but I don't want to risk introducing a performance regression with the rest of this change. I've filed https://bugs.webkit.org/show_bug.cgi?id=197620 to do this later.
+ As we no longer have the Gigacage using most of our VA memory, we can enable fast memories on iOS.
+ Using fast memories leads to roughly a 2% JetStream2 speedup.
+
+ * assembler/MacroAssemblerARM64E.h:
+ (JSC::MacroAssemblerARM64E::tagArrayPtr):
+ (JSC::MacroAssemblerARM64E::untagArrayPtr):
+ (JSC::MacroAssemblerARM64E::removeArrayPtrTag):
+ * b3/B3LowerToAir.cpp:
+ * b3/B3PatchpointSpecial.cpp:
+ (JSC::B3::PatchpointSpecial::admitsStack):
+ * b3/B3StackmapSpecial.cpp:
+ (JSC::B3::StackmapSpecial::forEachArgImpl):
+ (JSC::B3::StackmapSpecial::isArgValidForRep):
+ * b3/B3Validate.cpp:
+ * b3/B3ValueRep.cpp:
+ (JSC::B3::ValueRep::addUsedRegistersTo const):
+ (JSC::B3::ValueRep::dump const):
+ (WTF::printInternal):
+ * b3/B3ValueRep.h:
+ (JSC::B3::ValueRep::ValueRep):
+ (JSC::B3::ValueRep::isReg const):
+ * dfg/DFGOperations.cpp:
+ (JSC::DFG::newTypedArrayWithSize):
+ * dfg/DFGSpeculativeJIT.cpp:
+ (JSC::DFG::SpeculativeJIT::jumpForTypedArrayIsNeuteredIfOutOfBounds):
+ (JSC::DFG::SpeculativeJIT::cageTypedArrayStorage):
+ (JSC::DFG::SpeculativeJIT::compileGetIndexedPropertyStorage):
+ (JSC::DFG::SpeculativeJIT::compileGetTypedArrayByteOffset):
+ (JSC::DFG::SpeculativeJIT::compileNewTypedArrayWithSize):
+ * dfg/DFGSpeculativeJIT.h:
+ * dfg/DFGSpeculativeJIT64.cpp:
+ (JSC::DFG::SpeculativeJIT::compile):
+ * ftl/FTLLowerDFGToB3.cpp:
+ (JSC::FTL::DFG::LowerDFGToB3::compileGetIndexedPropertyStorage):
+ (JSC::FTL::DFG::LowerDFGToB3::compileGetTypedArrayByteOffset):
+ (JSC::FTL::DFG::LowerDFGToB3::compileNewTypedArray):
+ (JSC::FTL::DFG::LowerDFGToB3::compileDataViewGet):
+ (JSC::FTL::DFG::LowerDFGToB3::compileDataViewSet):
+ (JSC::FTL::DFG::LowerDFGToB3::untagArrayPtr):
+ (JSC::FTL::DFG::LowerDFGToB3::caged):
+ (JSC::FTL::DFG::LowerDFGToB3::speculateTypedArrayIsNotNeutered):
+ * jit/AssemblyHelpers.h:
+ (JSC::AssemblyHelpers::cageConditionally):
+ * jit/IntrinsicEmitter.cpp:
+ (JSC::IntrinsicGetterAccessCase::emitIntrinsicGetter):
+ * jit/JITPropertyAccess.cpp:
+ (JSC::JIT::emitDirectArgumentsGetByVal):
+ (JSC::JIT::emitIntTypedArrayGetByVal):
+ (JSC::JIT::emitFloatTypedArrayGetByVal):
+ (JSC::JIT::emitIntTypedArrayPutByVal):
+ (JSC::JIT::emitFloatTypedArrayPutByVal):
+ * jit/PolymorphicCallStubRoutine.cpp:
+ (JSC::PolymorphicCallNode::clearCallLinkInfo):
+ * llint/LowLevelInterpreter64.asm:
+ * offlineasm/arm64.rb:
+ * offlineasm/arm64e.rb: Added.
+ * offlineasm/ast.rb:
+ * offlineasm/instructions.rb:
+ * offlineasm/registers.rb:
+ * offlineasm/x86.rb:
+ * runtime/ArrayBuffer.cpp:
+ (JSC::SharedArrayBufferContents::SharedArrayBufferContents):
+ (JSC::SharedArrayBufferContents::~SharedArrayBufferContents):
+ (JSC::ArrayBufferContents::ArrayBufferContents):
+ (JSC::ArrayBufferContents::destroy):
+ (JSC::ArrayBufferContents::tryAllocate):
+ (JSC::ArrayBufferContents::makeShared):
+ (JSC::ArrayBufferContents::copyTo):
+ * runtime/ArrayBuffer.h:
+ (JSC::SharedArrayBufferContents::data const):
+ (JSC::ArrayBufferContents::data const):
+ (JSC::ArrayBuffer::data):
+ (JSC::ArrayBuffer::data const):
+ (JSC::ArrayBuffer::byteLength const):
+ * runtime/ArrayBufferView.cpp:
+ (JSC::ArrayBufferView::ArrayBufferView):
+ * runtime/ArrayBufferView.h:
+ (JSC::ArrayBufferView::baseAddress const):
+ (JSC::ArrayBufferView::byteLength const):
+ (JSC::ArrayBufferView::setRangeImpl):
+ (JSC::ArrayBufferView::getRangeImpl):
+ * runtime/CachedTypes.cpp:
+ (JSC::CachedScopedArgumentsTable::encode):
+ (JSC::CachedScopedArgumentsTable::decode const):
+ * runtime/CagedBarrierPtr.h:
+ (JSC::CagedBarrierPtr::CagedBarrierPtr):
+ (JSC::CagedBarrierPtr::set):
+ (JSC::CagedBarrierPtr::get const):
+ (JSC::CagedBarrierPtr::getMayBeNull const):
+ (JSC::CagedBarrierPtr::getUnsafe const):
+ (JSC::CagedBarrierPtr::at const):
+ (JSC::CagedBarrierPtr::operator== const):
+ (JSC::CagedBarrierPtr::operator bool const):
+ (JSC::CagedBarrierPtr::setWithoutBarrier):
+ (JSC::CagedBarrierPtr::operator* const): Deleted.
+ (JSC::CagedBarrierPtr::operator-> const): Deleted.
+ (JSC::CagedBarrierPtr::operator[] const): Deleted.
+ (): Deleted.
+ * runtime/DataView.cpp:
+ (JSC::DataView::DataView):
+ * runtime/DataView.h:
+ (JSC::DataView::get):
+ (JSC::DataView::set):
+ * runtime/DirectArguments.cpp:
+ (JSC::DirectArguments::visitChildren):
+ (JSC::DirectArguments::overrideThings):
+ (JSC::DirectArguments::unmapArgument):
+ * runtime/DirectArguments.h:
+ * runtime/GenericArguments.h:
+ * runtime/GenericArgumentsInlines.h:
+ (JSC::GenericArguments<Type>::visitChildren):
+ (JSC::GenericArguments<Type>::initModifiedArgumentsDescriptor):
+ (JSC::GenericArguments<Type>::setModifiedArgumentDescriptor):
+ (JSC::GenericArguments<Type>::isModifiedArgumentDescriptor):
+ * runtime/GenericTypedArrayView.h:
+ * runtime/GenericTypedArrayViewInlines.h:
+ (JSC::GenericTypedArrayView<Adaptor>::GenericTypedArrayView):
+ * runtime/JSArrayBufferView.cpp:
+ (JSC::JSArrayBufferView::ConstructionContext::ConstructionContext):
+ (JSC::JSArrayBufferView::JSArrayBufferView):
+ (JSC::JSArrayBufferView::finalize):
+ (JSC::JSArrayBufferView::slowDownAndWasteMemory):
+ * runtime/JSArrayBufferView.h:
+ (JSC::JSArrayBufferView::ConstructionContext::vector const):
+ (JSC::JSArrayBufferView::isNeutered):
+ (JSC::JSArrayBufferView::hasVector const):
+ (JSC::JSArrayBufferView::vector const):
+ * runtime/JSGenericTypedArrayViewInlines.h:
+ (JSC::JSGenericTypedArrayView<Adaptor>::createUninitialized):
+ (JSC::JSGenericTypedArrayView<Adaptor>::estimatedSize):
+ (JSC::JSGenericTypedArrayView<Adaptor>::visitChildren):
+ * runtime/Options.h:
+ * runtime/ScopedArgumentsTable.cpp:
+ (JSC::ScopedArgumentsTable::clone):
+ (JSC::ScopedArgumentsTable::setLength):
+ * runtime/ScopedArgumentsTable.h:
+ * runtime/SymbolTable.h:
+ * wasm/WasmAirIRGenerator.cpp:
+ (JSC::Wasm::AirIRGenerator::restoreWebAssemblyGlobalState):
+ (JSC::Wasm::AirIRGenerator::addCallIndirect):
+ * wasm/WasmB3IRGenerator.cpp:
+ (JSC::Wasm::B3IRGenerator::restoreWebAssemblyGlobalState):
+ (JSC::Wasm::B3IRGenerator::addCallIndirect):
+ * wasm/WasmBBQPlan.cpp:
+ (JSC::Wasm::BBQPlan::complete):
+ * wasm/WasmBinding.cpp:
+ (JSC::Wasm::wasmToWasm):
+ * wasm/WasmInstance.h:
+ (JSC::Wasm::Instance::cachedMemory const):
+ (JSC::Wasm::Instance::updateCachedMemory):
+ * wasm/WasmMemory.cpp:
+ (JSC::Wasm::Memory::Memory):
+ (JSC::Wasm::Memory::~Memory):
+ (JSC::Wasm::Memory::grow):
+ (JSC::Wasm::Memory::dump const):
+ * wasm/WasmMemory.h:
+ (JSC::Wasm::Memory::memory const):
+ * wasm/js/JSToWasm.cpp:
+ (JSC::Wasm::createJSToWasmWrapper):
+ * wasm/js/WebAssemblyFunction.cpp:
+ (JSC::WebAssemblyFunction::jsCallEntrypointSlow):
+
2019-05-08 Caio Lima <ticaiolima@gmail.com>
[BigInt] Add ValueMod into DFG
diff --git a/Source/JavaScriptCore/assembler/MacroAssemblerARM64E.h b/Source/JavaScriptCore/assembler/MacroAssemblerARM64E.h
index 41940cd..8c03d6a 100644
--- a/Source/JavaScriptCore/assembler/MacroAssemblerARM64E.h
+++ b/Source/JavaScriptCore/assembler/MacroAssemblerARM64E.h
@@ -82,6 +82,28 @@
m_assembler.xpaci(target);
}
+ ALWAYS_INLINE void tagArrayPtr(RegisterID length, RegisterID target)
+ {
+ m_assembler.pacdb(target, length);
+ }
+
+ ALWAYS_INLINE void untagArrayPtr(RegisterID length, RegisterID target)
+ {
+ m_assembler.autdb(target, length);
+ }
+
+ ALWAYS_INLINE void untagArrayPtr(Address length, RegisterID target)
+ {
+ auto lengthGPR = getCachedDataTempRegisterIDAndInvalidate();
+ load32(length, lengthGPR);
+ m_assembler.autdb(target, lengthGPR);
+ }
+
+ ALWAYS_INLINE void removeArrayPtrTag(RegisterID target)
+ {
+ m_assembler.xpacd(target);
+ }
+
static const RegisterID InvalidGPR = static_cast<RegisterID>(-1);
enum class CallSignatureType {
diff --git a/Source/JavaScriptCore/b3/B3LowerToAir.cpp b/Source/JavaScriptCore/b3/B3LowerToAir.cpp
index 1b3a92e5..b4098bc 100644
--- a/Source/JavaScriptCore/b3/B3LowerToAir.cpp
+++ b/Source/JavaScriptCore/b3/B3LowerToAir.cpp
@@ -1274,6 +1274,7 @@
arg = tmp(value.value());
break;
case ValueRep::SomeRegister:
+ case ValueRep::SomeLateRegister:
arg = tmp(value.value());
break;
case ValueRep::SomeRegisterWithClobber: {
diff --git a/Source/JavaScriptCore/b3/B3PatchpointSpecial.cpp b/Source/JavaScriptCore/b3/B3PatchpointSpecial.cpp
index daae243..1532edf 100644
--- a/Source/JavaScriptCore/b3/B3PatchpointSpecial.cpp
+++ b/Source/JavaScriptCore/b3/B3PatchpointSpecial.cpp
@@ -120,6 +120,7 @@
case ValueRep::SomeRegister:
case ValueRep::SomeRegisterWithClobber:
case ValueRep::SomeEarlyRegister:
+ case ValueRep::SomeLateRegister:
case ValueRep::Register:
case ValueRep::LateRegister:
return false;
diff --git a/Source/JavaScriptCore/b3/B3StackmapSpecial.cpp b/Source/JavaScriptCore/b3/B3StackmapSpecial.cpp
index 2f07111..298c94c 100644
--- a/Source/JavaScriptCore/b3/B3StackmapSpecial.cpp
+++ b/Source/JavaScriptCore/b3/B3StackmapSpecial.cpp
@@ -113,6 +113,7 @@
case ValueRep::SomeRegisterWithClobber:
role = Arg::UseDef;
break;
+ case ValueRep::SomeLateRegister:
case ValueRep::LateRegister:
role = Arg::LateUse;
break;
@@ -254,6 +255,7 @@
case ValueRep::SomeRegister:
case ValueRep::SomeRegisterWithClobber:
case ValueRep::SomeEarlyRegister:
+ case ValueRep::SomeLateRegister:
return arg.isTmp();
case ValueRep::LateRegister:
case ValueRep::Register:
diff --git a/Source/JavaScriptCore/b3/B3Validate.cpp b/Source/JavaScriptCore/b3/B3Validate.cpp
index f5cd7d2..f7b3c55 100644
--- a/Source/JavaScriptCore/b3/B3Validate.cpp
+++ b/Source/JavaScriptCore/b3/B3Validate.cpp
@@ -580,6 +580,7 @@
break;
case ValueRep::Register:
case ValueRep::LateRegister:
+ case ValueRep::SomeLateRegister:
if (value.rep().kind() == ValueRep::LateRegister)
VALIDATE(role == ConstraintRole::Use, ("At ", *context, ": ", value));
if (value.rep().reg().isGPR())
diff --git a/Source/JavaScriptCore/b3/B3ValueRep.cpp b/Source/JavaScriptCore/b3/B3ValueRep.cpp
index 45a1113..d4b47f1 100644
--- a/Source/JavaScriptCore/b3/B3ValueRep.cpp
+++ b/Source/JavaScriptCore/b3/B3ValueRep.cpp
@@ -42,6 +42,7 @@
case SomeRegister:
case SomeRegisterWithClobber:
case SomeEarlyRegister:
+ case SomeLateRegister:
case Constant:
return;
case LateRegister:
@@ -74,6 +75,7 @@
case SomeRegister:
case SomeRegisterWithClobber:
case SomeEarlyRegister:
+ case SomeLateRegister:
return;
case LateRegister:
case Register:
@@ -183,6 +185,9 @@
case ValueRep::SomeEarlyRegister:
out.print("SomeEarlyRegister");
return;
+ case ValueRep::SomeLateRegister:
+ out.print("SomeLateRegister");
+ return;
case ValueRep::Register:
out.print("Register");
return;
diff --git a/Source/JavaScriptCore/b3/B3ValueRep.h b/Source/JavaScriptCore/b3/B3ValueRep.h
index 463f27e..0ff39d9 100644
--- a/Source/JavaScriptCore/b3/B3ValueRep.h
+++ b/Source/JavaScriptCore/b3/B3ValueRep.h
@@ -74,7 +74,11 @@
// that the def happens before any of the effects of the stackmap. This is only valid for
// the result constraint of a Patchpoint.
SomeEarlyRegister,
-
+
+ // As an input representation, this tells us that B3 should pick some register, but implies
+ // the use happens after any defs. This is only works for patchpoints.
+ SomeLateRegister,
+
// As an input representation, this forces a particular register. As an output
// representation, this tells us what register B3 picked.
Register,
@@ -111,7 +115,7 @@
ValueRep(Kind kind)
: m_kind(kind)
{
- ASSERT(kind == WarmAny || kind == ColdAny || kind == LateColdAny || kind == SomeRegister || kind == SomeRegisterWithClobber || kind == SomeEarlyRegister);
+ ASSERT(kind == WarmAny || kind == ColdAny || kind == LateColdAny || kind == SomeRegister || kind == SomeRegisterWithClobber || kind == SomeEarlyRegister || kind == SomeLateRegister);
}
static ValueRep reg(Reg reg)
@@ -185,7 +189,7 @@
bool isAny() const { return kind() == WarmAny || kind() == ColdAny || kind() == LateColdAny; }
- bool isReg() const { return kind() == Register || kind() == LateRegister; }
+ bool isReg() const { return kind() == Register || kind() == LateRegister || kind() == SomeLateRegister; }
Reg reg() const
{
diff --git a/Source/JavaScriptCore/dfg/DFGOperations.cpp b/Source/JavaScriptCore/dfg/DFGOperations.cpp
index 316d846..2ea4eb3 100644
--- a/Source/JavaScriptCore/dfg/DFGOperations.cpp
+++ b/Source/JavaScriptCore/dfg/DFGOperations.cpp
@@ -198,7 +198,7 @@
}
if (vector)
- return bitwise_cast<char*>(ViewClass::createWithFastVector(exec, structure, size, vector));
+ return bitwise_cast<char*>(ViewClass::createWithFastVector(exec, structure, size, untagArrayPtr(vector, size)));
RELEASE_AND_RETURN(scope, bitwise_cast<char*>(ViewClass::create(exec, structure, size)));
}
diff --git a/Source/JavaScriptCore/dfg/DFGSpeculativeJIT.cpp b/Source/JavaScriptCore/dfg/DFGSpeculativeJIT.cpp
index 389e510..f50a8ad 100644
--- a/Source/JavaScriptCore/dfg/DFGSpeculativeJIT.cpp
+++ b/Source/JavaScriptCore/dfg/DFGSpeculativeJIT.cpp
@@ -40,6 +40,7 @@
#include "DFGSlowPathGenerator.h"
#include "DFGSnippetParams.h"
#include "DirectArguments.h"
+#include "DisallowMacroScratchRegisterUsage.h"
#include "JITAddGenerator.h"
#include "JITBitAndGenerator.h"
#include "JITBitOrGenerator.h"
@@ -2871,9 +2872,21 @@
MacroAssembler::Address(base, JSArrayBufferView::offsetOfMode()),
TrustedImm32(WastefulTypedArray));
- JITCompiler::Jump hasNullVector = m_jit.branchTestPtr(
+ JITCompiler::Jump hasNullVector;
+#if !GIGACAGE_ENABLED && CPU(ARM64E)
+ {
+ GPRReg scratch = m_jit.scratchRegister();
+ DisallowMacroScratchRegisterUsage disallowScratch(m_jit);
+
+ m_jit.loadPtr(MacroAssembler::Address(base, JSArrayBufferView::offsetOfVector()), scratch);
+ m_jit.removeArrayPtrTag(scratch);
+ hasNullVector = m_jit.branchTestPtr(MacroAssembler::Zero, scratch);
+ }
+#else // !GIGACAGE_ENABLED && CPU(ARM64E)
+ hasNullVector = m_jit.branchTestPtr(
MacroAssembler::Zero,
MacroAssembler::Address(base, JSArrayBufferView::offsetOfVector()));
+#endif
speculationCheck(Uncountable, JSValueSource(), node, hasNullVector);
notWasteful.link(&m_jit);
}
@@ -6708,9 +6721,10 @@
storageResult(storageGPR, node);
}
-void SpeculativeJIT::cageTypedArrayStorage(GPRReg storageReg)
+void SpeculativeJIT::cageTypedArrayStorage(GPRReg baseReg, GPRReg storageReg)
{
#if GIGACAGE_ENABLED
+ UNUSED_PARAM(baseReg);
if (!Gigacage::shouldBeEnabled())
return;
@@ -6722,7 +6736,10 @@
}
m_jit.cage(Gigacage::Primitive, storageReg);
+#elif CPU(ARM64E)
+ m_jit.untagArrayPtr(MacroAssembler::Address(baseReg, JSArrayBufferView::offsetOfLength()), storageReg);
#else
+ UNUSED_PARAM(baseReg);
UNUSED_PARAM(storageReg);
#endif
}
@@ -6746,16 +6763,17 @@
m_jit.loadPtr(MacroAssembler::Address(storageReg, StringImpl::dataOffset()), storageReg);
break;
-
- default:
+
+ default: {
auto typedArrayType = node->arrayMode().typedArrayType();
ASSERT_UNUSED(typedArrayType, isTypedView(typedArrayType));
m_jit.loadPtr(JITCompiler::Address(baseReg, JSArrayBufferView::offsetOfVector()), storageReg);
- cageTypedArrayStorage(storageReg);
+ cageTypedArrayStorage(baseReg, storageReg);
break;
}
-
+ }
+
storageResult(storageReg, node);
}
@@ -6785,7 +6803,7 @@
m_jit.loadPtr(MacroAssembler::Address(baseGPR, JSObject::butterflyOffset()), dataGPR);
m_jit.cage(Gigacage::JSValue, dataGPR);
- cageTypedArrayStorage(vectorGPR);
+ cageTypedArrayStorage(baseGPR, vectorGPR);
m_jit.loadPtr(MacroAssembler::Address(dataGPR, Butterfly::offsetOfArrayBuffer()), arrayBufferGPR);
// FIXME: This needs caging.
@@ -9783,6 +9801,11 @@
MacroAssembler::BaseIndex(storageGPR, scratchGPR, MacroAssembler::TimesFour));
m_jit.branchTest32(MacroAssembler::NonZero, scratchGPR).linkTo(loop, &m_jit);
done.link(&m_jit);
+#if !GIGACAGE_ENABLED && CPU(ARM64E)
+ // sizeGPR is still boxed as a number and there is no 32-bit variant of the PAC instructions.
+ m_jit.zeroExtend32ToPtr(sizeGPR, scratchGPR);
+ m_jit.tagArrayPtr(scratchGPR, storageGPR);
+#endif
auto butterfly = TrustedImmPtr(nullptr);
emitAllocateJSObject<JSArrayBufferView>(
@@ -9804,7 +9827,7 @@
addSlowPathGenerator(slowPathCall(
slowCases, this, operationNewTypedArrayWithSizeForType(typedArrayType),
resultGPR, structure, sizeGPR, storageGPR));
-
+
cellResult(resultGPR, node);
}
diff --git a/Source/JavaScriptCore/dfg/DFGSpeculativeJIT.h b/Source/JavaScriptCore/dfg/DFGSpeculativeJIT.h
index e5c5c56..d9ce5ba 100644
--- a/Source/JavaScriptCore/dfg/DFGSpeculativeJIT.h
+++ b/Source/JavaScriptCore/dfg/DFGSpeculativeJIT.h
@@ -1651,7 +1651,7 @@
template<bool strict>
GPRReg fillSpeculateInt32Internal(Edge, DataFormat& returnFormat);
- void cageTypedArrayStorage(GPRReg);
+ void cageTypedArrayStorage(GPRReg, GPRReg);
void recordSetLocal(
VirtualRegister bytecodeReg, VirtualRegister machineReg, DataFormat format)
diff --git a/Source/JavaScriptCore/dfg/DFGSpeculativeJIT64.cpp b/Source/JavaScriptCore/dfg/DFGSpeculativeJIT64.cpp
index f569724..e9f467f 100644
--- a/Source/JavaScriptCore/dfg/DFGSpeculativeJIT64.cpp
+++ b/Source/JavaScriptCore/dfg/DFGSpeculativeJIT64.cpp
@@ -4678,7 +4678,7 @@
m_jit.branch64(MacroAssembler::AboveOrEqual, t2, t1));
m_jit.loadPtr(JITCompiler::Address(dataViewGPR, JSArrayBufferView::offsetOfVector()), t2);
- cageTypedArrayStorage(t2);
+ cageTypedArrayStorage(dataViewGPR, t2);
m_jit.zeroExtend32ToPtr(indexGPR, t1);
auto baseIndex = JITCompiler::BaseIndex(t2, t1, MacroAssembler::TimesOne);
@@ -4874,7 +4874,7 @@
m_jit.branch64(MacroAssembler::AboveOrEqual, t2, t1));
m_jit.loadPtr(JITCompiler::Address(dataViewGPR, JSArrayBufferView::offsetOfVector()), t2);
- cageTypedArrayStorage(t2);
+ cageTypedArrayStorage(dataViewGPR, t2);
m_jit.zeroExtend32ToPtr(indexGPR, t1);
auto baseIndex = JITCompiler::BaseIndex(t2, t1, MacroAssembler::TimesOne);
diff --git a/Source/JavaScriptCore/ftl/FTLLowerDFGToB3.cpp b/Source/JavaScriptCore/ftl/FTLLowerDFGToB3.cpp
index be6905c..5600781 100644
--- a/Source/JavaScriptCore/ftl/FTLLowerDFGToB3.cpp
+++ b/Source/JavaScriptCore/ftl/FTLLowerDFGToB3.cpp
@@ -3868,7 +3868,7 @@
DFG_ASSERT(m_graph, m_node, isTypedView(m_node->arrayMode().typedArrayType()), m_node->arrayMode().typedArrayType());
LValue vector = m_out.loadPtr(cell, m_heaps.JSArrayBufferView_vector);
- setStorage(caged(Gigacage::Primitive, vector));
+ setStorage(caged(Gigacage::Primitive, vector, cell));
}
void compileCheckArray()
@@ -3912,10 +3912,10 @@
m_out.appendTo(notNull, continuation);
- LValue butterflyPtr = caged(Gigacage::JSValue, m_out.loadPtr(basePtr, m_heaps.JSObject_butterfly));
+ LValue butterflyPtr = caged(Gigacage::JSValue, m_out.loadPtr(basePtr, m_heaps.JSObject_butterfly), basePtr);
LValue arrayBufferPtr = m_out.loadPtr(butterflyPtr, m_heaps.Butterfly_arrayBuffer);
- LValue vectorPtr = caged(Gigacage::Primitive, vector);
+ LValue vectorPtr = caged(Gigacage::Primitive, vector, basePtr);
// FIXME: This needs caging.
// https://bugs.webkit.org/show_bug.cgi?id=175515
@@ -6467,6 +6467,17 @@
m_out.int64Zero,
m_heaps.typedArrayProperties);
+#if !GIGACAGE_ENABLED && CPU(ARM64E)
+ PatchpointValue* authenticate = m_out.patchpoint(pointerType());
+ authenticate->appendSomeRegister(storage);
+ authenticate->append(size, B3::ValueRep(B3::ValueRep::SomeLateRegister));
+ authenticate->setGenerator([=] (CCallHelpers& jit, const StackmapGenerationParams& params) {
+ jit.move(params[1].gpr(), params[0].gpr());
+ jit.tagArrayPtr(params[2].gpr(), params[0].gpr());
+ });
+ storage = authenticate;
+#endif
+
ValueFromBlock haveStorage = m_out.anchor(storage);
LValue fastResultValue =
@@ -12685,7 +12696,7 @@
indexToCheck = m_out.add(indexToCheck, m_out.constInt64(data.byteSize - 1));
speculate(OutOfBounds, noValue(), nullptr, m_out.aboveOrEqual(indexToCheck, length));
- LValue vector = caged(Gigacage::Primitive, m_out.loadPtr(dataView, m_heaps.JSArrayBufferView_vector));
+ LValue vector = caged(Gigacage::Primitive, m_out.loadPtr(dataView, m_heaps.JSArrayBufferView_vector), dataView);
TypedPointer pointer(m_heaps.typedArrayProperties, m_out.add(vector, m_out.zeroExtPtr(index)));
@@ -12844,7 +12855,7 @@
RELEASE_ASSERT_NOT_REACHED();
}
- LValue vector = caged(Gigacage::Primitive, m_out.loadPtr(dataView, m_heaps.JSArrayBufferView_vector));
+ LValue vector = caged(Gigacage::Primitive, m_out.loadPtr(dataView, m_heaps.JSArrayBufferView_vector), dataView);
TypedPointer pointer(m_heaps.typedArrayProperties, m_out.add(vector, m_out.zeroExtPtr(index)));
if (data.isFloatingPoint) {
@@ -14091,10 +14102,29 @@
m_out.appendTo(performStore, lastNext);
}
}
-
- LValue caged(Gigacage::Kind kind, LValue ptr)
+
+ LValue untagArrayPtr(LValue ptr, LValue size)
+ {
+
+#if !GIGACAGE_ENABLED && CPU(ARM64E)
+ PatchpointValue* authenticate = m_out.patchpoint(pointerType());
+ authenticate->appendSomeRegister(ptr);
+ authenticate->append(size, B3::ValueRep(B3::ValueRep::SomeLateRegister));
+ authenticate->setGenerator([=] (CCallHelpers& jit, const StackmapGenerationParams& params) {
+ jit.move(params[1].gpr(), params[0].gpr());
+ jit.untagArrayPtr(params[2].gpr(), params[0].gpr());
+ });
+ return authenticate;
+#else
+ UNUSED_PARAM(size);
+ return ptr;
+#endif
+ }
+
+ LValue caged(Gigacage::Kind kind, LValue ptr, LValue base)
{
#if GIGACAGE_ENABLED
+ UNUSED_PARAM(base);
if (!Gigacage::isEnabled(kind))
return ptr;
@@ -14123,8 +14153,16 @@
// and possibly other smart things if we want to be able to remove this opaque.
// https://bugs.webkit.org/show_bug.cgi?id=175493
return m_out.opaque(result);
+#elif CPU(ARM64E)
+ if (kind == Gigacage::Primitive) {
+ LValue size = m_out.load32(base, m_heaps.JSArrayBufferView_length);
+ return untagArrayPtr(ptr, size);
+ }
+
+ return ptr;
#else
UNUSED_PARAM(kind);
+ UNUSED_PARAM(base);
return ptr;
#endif
}
@@ -16538,6 +16576,16 @@
LBasicBlock lastNext = m_out.appendTo(isWasteful, continuation);
LValue vector = m_out.loadPtr(base, m_heaps.JSArrayBufferView_vector);
+#if !GIGACAGE_ENABLED && CPU(ARM64E)
+ // FIXME: We could probably make this a mask. https://bugs.webkit.org/show_bug.cgi?id=197701
+ PatchpointValue* authenticate = m_out.patchpoint(pointerType());
+ authenticate->appendSomeRegister(vector);
+ authenticate->setGenerator([=] (CCallHelpers& jit, const StackmapGenerationParams& params) {
+ jit.move(params[1].gpr(), params[0].gpr());
+ jit.removeArrayPtrTag(params[0].gpr());
+ });
+ vector = authenticate;
+#endif
speculate(Uncountable, jsValueValue(vector), m_node, m_out.isZero64(vector));
m_out.jump(continuation);
diff --git a/Source/JavaScriptCore/jit/AssemblyHelpers.h b/Source/JavaScriptCore/jit/AssemblyHelpers.h
index 7602b2a..d9268e8 100644
--- a/Source/JavaScriptCore/jit/AssemblyHelpers.h
+++ b/Source/JavaScriptCore/jit/AssemblyHelpers.h
@@ -1569,7 +1569,7 @@
#endif
}
- void cageConditionally(Gigacage::Kind kind, GPRReg storage, GPRReg scratch)
+ void cageConditionally(Gigacage::Kind kind, GPRReg storage, GPRReg scratchOrLength)
{
#if GIGACAGE_ENABLED
if (!Gigacage::isEnabled(kind))
@@ -1578,15 +1578,18 @@
if (kind != Gigacage::Primitive || Gigacage::isDisablingPrimitiveGigacageDisabled())
return cage(kind, storage);
- loadPtr(&Gigacage::basePtr(kind), scratch);
- Jump done = branchTestPtr(Zero, scratch);
+ loadPtr(&Gigacage::basePtr(kind), scratchOrLength);
+ Jump done = branchTestPtr(Zero, scratchOrLength);
andPtr(TrustedImmPtr(Gigacage::mask(kind)), storage);
- addPtr(scratch, storage);
+ addPtr(scratchOrLength, storage);
done.link(this);
+#elif CPU(ARM64E)
+ if (kind == Gigacage::Primitive)
+ untagArrayPtr(scratchOrLength, storage);
#else
UNUSED_PARAM(kind);
UNUSED_PARAM(storage);
- UNUSED_PARAM(scratch);
+ UNUSED_PARAM(scratchOrLength);
#endif
}
diff --git a/Source/JavaScriptCore/jit/IntrinsicEmitter.cpp b/Source/JavaScriptCore/jit/IntrinsicEmitter.cpp
index cae39e9..06c6127 100644
--- a/Source/JavaScriptCore/jit/IntrinsicEmitter.cpp
+++ b/Source/JavaScriptCore/jit/IntrinsicEmitter.cpp
@@ -114,6 +114,9 @@
jit.loadPtr(MacroAssembler::Address(baseGPR, JSObject::butterflyOffset()), scratchGPR);
jit.loadPtr(MacroAssembler::Address(baseGPR, JSArrayBufferView::offsetOfVector()), valueGPR);
+#if !GIGACAGE_ENABLED && CPU(ARM64E)
+ jit.removeArrayPtrTag(valueGPR);
+#endif
jit.loadPtr(MacroAssembler::Address(scratchGPR, Butterfly::offsetOfArrayBuffer()), scratchGPR);
jit.loadPtr(MacroAssembler::Address(scratchGPR, ArrayBuffer::offsetOfData()), scratchGPR);
jit.subPtr(scratchGPR, valueGPR);
diff --git a/Source/JavaScriptCore/jit/JITPropertyAccess.cpp b/Source/JavaScriptCore/jit/JITPropertyAccess.cpp
index 0b34400..519ad7a 100644
--- a/Source/JavaScriptCore/jit/JITPropertyAccess.cpp
+++ b/Source/JavaScriptCore/jit/JITPropertyAccess.cpp
@@ -1589,7 +1589,7 @@
load32(Address(base, DirectArguments::offsetOfLength()), scratch2);
slowCases.append(branch32(AboveOrEqual, property, scratch2));
slowCases.append(branchTestPtr(NonZero, Address(base, DirectArguments::offsetOfMappedArguments())));
-
+
loadValue(BaseIndex(base, property, TimesEight, DirectArguments::storageOffset()), result);
return slowCases;
@@ -1669,7 +1669,8 @@
load8(Address(base, JSCell::typeInfoTypeOffset()), scratch);
badType = patchableBranch32(NotEqual, scratch, TrustedImm32(typeForTypedArrayType(type)));
- slowCases.append(branch32(AboveOrEqual, property, Address(base, JSArrayBufferView::offsetOfLength())));
+ load32(Address(base, JSArrayBufferView::offsetOfLength()), scratch2);
+ slowCases.append(branch32(AboveOrEqual, property, scratch2));
loadPtr(Address(base, JSArrayBufferView::offsetOfVector()), scratch);
cageConditionally(Gigacage::Primitive, scratch, scratch2);
@@ -1732,7 +1733,8 @@
load8(Address(base, JSCell::typeInfoTypeOffset()), scratch);
badType = patchableBranch32(NotEqual, scratch, TrustedImm32(typeForTypedArrayType(type)));
- slowCases.append(branch32(AboveOrEqual, property, Address(base, JSArrayBufferView::offsetOfLength())));
+ load32(Address(base, JSArrayBufferView::offsetOfLength()), scratch2);
+ slowCases.append(branch32(AboveOrEqual, property, scratch2));
loadPtr(Address(base, JSArrayBufferView::offsetOfVector()), scratch);
cageConditionally(Gigacage::Primitive, scratch, scratch2);
@@ -1782,7 +1784,8 @@
load8(Address(base, JSCell::typeInfoTypeOffset()), earlyScratch);
badType = patchableBranch32(NotEqual, earlyScratch, TrustedImm32(typeForTypedArrayType(type)));
- Jump inBounds = branch32(Below, property, Address(base, JSArrayBufferView::offsetOfLength()));
+ load32(Address(base, JSArrayBufferView::offsetOfLength()), lateScratch2);
+ Jump inBounds = branch32(Below, property, lateScratch2);
emitArrayProfileOutOfBoundsSpecialCase(profile);
slowCases.append(jump());
inBounds.link(this);
@@ -1857,7 +1860,8 @@
load8(Address(base, JSCell::typeInfoTypeOffset()), earlyScratch);
badType = patchableBranch32(NotEqual, earlyScratch, TrustedImm32(typeForTypedArrayType(type)));
- Jump inBounds = branch32(Below, property, Address(base, JSArrayBufferView::offsetOfLength()));
+ load32(Address(base, JSArrayBufferView::offsetOfLength()), lateScratch2);
+ Jump inBounds = branch32(Below, property, lateScratch2);
emitArrayProfileOutOfBoundsSpecialCase(profile);
slowCases.append(jump());
inBounds.link(this);
diff --git a/Source/JavaScriptCore/jit/PolymorphicCallStubRoutine.cpp b/Source/JavaScriptCore/jit/PolymorphicCallStubRoutine.cpp
index 7cf74b1..91bb585 100644
--- a/Source/JavaScriptCore/jit/PolymorphicCallStubRoutine.cpp
+++ b/Source/JavaScriptCore/jit/PolymorphicCallStubRoutine.cpp
@@ -57,9 +57,6 @@
void PolymorphicCallNode::clearCallLinkInfo()
{
- if (Options::dumpDisassembly())
- dataLog("Clearing call link info for polymorphic call at ", m_callLinkInfo->callReturnLocation(), ", ", m_callLinkInfo->codeOrigin(), "\n");
-
m_callLinkInfo = nullptr;
}
diff --git a/Source/JavaScriptCore/llint/LowLevelInterpreter64.asm b/Source/JavaScriptCore/llint/LowLevelInterpreter64.asm
index 3a13c67..c807435 100644
--- a/Source/JavaScriptCore/llint/LowLevelInterpreter64.asm
+++ b/Source/JavaScriptCore/llint/LowLevelInterpreter64.asm
@@ -408,19 +408,28 @@
end)
end
-macro uncage(basePtr, mask, ptr, scratch)
+macro uncage(basePtr, mask, ptr, scratchOrLength)
if GIGACAGE_ENABLED and not C_LOOP
- loadp basePtr, scratch
- btpz scratch, .done
+ loadp basePtr, scratchOrLength
+ btpz scratchOrLength, .done
andp mask, ptr
- addp scratch, ptr
+ addp scratchOrLength, ptr
.done:
end
end
-macro loadCaged(basePtr, mask, source, dest, scratch)
+macro loadCagedPrimitive(source, dest, scratchOrLength)
loadp source, dest
- uncage(basePtr, mask, dest, scratch)
+ if GIGACAGE_ENABLED
+ uncage(_g_gigacageBasePtrs + Gigacage::BasePtrs::primitive, constexpr Gigacage::primitiveGigacageMask, dest, scratchOrLength)
+ elsif ARM64E
+ untagArrayPtr scratchOrLength, dest
+ end
+end
+
+macro loadCagedJSValue(source, dest, scratchOrLength)
+ loadp source, dest
+ uncage(_g_gigacageBasePtrs + Gigacage::BasePtrs::jsValue, constexpr Gigacage::jsValueGigacageMask, dest, scratchOrLength)
end
macro loadVariable(get, fieldName, valueReg)
@@ -1310,7 +1319,7 @@
arrayProfile(OpGetById::Metadata::m_modeMetadata.arrayLengthMode.arrayProfile, t0, t2, t5)
btiz t0, IsArray, .opGetByIdSlow
btiz t0, IndexingShapeMask, .opGetByIdSlow
- loadCaged(_g_gigacageBasePtrs + Gigacage::BasePtrs::jsValue, constexpr Gigacage::jsValueGigacageMask, JSObject::m_butterfly[t3], t0, t1)
+ loadCagedJSValue(JSObject::m_butterfly[t3], t0, t1)
loadi -sizeof IndexingHeader + IndexingHeader::u.lengths.publicLength[t0], t0
bilt t0, 0, .opGetByIdSlow
orq tagTypeNumber, t0
@@ -1433,7 +1442,7 @@
loadConstantOrVariableInt32(size, t3, t1, .opGetByValSlow)
sxi2q t1, t1
- loadCaged(_g_gigacageBasePtrs + Gigacage::BasePtrs::jsValue, constexpr Gigacage::jsValueGigacageMask, JSObject::m_butterfly[t0], t3, tagTypeNumber)
+ loadCagedJSValue(JSObject::m_butterfly[t0], t3, tagTypeNumber)
move TagTypeNumber, tagTypeNumber
andi IndexingShapeMask, t2
@@ -1477,7 +1486,17 @@
biaeq t2, NumberOfTypedArrayTypesExcludingDataView, .opGetByValSlow
# Sweet, now we know that we have a typed array. Do some basic things now.
- biaeq t1, JSArrayBufferView::m_length[t0], .opGetByValSlow
+
+ if ARM64E
+ const scratchOrLength = t6
+ loadi JSArrayBufferView::m_length[t0], scratchOrLength
+ biaeq t1, scratchOrLength, .opGetByValSlow
+ else
+ const scratchOrLength = t0
+ biaeq t1, JSArrayBufferView::m_length[t0], .opGetByValSlow
+ end
+
+ loadCagedPrimitive(JSArrayBufferView::m_vector[t0], t3, scratchOrLength)
# Now bisect through the various types:
# Int8ArrayType,
@@ -1499,7 +1518,6 @@
bia t2, Int8ArrayType - FirstTypedArrayType, .opGetByValUint8ArrayOrUint8ClampedArray
# We have Int8ArrayType.
- loadCaged(_g_gigacageBasePtrs + Gigacage::BasePtrs::primitive, constexpr Gigacage::primitiveGigacageMask, JSArrayBufferView::m_vector[t0], t3, t2)
loadbs [t3, t1], t0
finishIntGetByVal(t0, t1)
@@ -1507,13 +1525,11 @@
bia t2, Uint8ArrayType - FirstTypedArrayType, .opGetByValUint8ClampedArray
# We have Uint8ArrayType.
- loadCaged(_g_gigacageBasePtrs + Gigacage::BasePtrs::primitive, constexpr Gigacage::primitiveGigacageMask, JSArrayBufferView::m_vector[t0], t3, t2)
loadb [t3, t1], t0
finishIntGetByVal(t0, t1)
.opGetByValUint8ClampedArray:
# We have Uint8ClampedArrayType.
- loadCaged(_g_gigacageBasePtrs + Gigacage::BasePtrs::primitive, constexpr Gigacage::primitiveGigacageMask, JSArrayBufferView::m_vector[t0], t3, t2)
loadb [t3, t1], t0
finishIntGetByVal(t0, t1)
@@ -1522,13 +1538,11 @@
bia t2, Int16ArrayType - FirstTypedArrayType, .opGetByValUint16Array
# We have Int16ArrayType.
- loadCaged(_g_gigacageBasePtrs + Gigacage::BasePtrs::primitive, constexpr Gigacage::primitiveGigacageMask, JSArrayBufferView::m_vector[t0], t3, t2)
loadhs [t3, t1, 2], t0
finishIntGetByVal(t0, t1)
.opGetByValUint16Array:
# We have Uint16ArrayType.
- loadCaged(_g_gigacageBasePtrs + Gigacage::BasePtrs::primitive, constexpr Gigacage::primitiveGigacageMask, JSArrayBufferView::m_vector[t0], t3, t2)
loadh [t3, t1, 2], t0
finishIntGetByVal(t0, t1)
@@ -1540,13 +1554,11 @@
bia t2, Int32ArrayType - FirstTypedArrayType, .opGetByValUint32Array
# We have Int32ArrayType.
- loadCaged(_g_gigacageBasePtrs + Gigacage::BasePtrs::primitive, constexpr Gigacage::primitiveGigacageMask, JSArrayBufferView::m_vector[t0], t3, t2)
loadi [t3, t1, 4], t0
finishIntGetByVal(t0, t1)
.opGetByValUint32Array:
# We have Uint32ArrayType.
- loadCaged(_g_gigacageBasePtrs + Gigacage::BasePtrs::primitive, constexpr Gigacage::primitiveGigacageMask, JSArrayBufferView::m_vector[t0], t3, t2)
# This is the hardest part because of large unsigned values.
loadi [t3, t1, 4], t0
bilt t0, 0, .opGetByValSlow # This case is still awkward to implement in LLInt.
@@ -1558,7 +1570,6 @@
bieq t2, Float32ArrayType - FirstTypedArrayType, .opGetByValSlow
# We have Float64ArrayType.
- loadCaged(_g_gigacageBasePtrs + Gigacage::BasePtrs::primitive, constexpr Gigacage::primitiveGigacageMask, JSArrayBufferView::m_vector[t0], t3, t2)
loadd [t3, t1, 8], ft0
bdnequn ft0, ft0, .opGetByValSlow
finishDoubleGetByVal(ft0, t0, t1)
@@ -1594,7 +1605,7 @@
get(m_property, t0)
loadConstantOrVariableInt32(size, t0, t3, .opPutByValSlow)
sxi2q t3, t3
- loadCaged(_g_gigacageBasePtrs + Gigacage::BasePtrs::jsValue, constexpr Gigacage::jsValueGigacageMask, JSObject::m_butterfly[t1], t0, tagTypeNumber)
+ loadCagedJSValue(JSObject::m_butterfly[t1], t0, tagTypeNumber)
move TagTypeNumber, tagTypeNumber
btinz t2, CopyOnWrite, .opPutByValSlow
andi IndexingShapeMask, t2
diff --git a/Source/JavaScriptCore/offlineasm/arm64.rb b/Source/JavaScriptCore/offlineasm/arm64.rb
index c09e157..9c0cbdc 100644
--- a/Source/JavaScriptCore/offlineasm/arm64.rb
+++ b/Source/JavaScriptCore/offlineasm/arm64.rb
@@ -1,4 +1,4 @@
-# Copyright (C) 2011-2018 Apple Inc. All rights reserved.
+# Copyright (C) 2011-2019 Apple Inc. All rights reserved.
# Copyright (C) 2014 University of Szeged. All rights reserved.
#
# Redistribution and use in source and binary forms, with or without
@@ -123,7 +123,9 @@
when 't4'
arm64GPRName('x4', kind)
when 't5'
- arm64GPRName('x5', kind)
+ arm64GPRName('x5', kind)
+ when 't6'
+ arm64GPRName('x6', kind)
when 'cfr'
arm64GPRName('x29', kind)
when 'csr0'
@@ -361,8 +363,7 @@
end
class Sequence
- def getModifiedListARM64
- result = @list
+ def getModifiedListARM64(result = @list)
result = riscLowerNot(result)
result = riscLowerSimpleBranchOps(result)
@@ -387,7 +388,7 @@
"jmp", "call", "leap", "leaq"
size = $currentSettings["ADDRESS64"] ? 8 : 4
else
- raise "Bad instruction #{node.opcode} for heap access at #{node.codeOriginString}"
+ raise "Bad instruction #{node.opcode} for heap access at #{node.codeOriginString}: #{node.dump}"
end
if address.is_a? BaseIndex
diff --git a/Source/JavaScriptCore/offlineasm/arm64e.rb b/Source/JavaScriptCore/offlineasm/arm64e.rb
new file mode 100644
index 0000000..359bec7
--- /dev/null
+++ b/Source/JavaScriptCore/offlineasm/arm64e.rb
@@ -0,0 +1,117 @@
+# Copyright (C) 2018-2019 Apple Inc. All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions
+# are met:
+# 1. Redistributions of source code must retain the above copyright
+# notice, this list of conditions and the following disclaimer.
+# 2. Redistributions in binary form must reproduce the above copyright
+# notice, this list of conditions and the following disclaimer in the
+# documentation and/or other materials provided with the distribution.
+#
+# THIS SOFTWARE IS PROVIDED BY APPLE INC. ``AS IS'' AND ANY
+# EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+# PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR
+# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
+# EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
+# PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
+# PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
+# OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+class ARM64E
+ # FIXME: This is fragile and needs to match the enum value in PtrTag.h.
+ CFunctionPtrTag = 2
+end
+
+class Sequence
+ def getModifiedListARM64E
+ result = riscLowerMisplacedAddresses(@list)
+ getModifiedListARM64(result)
+ end
+end
+
+class Instruction
+ def self.lowerMisplacedAddressesARM64E(node, newList)
+ wasHandled = false
+ if node.is_a? Instruction
+ postInstructions = []
+ annotation = node.annotation
+ codeOrigin = node.codeOrigin
+ case node.opcode
+ when "jmp", "call"
+ if node.operands.size > 1
+ if node.operands[1].is_a? RegisterID
+ tag = riscAsRegister(newList, postInstructions, node.operands[1], "p", false)
+ else
+ tag = Tmp.new(codeOrigin, :gpr)
+ newList << Instruction.new(codeOrigin, "move", [node.operands[1], tag], annotation)
+ end
+ operands = [riscAsRegister(newList, postInstructions, node.operands[0], "p", false), tag]
+ newList << Instruction.new(codeOrigin, node.opcode, operands, annotation)
+ wasHandled = true
+ end
+ when "untagArrayPtr"
+ newOperands = node.operands.map {
+ | operand |
+ if operand.address?
+ tmp = Tmp.new(codeOrigin, :gpr)
+ newList << Instruction.new(codeOrigin, "loadp", [operand, tmp], annotation)
+ tmp
+ else
+ operand
+ end
+ }
+ newList << Instruction.new(codeOrigin, node.opcode, newOperands, annotation)
+ wasHandled = true
+ end
+ newList += postInstructions if wasHandled
+ end
+ return wasHandled, newList
+ end
+
+ def lowerARM64E
+ case opcode
+ when "call"
+ if operands.size == 1 or operands[0].label?
+ lowerARM64
+ elsif operands[1] == ARM64E::CFunctionPtrTag
+ emitARM64Unflipped("blraaz", [operands[0]], :ptr)
+ else
+ emitARM64Unflipped("blrab", operands, :ptr)
+ end
+ when "jmp"
+ if operands[0].label?
+ lowerARM64
+ else
+ emitARM64Unflipped("brab", operands, :ptr)
+ end
+ when "tagReturnAddress"
+ raise if operands.size < 1 or not operands[0].is_a? RegisterID
+ if operands[0].is_a? RegisterID and operands[0].name == "sp"
+ $asm.puts "pacibsp"
+ else
+ emitARM64Unflipped("pacib lr,", operands, :ptr)
+ end
+ when "untagReturnAddress"
+ raise if operands.size < 1 or not operands[0].is_a? RegisterID
+ if operands[0].is_a? RegisterID and operands[0].name == "sp"
+ $asm.puts "autibsp"
+ else
+ emitARM64Unflipped("autib lr,", operands, :ptr)
+ end
+ when "removeCodePtrTag"
+ raise unless operands[0].is_a? RegisterID
+ emitARM64Unflipped("xpaci ", operands, :ptr)
+ when "untagArrayPtr"
+ raise if operands.size != 2 or not operands.each { |operand| operand.is_a? RegisterID or operand.is_a? Tmp }
+ emitARM64("autdb ", operands, :ptr)
+ when "ret"
+ $asm.puts "retab"
+ else
+ lowerARM64
+ end
+ end
+end
diff --git a/Source/JavaScriptCore/offlineasm/ast.rb b/Source/JavaScriptCore/offlineasm/ast.rb
index 586ba5c..2edca72 100644
--- a/Source/JavaScriptCore/offlineasm/ast.rb
+++ b/Source/JavaScriptCore/offlineasm/ast.rb
@@ -938,7 +938,7 @@
$asm.putGlobalAnnotation
when "emit"
$asm.puts "#{operands[0].dump}"
- when "tagReturnAddress", "untagReturnAddress", "removeCodePtrTag"
+ when "tagReturnAddress", "untagReturnAddress", "removeCodePtrTag", "untagArrayPtr"
else
raise "Unhandled opcode #{opcode} at #{codeOriginString}"
end
diff --git a/Source/JavaScriptCore/offlineasm/instructions.rb b/Source/JavaScriptCore/offlineasm/instructions.rb
index c658b9b..69e4b6aa 100644
--- a/Source/JavaScriptCore/offlineasm/instructions.rb
+++ b/Source/JavaScriptCore/offlineasm/instructions.rb
@@ -253,7 +253,8 @@
"memfence",
"tagReturnAddress",
"untagReturnAddress",
- "removeCodePtrTag"
+ "removeCodePtrTag",
+ "untagArrayPtr",
]
X86_INSTRUCTIONS =
diff --git a/Source/JavaScriptCore/offlineasm/registers.rb b/Source/JavaScriptCore/offlineasm/registers.rb
index b6ed36d..aa8a40f 100644
--- a/Source/JavaScriptCore/offlineasm/registers.rb
+++ b/Source/JavaScriptCore/offlineasm/registers.rb
@@ -31,6 +31,7 @@
"t3",
"t4",
"t5",
+ "t6",
"cfr",
"a0",
"a1",
diff --git a/Source/JavaScriptCore/offlineasm/x86.rb b/Source/JavaScriptCore/offlineasm/x86.rb
index 7969964..f2deba8 100644
--- a/Source/JavaScriptCore/offlineasm/x86.rb
+++ b/Source/JavaScriptCore/offlineasm/x86.rb
@@ -49,7 +49,8 @@
# rdx => t2, a2, r1
# rcx => t3, a3
# r8 => t4
-# r10 => t5
+# r9 => t5
+# r10 => t6
# rbx => csr0 (callee-save, PB, unused in baseline)
# r12 => csr1 (callee-save)
# r13 => csr2 (callee-save)
diff --git a/Source/JavaScriptCore/runtime/ArrayBuffer.cpp b/Source/JavaScriptCore/runtime/ArrayBuffer.cpp
index 8f8556b..b16ec38 100644
--- a/Source/JavaScriptCore/runtime/ArrayBuffer.cpp
+++ b/Source/JavaScriptCore/runtime/ArrayBuffer.cpp
@@ -33,15 +33,17 @@
namespace JSC {
-SharedArrayBufferContents::SharedArrayBufferContents(void* data, ArrayBufferDestructorFunction&& destructor)
- : m_data(data)
+SharedArrayBufferContents::SharedArrayBufferContents(void* data, unsigned size, ArrayBufferDestructorFunction&& destructor)
+ : m_data(data, size)
, m_destructor(WTFMove(destructor))
+ , m_sizeInBytes(size)
{
}
SharedArrayBufferContents::~SharedArrayBufferContents()
{
- m_destructor(m_data.getMayBeNull());
+ // FIXME: we shouldn't use getUnsafe here https://bugs.webkit.org/show_bug.cgi?id=197698
+ m_destructor(m_data.getUnsafe());
}
ArrayBufferContents::ArrayBufferContents()
@@ -56,7 +58,7 @@
}
ArrayBufferContents::ArrayBufferContents(void* data, unsigned sizeInBytes, ArrayBufferDestructorFunction&& destructor)
- : m_data(data)
+ : m_data(data, sizeInBytes)
, m_sizeInBytes(sizeInBytes)
{
RELEASE_ASSERT(m_sizeInBytes <= MAX_ARRAY_BUFFER_SIZE);
@@ -82,7 +84,8 @@
void ArrayBufferContents::destroy()
{
- m_destructor(m_data.getMayBeNull());
+ // FIXME: We shouldn't use getUnsafe here: https://bugs.webkit.org/show_bug.cgi?id=197698
+ m_destructor(m_data.getUnsafe());
}
void ArrayBufferContents::reset()
@@ -106,14 +109,16 @@
size_t size = static_cast<size_t>(numElements) * static_cast<size_t>(elementByteSize);
if (!size)
size = 1; // Make sure malloc actually allocates something, but not too much. We use null to mean that the buffer is neutered.
- m_data = Gigacage::tryMalloc(Gigacage::Primitive, size);
- if (!m_data) {
+
+ void* data = Gigacage::tryMalloc(Gigacage::Primitive, numElements * elementByteSize);
+ m_data = DataType(data, size);
+ if (!data) {
reset();
return;
}
if (policy == ZeroInitialize)
- memset(m_data.get(), 0, size);
+ memset(data, 0, size);
m_sizeInBytes = numElements * elementByteSize;
RELEASE_ASSERT(m_sizeInBytes <= MAX_ARRAY_BUFFER_SIZE);
@@ -122,7 +127,7 @@
void ArrayBufferContents::makeShared()
{
- m_shared = adoptRef(new SharedArrayBufferContents(m_data.getMayBeNull(), WTFMove(m_destructor)));
+ m_shared = adoptRef(new SharedArrayBufferContents(data(), sizeInBytes(), WTFMove(m_destructor)));
m_destructor = [] (void*) { };
}
@@ -143,7 +148,7 @@
other.tryAllocate(m_sizeInBytes, sizeof(char), ArrayBufferContents::DontInitialize);
if (!other.m_data)
return;
- memcpy(other.m_data.get(), m_data.get(), m_sizeInBytes);
+ memcpy(other.data(), data(), m_sizeInBytes);
other.m_sizeInBytes = m_sizeInBytes;
RELEASE_ASSERT(other.m_sizeInBytes <= MAX_ARRAY_BUFFER_SIZE);
}
diff --git a/Source/JavaScriptCore/runtime/ArrayBuffer.h b/Source/JavaScriptCore/runtime/ArrayBuffer.h
index baf2b85..7c05bbc 100644
--- a/Source/JavaScriptCore/runtime/ArrayBuffer.h
+++ b/Source/JavaScriptCore/runtime/ArrayBuffer.h
@@ -48,14 +48,16 @@
class SharedArrayBufferContents : public ThreadSafeRefCounted<SharedArrayBufferContents> {
public:
- SharedArrayBufferContents(void* data, ArrayBufferDestructorFunction&&);
+ SharedArrayBufferContents(void* data, unsigned size, ArrayBufferDestructorFunction&&);
~SharedArrayBufferContents();
- void* data() const { return m_data.getMayBeNull(); }
+ void* data() const { return m_data.getMayBeNull(m_sizeInBytes); }
private:
- CagedPtr<Gigacage::Primitive, void> m_data;
+ using DataType = CagedPtr<Gigacage::Primitive, void, tagCagedPtr>;
+ DataType m_data;
ArrayBufferDestructorFunction m_destructor;
+ unsigned m_sizeInBytes;
};
class ArrayBufferContents {
@@ -73,7 +75,7 @@
explicit operator bool() { return !!m_data; }
- void* data() const { return m_data.getMayBeNull(); }
+ void* data() const { return m_data.getMayBeNull(sizeInBytes()); }
unsigned sizeInBytes() const { return m_sizeInBytes; }
bool isShared() const { return m_shared; }
@@ -98,7 +100,8 @@
ArrayBufferDestructorFunction m_destructor;
RefPtr<SharedArrayBufferContents> m_shared;
- CagedPtr<Gigacage::Primitive, void> m_data;
+ using DataType = CagedPtr<Gigacage::Primitive, void, tagCagedPtr>;
+ DataType m_data;
unsigned m_sizeInBytes;
};
@@ -174,17 +177,17 @@
void* ArrayBuffer::data()
{
- return m_contents.m_data.getMayBeNull();
+ return m_contents.data();
}
const void* ArrayBuffer::data() const
{
- return m_contents.m_data.getMayBeNull();
+ return m_contents.data();
}
unsigned ArrayBuffer::byteLength() const
{
- return m_contents.m_sizeInBytes;
+ return m_contents.sizeInBytes();
}
bool ArrayBuffer::isShared() const
diff --git a/Source/JavaScriptCore/runtime/ArrayBufferView.cpp b/Source/JavaScriptCore/runtime/ArrayBufferView.cpp
index 9fed7c7..b795fe4 100644
--- a/Source/JavaScriptCore/runtime/ArrayBufferView.cpp
+++ b/Source/JavaScriptCore/runtime/ArrayBufferView.cpp
@@ -25,17 +25,22 @@
#include "config.h"
#include "ArrayBufferView.h"
+#include <wtf/CheckedArithmetic.h>
namespace JSC {
ArrayBufferView::ArrayBufferView(
- RefPtr<ArrayBuffer>&& buffer,
- unsigned byteOffset)
+ RefPtr<ArrayBuffer>&& buffer, unsigned byteOffset, unsigned byteLength)
: m_byteOffset(byteOffset)
, m_isNeuterable(true)
+ , m_byteLength(byteLength)
, m_buffer(WTFMove(buffer))
{
- m_baseAddress = m_buffer ? (static_cast<char*>(m_buffer->data()) + m_byteOffset) : 0;
+ Checked<unsigned, CrashOnOverflow> length(byteOffset);
+ length += byteLength;
+ RELEASE_ASSERT_WITH_SECURITY_IMPLICATION(length <= m_buffer->byteLength());
+ if (m_buffer)
+ m_baseAddress = BaseAddress(static_cast<char*>(m_buffer->data()) + m_byteOffset, byteLength);
}
ArrayBufferView::~ArrayBufferView()
diff --git a/Source/JavaScriptCore/runtime/ArrayBufferView.h b/Source/JavaScriptCore/runtime/ArrayBufferView.h
index 754784e..39fe6bb 100644
--- a/Source/JavaScriptCore/runtime/ArrayBufferView.h
+++ b/Source/JavaScriptCore/runtime/ArrayBufferView.h
@@ -71,8 +71,8 @@
void* baseAddress() const
{
if (isNeutered())
- return 0;
- return m_baseAddress.getMayBeNull();
+ return nullptr;
+ return m_baseAddress.getMayBeNull(byteLength());
}
void* data() const { return baseAddress(); }
@@ -84,7 +84,7 @@
return m_byteOffset;
}
- virtual unsigned byteLength() const = 0;
+ unsigned byteLength() const { return m_byteLength; }
JS_EXPORT_PRIVATE void setNeuterable(bool flag);
bool isNeuterable() const { return m_isNeuterable; }
@@ -113,13 +113,12 @@
virtual JSArrayBufferView* wrap(ExecState*, JSGlobalObject*) = 0;
protected:
- JS_EXPORT_PRIVATE ArrayBufferView(RefPtr<ArrayBuffer>&&, unsigned byteOffset);
+ JS_EXPORT_PRIVATE ArrayBufferView(RefPtr<ArrayBuffer>&&, unsigned byteOffset, unsigned byteLength);
inline bool setImpl(ArrayBufferView*, unsigned byteOffset);
- // Caller passes in bufferByteLength to avoid a virtual function call.
- inline bool setRangeImpl(const void* data, size_t dataByteLength, unsigned byteOffset, unsigned bufferByteLength);
- inline bool getRangeImpl(void* destination, size_t dataByteLength, unsigned byteOffset, unsigned bufferByteLength);
+ inline bool setRangeImpl(const void* data, size_t dataByteLength, unsigned byteOffset);
+ inline bool getRangeImpl(void* destination, size_t dataByteLength, unsigned byteOffset);
inline bool zeroRangeImpl(unsigned byteOffset, size_t rangeByteLength);
@@ -150,9 +149,11 @@
unsigned m_byteOffset : 31;
bool m_isNeuterable : 1;
+ unsigned m_byteLength;
+ using BaseAddress = CagedPtr<Gigacage::Primitive, void, tagCagedPtr>;
// This is the address of the ArrayBuffer's storage, plus the byte offset.
- CagedPtr<Gigacage::Primitive, void> m_baseAddress;
+ BaseAddress m_baseAddress;
private:
friend class ArrayBuffer;
@@ -173,12 +174,10 @@
return true;
}
-bool ArrayBufferView::setRangeImpl(const void* data, size_t dataByteLength, unsigned byteOffset, unsigned bufferByteLength)
+bool ArrayBufferView::setRangeImpl(const void* data, size_t dataByteLength, unsigned byteOffset)
{
- // Do not replace with RELEASE_ASSERT; we want to avoid the virtual byteLength() function call in release.
- ASSERT_WITH_SECURITY_IMPLICATION(bufferByteLength == byteLength());
- if (byteOffset > bufferByteLength
- || byteOffset + dataByteLength > bufferByteLength
+ if (byteOffset > byteLength()
+ || byteOffset + dataByteLength > byteLength()
|| byteOffset + dataByteLength < byteOffset) {
// Out of range offset or overflow
return false;
@@ -189,12 +188,10 @@
return true;
}
-bool ArrayBufferView::getRangeImpl(void* destination, size_t dataByteLength, unsigned byteOffset, unsigned bufferByteLength)
+bool ArrayBufferView::getRangeImpl(void* destination, size_t dataByteLength, unsigned byteOffset)
{
- // Do not replace with RELEASE_ASSERT; we want to avoid the virtual byteLength() function call in release.
- ASSERT_WITH_SECURITY_IMPLICATION(bufferByteLength == byteLength());
- if (byteOffset > bufferByteLength
- || byteOffset + dataByteLength > bufferByteLength
+ if (byteOffset > byteLength()
+ || byteOffset + dataByteLength > byteLength()
|| byteOffset + dataByteLength < byteOffset) {
// Out of range offset or overflow
return false;
diff --git a/Source/JavaScriptCore/runtime/CachedTypes.cpp b/Source/JavaScriptCore/runtime/CachedTypes.cpp
index cbbf5db..22f3c2e 100644
--- a/Source/JavaScriptCore/runtime/CachedTypes.cpp
+++ b/Source/JavaScriptCore/runtime/CachedTypes.cpp
@@ -1028,13 +1028,13 @@
void encode(Encoder& encoder, const ScopedArgumentsTable& scopedArgumentsTable)
{
m_length = scopedArgumentsTable.m_length;
- m_arguments.encode(encoder, scopedArgumentsTable.m_arguments.get(), m_length);
+ m_arguments.encode(encoder, scopedArgumentsTable.m_arguments.get(m_length), m_length);
}
ScopedArgumentsTable* decode(Decoder& decoder) const
{
ScopedArgumentsTable* scopedArgumentsTable = ScopedArgumentsTable::create(decoder.vm(), m_length);
- m_arguments.decode(decoder, scopedArgumentsTable->m_arguments.get(), m_length);
+ m_arguments.decode(decoder, scopedArgumentsTable->m_arguments.get(m_length), m_length);
return scopedArgumentsTable;
}
diff --git a/Source/JavaScriptCore/runtime/CagedBarrierPtr.h b/Source/JavaScriptCore/runtime/CagedBarrierPtr.h
index 918ddf2..425a87c 100644
--- a/Source/JavaScriptCore/runtime/CagedBarrierPtr.h
+++ b/Source/JavaScriptCore/runtime/CagedBarrierPtr.h
@@ -35,87 +35,41 @@
// This is a convenient combo of AuxiliaryBarrier and CagedPtr.
-template<Gigacage::Kind passedKind, typename T>
+template<Gigacage::Kind passedKind, typename T, bool shouldTag = false>
class CagedBarrierPtr {
public:
static constexpr Gigacage::Kind kind = passedKind;
- typedef T Type;
+ using Type = T;
+ using CagedType = CagedPtr<kind, Type, shouldTag>;
- CagedBarrierPtr() { }
+ CagedBarrierPtr() = default;
template<typename U>
- CagedBarrierPtr(VM& vm, JSCell* cell, U&& value)
+ CagedBarrierPtr(VM& vm, JSCell* cell, U&& value, unsigned size)
{
- m_barrier.set(vm, cell, std::forward<U>(value));
+ m_barrier.set(vm, cell, CagedType(std::forward<U>(value), size));
}
void clear() { m_barrier.clear(); }
template<typename U>
- void set(VM& vm, JSCell* cell, U&& value)
+ void set(VM& vm, JSCell* cell, U&& value, unsigned size)
{
- m_barrier.set(vm, cell, std::forward<U>(value));
+ m_barrier.set(vm, cell, CagedType(std::forward<U>(value), size));
}
- T* get() const { return m_barrier.get().get(); }
- T* getMayBeNull() const { return m_barrier.get().getMayBeNull(); }
-
- bool operator==(const CagedBarrierPtr& other) const
- {
- return getMayBeNull() == other.getMayBeNull();
- }
-
- bool operator!=(const CagedBarrierPtr& other) const
- {
- return !(*this == other);
- }
-
- explicit operator bool() const
- {
- return *this != CagedBarrierPtr();
- }
-
- template<typename U>
- void setWithoutBarrier(U&& value) { m_barrier.setWithoutBarrier(std::forward<U>(value)); }
-
- T& operator*() const { return *get(); }
- T* operator->() const { return get(); }
-
- template<typename IndexType>
- T& operator[](IndexType index) const { return get()[index]; }
-
-private:
- AuxiliaryBarrier<CagedPtr<kind, T>> m_barrier;
-};
+ T* get(unsigned size) const { return m_barrier.get().get(size); }
+ T* getMayBeNull(unsigned size) const { return m_barrier.get().getMayBeNull(size); }
+ T* getUnsafe() const { return m_barrier.get().getUnsafe(); }
-template<Gigacage::Kind passedKind>
-class CagedBarrierPtr<passedKind, void> {
-public:
- static constexpr Gigacage::Kind kind = passedKind;
- typedef void Type;
-
- CagedBarrierPtr() { }
-
- template<typename U>
- CagedBarrierPtr(VM& vm, JSCell* cell, U&& value)
- {
- m_barrier.set(vm, cell, std::forward<U>(value));
- }
-
- void clear() { m_barrier.clear(); }
-
- template<typename U>
- void set(VM& vm, JSCell* cell, U&& value)
- {
- m_barrier.set(vm, cell, std::forward<U>(value));
- }
-
- void* get() const { return m_barrier.get().get(); }
- void* getMayBeNull() const { return m_barrier.get().getMayBeNull(); }
-
+ // We need the template here so that the type of U is deduced at usage time rather than class time. U should always be T.
+ template<typename U = T>
+ typename std::enable_if<!std::is_same<void, U>::value, T>::type&
+ /* T& */ at(unsigned index, unsigned size) const { return get(size)[index]; }
+
bool operator==(const CagedBarrierPtr& other) const
{
- return getMayBeNull() == other.getMayBeNull();
+ return m_barrier.get() == other.m_barrier.get();
}
bool operator!=(const CagedBarrierPtr& other) const
@@ -125,14 +79,14 @@
explicit operator bool() const
{
- return *this != CagedBarrierPtr();
+ return !!m_barrier.get();
}
template<typename U>
- void setWithoutBarrier(U&& value) { m_barrier.setWithoutBarrier(std::forward<U>(value)); }
+ void setWithoutBarrier(U&& value, unsigned size) { m_barrier.setWithoutBarrier(CagedType(std::forward<U>(value), size)); }
private:
- AuxiliaryBarrier<CagedPtr<kind, void>> m_barrier;
+ AuxiliaryBarrier<CagedType> m_barrier;
};
} // namespace JSC
diff --git a/Source/JavaScriptCore/runtime/DataView.cpp b/Source/JavaScriptCore/runtime/DataView.cpp
index 3b5871b..5e073f5 100644
--- a/Source/JavaScriptCore/runtime/DataView.cpp
+++ b/Source/JavaScriptCore/runtime/DataView.cpp
@@ -33,8 +33,7 @@
namespace JSC {
DataView::DataView(RefPtr<ArrayBuffer>&& buffer, unsigned byteOffset, unsigned byteLength)
- : ArrayBufferView(WTFMove(buffer), byteOffset)
- , m_byteLength(byteLength)
+ : ArrayBufferView(WTFMove(buffer), byteOffset, byteLength)
{
}
diff --git a/Source/JavaScriptCore/runtime/DataView.h b/Source/JavaScriptCore/runtime/DataView.h
index 7a39763..22a23fd 100644
--- a/Source/JavaScriptCore/runtime/DataView.h
+++ b/Source/JavaScriptCore/runtime/DataView.h
@@ -38,11 +38,6 @@
JS_EXPORT_PRIVATE static Ref<DataView> create(RefPtr<ArrayBuffer>&&, unsigned byteOffset, unsigned length);
static Ref<DataView> create(RefPtr<ArrayBuffer>&&);
- unsigned byteLength() const override
- {
- return m_byteLength;
- }
-
TypedArrayType getType() const override
{
return TypeDataView;
@@ -62,7 +57,7 @@
} else
RELEASE_ASSERT(offset + sizeof(T) <= byteLength());
return flipBytesIfLittleEndian(
- *reinterpret_cast<T*>(static_cast<uint8_t*>(m_baseAddress.get()) + offset),
+ *reinterpret_cast<T*>(static_cast<uint8_t*>(m_baseAddress.get(byteLength())) + offset),
littleEndian);
}
@@ -86,12 +81,9 @@
*status = true;
} else
RELEASE_ASSERT(offset + sizeof(T) <= byteLength());
- *reinterpret_cast<T*>(static_cast<uint8_t*>(m_baseAddress.get()) + offset) =
+ *reinterpret_cast<T*>(static_cast<uint8_t*>(m_baseAddress.get(byteLength())) + offset) =
flipBytesIfLittleEndian(value, littleEndian);
}
-
-private:
- unsigned m_byteLength;
};
} // namespace JSC
diff --git a/Source/JavaScriptCore/runtime/DirectArguments.cpp b/Source/JavaScriptCore/runtime/DirectArguments.cpp
index e001be2..3ee876b 100644
--- a/Source/JavaScriptCore/runtime/DirectArguments.cpp
+++ b/Source/JavaScriptCore/runtime/DirectArguments.cpp
@@ -101,7 +101,7 @@
visitor.append(thisObject->m_callee);
if (thisObject->m_mappedArguments)
- visitor.markAuxiliary(thisObject->m_mappedArguments.get());
+ visitor.markAuxiliary(thisObject->m_mappedArguments.get(thisObject->internalLength()));
GenericArguments<DirectArguments>::visitChildren(thisCell, visitor);
}
@@ -120,8 +120,8 @@
void* backingStore = vm.gigacageAuxiliarySpace(m_mappedArguments.kind).allocateNonVirtual(vm, mappedArgumentsSize(), nullptr, AllocationFailureMode::Assert);
bool* overrides = static_cast<bool*>(backingStore);
- m_mappedArguments.set(vm, this, overrides);
- for (unsigned i = m_length; i--;)
+ m_mappedArguments.set(vm, this, overrides, internalLength());
+ for (unsigned i = internalLength(); i--;)
overrides[i] = false;
}
@@ -134,7 +134,7 @@
void DirectArguments::unmapArgument(VM& vm, unsigned index)
{
overrideThingsIfNecessary(vm);
- m_mappedArguments[index] = true;
+ m_mappedArguments.at(index, internalLength()) = true;
}
void DirectArguments::copyToArguments(ExecState* exec, VirtualRegister firstElementDest, unsigned offset, unsigned length)
diff --git a/Source/JavaScriptCore/runtime/DirectArguments.h b/Source/JavaScriptCore/runtime/DirectArguments.h
index 32cdd0b..9ca01f3 100644
--- a/Source/JavaScriptCore/runtime/DirectArguments.h
+++ b/Source/JavaScriptCore/runtime/DirectArguments.h
@@ -86,7 +86,7 @@
bool isMappedArgument(uint32_t i) const
{
- return i < m_length && (!m_mappedArguments || !m_mappedArguments[i]);
+ return i < m_length && (!m_mappedArguments || !m_mappedArguments.at(i, m_length));
}
bool isMappedArgumentInDFG(uint32_t i) const
@@ -182,7 +182,8 @@
WriteBarrier<JSFunction> m_callee;
uint32_t m_length; // Always the actual length of captured arguments and never what was stored into the length property.
uint32_t m_minCapacity; // The max of this and length determines the capacity of this object. It may be the actual capacity, or maybe something smaller. We arrange it this way to be kind to the JITs.
- CagedBarrierPtr<Gigacage::Primitive, bool> m_mappedArguments; // If non-null, it means that length, callee, and caller are fully materialized properties.
+ using MappedArguments = CagedBarrierPtr<Gigacage::Primitive, bool>;
+ MappedArguments m_mappedArguments; // If non-null, it means that length, callee, and caller are fully materialized properties.
};
} // namespace JSC
diff --git a/Source/JavaScriptCore/runtime/GenericArguments.h b/Source/JavaScriptCore/runtime/GenericArguments.h
index 6a6380e..b1c7868 100644
--- a/Source/JavaScriptCore/runtime/GenericArguments.h
+++ b/Source/JavaScriptCore/runtime/GenericArguments.h
@@ -60,8 +60,9 @@
bool isModifiedArgumentDescriptor(unsigned index, unsigned length);
void copyToArguments(ExecState*, VirtualRegister firstElementDest, unsigned offset, unsigned length);
-
- CagedBarrierPtr<Gigacage::Primitive, bool> m_modifiedArgumentsDescriptor;
+
+ using ModifiedArgumentsPtr = CagedBarrierPtr<Gigacage::Primitive, bool>;
+ ModifiedArgumentsPtr m_modifiedArgumentsDescriptor;
};
} // namespace JSC
diff --git a/Source/JavaScriptCore/runtime/GenericArgumentsInlines.h b/Source/JavaScriptCore/runtime/GenericArgumentsInlines.h
index 2be41fe..d56ebf5 100644
--- a/Source/JavaScriptCore/runtime/GenericArgumentsInlines.h
+++ b/Source/JavaScriptCore/runtime/GenericArgumentsInlines.h
@@ -38,7 +38,7 @@
Base::visitChildren(thisCell, visitor);
if (thisObject->m_modifiedArgumentsDescriptor)
- visitor.markAuxiliary(thisObject->m_modifiedArgumentsDescriptor.get());
+ visitor.markAuxiliary(thisObject->m_modifiedArgumentsDescriptor.getUnsafe());
}
template<typename Type>
@@ -265,7 +265,7 @@
if (argsLength) {
void* backingStore = vm.gigacageAuxiliarySpace(m_modifiedArgumentsDescriptor.kind).allocateNonVirtual(vm, WTF::roundUpToMultipleOf<8>(argsLength), nullptr, AllocationFailureMode::Assert);
bool* modifiedArguments = static_cast<bool*>(backingStore);
- m_modifiedArgumentsDescriptor.set(vm, this, modifiedArguments);
+ m_modifiedArgumentsDescriptor.set(vm, this, modifiedArguments, argsLength);
for (unsigned i = argsLength; i--;)
modifiedArguments[i] = false;
}
@@ -283,7 +283,7 @@
{
initModifiedArgumentsDescriptorIfNecessary(vm, length);
if (index < length)
- m_modifiedArgumentsDescriptor[index] = true;
+ m_modifiedArgumentsDescriptor.at(index, length) = true;
}
template<typename Type>
@@ -292,7 +292,7 @@
if (!m_modifiedArgumentsDescriptor)
return false;
if (index < length)
- return m_modifiedArgumentsDescriptor[index];
+ return m_modifiedArgumentsDescriptor.at(index, length);
return false;
}
diff --git a/Source/JavaScriptCore/runtime/GenericTypedArrayView.h b/Source/JavaScriptCore/runtime/GenericTypedArrayView.h
index ef2e80f..be2d420 100644
--- a/Source/JavaScriptCore/runtime/GenericTypedArrayView.h
+++ b/Source/JavaScriptCore/runtime/GenericTypedArrayView.h
@@ -58,8 +58,7 @@
return setRangeImpl(
reinterpret_cast<const char*>(data),
count * sizeof(typename Adaptor::Type),
- offset * sizeof(typename Adaptor::Type),
- internalByteLength());
+ offset * sizeof(typename Adaptor::Type));
}
bool zeroRange(unsigned offset, size_t count)
@@ -73,12 +72,7 @@
{
if (isNeutered())
return 0;
- return m_length;
- }
-
- unsigned byteLength() const override
- {
- return internalByteLength();
+ return byteLength() / sizeof(typename Adaptor::Type);
}
typename Adaptor::Type item(unsigned index) const
@@ -105,7 +99,7 @@
reinterpret_cast<char*>(data),
count * sizeof(typename Adaptor::Type),
offset * sizeof(typename Adaptor::Type),
- internalByteLength());
+ byteLength());
}
bool checkInboundData(unsigned offset, size_t count) const
@@ -126,14 +120,6 @@
}
JSArrayBufferView* wrap(ExecState*, JSGlobalObject*) override;
-
-private:
- unsigned internalByteLength() const
- {
- return length() * sizeof(typename Adaptor::Type);
- }
-
- unsigned m_length;
};
} // namespace JSC
diff --git a/Source/JavaScriptCore/runtime/GenericTypedArrayViewInlines.h b/Source/JavaScriptCore/runtime/GenericTypedArrayViewInlines.h
index e660cfc..746cbe2 100644
--- a/Source/JavaScriptCore/runtime/GenericTypedArrayViewInlines.h
+++ b/Source/JavaScriptCore/runtime/GenericTypedArrayViewInlines.h
@@ -32,9 +32,8 @@
template<typename Adaptor>
GenericTypedArrayView<Adaptor>::GenericTypedArrayView(
- RefPtr<ArrayBuffer>&& buffer, unsigned byteOffset, unsigned length)
- : ArrayBufferView(WTFMove(buffer), byteOffset)
- , m_length(length)
+RefPtr<ArrayBuffer>&& buffer, unsigned byteOffset, unsigned length)
+ : ArrayBufferView(WTFMove(buffer), byteOffset, length * sizeof(typename Adaptor::Type))
{
}
diff --git a/Source/JavaScriptCore/runtime/JSArrayBufferView.cpp b/Source/JavaScriptCore/runtime/JSArrayBufferView.cpp
index c2de003..d1d8679 100644
--- a/Source/JavaScriptCore/runtime/JSArrayBufferView.cpp
+++ b/Source/JavaScriptCore/runtime/JSArrayBufferView.cpp
@@ -1,5 +1,5 @@
/*
- * Copyright (C) 2013-2018 Apple Inc. All rights reserved.
+ * Copyright (C) 2013-2019 Apple Inc. All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
@@ -50,11 +50,12 @@
JSArrayBufferView::ConstructionContext::ConstructionContext(
Structure* structure, uint32_t length, void* vector)
: m_structure(structure)
- , m_vector(vector)
+ , m_vector(vector, length)
, m_length(length)
, m_mode(FastTypedArray)
, m_butterfly(nullptr)
{
+ ASSERT(vector == removeArrayPtrTag(vector));
RELEASE_ASSERT(length <= fastSizeLimit);
}
@@ -74,11 +75,11 @@
return;
m_structure = structure;
- m_vector = temp;
+ m_vector = VectorType(temp, length);
m_mode = FastTypedArray;
if (mode == ZeroFill) {
- uint64_t* asWords = static_cast<uint64_t*>(m_vector.getMayBeNull());
+ uint64_t* asWords = static_cast<uint64_t*>(vector());
for (unsigned i = size / sizeof(uint64_t); i--;)
asWords[i] = 0;
}
@@ -91,11 +92,11 @@
return;
size_t size = static_cast<size_t>(length) * static_cast<size_t>(elementSize);
- m_vector = Gigacage::tryMalloc(Gigacage::Primitive, size);
+ m_vector = VectorType(Gigacage::tryMalloc(Gigacage::Primitive, size), length);
if (!m_vector)
return;
if (mode == ZeroFill)
- memset(m_vector.get(), 0, size);
+ memset(vector(), 0, size);
vm.heap.reportExtraMemoryAllocated(static_cast<size_t>(length) * elementSize);
@@ -110,7 +111,8 @@
, m_length(length)
, m_mode(WastefulTypedArray)
{
- m_vector = static_cast<uint8_t*>(arrayBuffer->data()) + byteOffset;
+ ASSERT(arrayBuffer->data() == removeArrayPtrTag(arrayBuffer->data()));
+ m_vector = VectorType(static_cast<uint8_t*>(arrayBuffer->data()) + byteOffset, length);
IndexingHeader indexingHeader;
indexingHeader.setArrayBuffer(arrayBuffer.get());
m_butterfly = Butterfly::create(vm, 0, 0, 0, true, indexingHeader, 0);
@@ -124,7 +126,8 @@
, m_mode(DataViewMode)
, m_butterfly(0)
{
- m_vector = static_cast<uint8_t*>(arrayBuffer->data()) + byteOffset;
+ ASSERT(arrayBuffer->data() == removeArrayPtrTag(arrayBuffer->data()));
+ m_vector = VectorType(static_cast<uint8_t*>(arrayBuffer->data()) + byteOffset, length);
}
JSArrayBufferView::JSArrayBufferView(VM& vm, ConstructionContext& context)
@@ -133,7 +136,8 @@
, m_mode(context.mode())
{
setButterfly(vm, context.butterfly());
- m_vector.setWithoutBarrier(context.vector());
+ ASSERT(context.vector() == removeArrayPtrTag(context.vector()));
+ m_vector.setWithoutBarrier(context.vector(), m_length);
}
void JSArrayBufferView::finishCreation(VM& vm)
@@ -194,7 +198,7 @@
JSArrayBufferView* thisObject = static_cast<JSArrayBufferView*>(cell);
ASSERT(thisObject->m_mode == OversizeTypedArray || thisObject->m_mode == WastefulTypedArray);
if (thisObject->m_mode == OversizeTypedArray)
- Gigacage::free(Gigacage::Primitive, thisObject->m_vector.get());
+ Gigacage::free(Gigacage::Primitive, thisObject->vector());
}
JSArrayBuffer* JSArrayBufferView::unsharedJSBuffer(ExecState* exec)
@@ -283,7 +287,7 @@
{
auto locker = holdLock(cellLock());
butterfly()->indexingHeader()->setArrayBuffer(buffer.get());
- m_vector.setWithoutBarrier(buffer->data());
+ m_vector.setWithoutBarrier(buffer->data(), m_length);
WTF::storeStoreFence();
m_mode = WastefulTypedArray;
}
diff --git a/Source/JavaScriptCore/runtime/JSArrayBufferView.h b/Source/JavaScriptCore/runtime/JSArrayBufferView.h
index ef7ebe1..35ce36c 100644
--- a/Source/JavaScriptCore/runtime/JSArrayBufferView.h
+++ b/Source/JavaScriptCore/runtime/JSArrayBufferView.h
@@ -27,6 +27,7 @@
#include "AuxiliaryBarrier.h"
#include "JSObject.h"
+#include <wtf/TaggedArrayStoragePtr.h>
namespace JSC {
@@ -96,6 +97,7 @@
public:
typedef JSNonFinalObject Base;
static const unsigned fastSizeLimit = 1000;
+ using VectorPtr = CagedBarrierPtr<Gigacage::Primitive, void, tagCagedPtr>;
static size_t sizeOf(uint32_t length, uint32_t elementSize)
{
@@ -133,14 +135,15 @@
bool operator!() const { return !m_structure; }
Structure* structure() const { return m_structure; }
- void* vector() const { return m_vector.getMayBeNull(); }
+ void* vector() const { return m_vector.get(m_length); }
uint32_t length() const { return m_length; }
TypedArrayMode mode() const { return m_mode; }
Butterfly* butterfly() const { return m_butterfly; }
private:
Structure* m_structure;
- CagedPtr<Gigacage::Primitive, void> m_vector;
+ using VectorType = CagedPtr<Gigacage::Primitive, void, tagCagedPtr>;
+ VectorType m_vector;
uint32_t m_length;
TypedArrayMode m_mode;
Butterfly* m_butterfly;
@@ -164,10 +167,11 @@
JSArrayBuffer* possiblySharedJSBuffer(ExecState* exec);
RefPtr<ArrayBufferView> unsharedImpl();
JS_EXPORT_PRIVATE RefPtr<ArrayBufferView> possiblySharedImpl();
- bool isNeutered() { return hasArrayBuffer() && !vector(); }
+ bool isNeutered() { return hasArrayBuffer() && !hasVector(); }
void neuter();
-
- void* vector() const { return m_vector.getMayBeNull(); }
+
+ bool hasVector() const { return !!m_vector; }
+ void* vector() const { return m_vector.getMayBeNull(length()); }
unsigned byteOffset();
unsigned length() const { return m_length; }
@@ -191,7 +195,7 @@
static String toStringName(const JSObject*, ExecState*);
- CagedBarrierPtr<Gigacage::Primitive, void> m_vector;
+ VectorPtr m_vector;
uint32_t m_length;
TypedArrayMode m_mode;
};
diff --git a/Source/JavaScriptCore/runtime/JSGenericTypedArrayViewInlines.h b/Source/JavaScriptCore/runtime/JSGenericTypedArrayViewInlines.h
index a27fe78..d42af64 100644
--- a/Source/JavaScriptCore/runtime/JSGenericTypedArrayViewInlines.h
+++ b/Source/JavaScriptCore/runtime/JSGenericTypedArrayViewInlines.h
@@ -77,8 +77,7 @@
}
template<typename Adaptor>
-JSGenericTypedArrayView<Adaptor>* JSGenericTypedArrayView<Adaptor>::createUninitialized(
- ExecState* exec, Structure* structure, unsigned length)
+JSGenericTypedArrayView<Adaptor>* JSGenericTypedArrayView<Adaptor>::createUninitialized(ExecState* exec, Structure* structure, unsigned length)
{
VM& vm = exec->vm();
auto scope = DECLARE_THROW_SCOPE(vm);
@@ -507,7 +506,7 @@
if (thisObject->m_mode == OversizeTypedArray)
return Base::estimatedSize(thisObject, vm) + thisObject->byteSize();
- if (thisObject->m_mode == FastTypedArray && thisObject->m_vector)
+ if (thisObject->m_mode == FastTypedArray && thisObject->hasVector())
return Base::estimatedSize(thisObject, vm) + thisObject->byteSize();
return Base::estimatedSize(thisObject, vm);
@@ -526,7 +525,7 @@
{
auto locker = holdLock(thisObject->cellLock());
mode = thisObject->m_mode;
- vector = thisObject->m_vector.getMayBeNull();
+ vector = thisObject->vector();
byteSize = thisObject->byteSize();
}
diff --git a/Source/JavaScriptCore/runtime/Options.h b/Source/JavaScriptCore/runtime/Options.h
index aa37c00..72e4413 100644
--- a/Source/JavaScriptCore/runtime/Options.h
+++ b/Source/JavaScriptCore/runtime/Options.h
@@ -491,8 +491,7 @@
v(unsigned, webAssemblyLoopDecrement, 15, Normal, "The amount the tier up countdown is decremented on each loop backedge.") \
v(unsigned, webAssemblyFunctionEntryDecrement, 1, Normal, "The amount the tier up countdown is decremented on each function entry.") \
\
- /* FIXME: enable fast memories on iOS and pre-allocate them. https://bugs.webkit.org/show_bug.cgi?id=170774 */ \
- v(bool, useWebAssemblyFastMemory, !isIOS(), Normal, "If true, we will try to use a 32-bit address space with a signal handler to bounds check wasm memory.") \
+ v(bool, useWebAssemblyFastMemory, true, Normal, "If true, we will try to use a 32-bit address space with a signal handler to bounds check wasm memory.") \
v(bool, logWebAssemblyMemory, false, Normal, nullptr) \
v(unsigned, webAssemblyFastMemoryRedzonePages, 128, Normal, "WebAssembly fast memories use 4GiB virtual allocations, plus a redzone (counted as multiple of 64KiB WebAssembly pages) at the end to catch reg+imm accesses which exceed 32-bit, anything beyond the redzone is explicitly bounds-checked") \
v(bool, crashIfWebAssemblyCantFastMemory, false, Normal, "If true, we will crash if we can't obtain fast memory for wasm.") \
diff --git a/Source/JavaScriptCore/runtime/ScopedArgumentsTable.cpp b/Source/JavaScriptCore/runtime/ScopedArgumentsTable.cpp
index 955a307..5b0e9af 100644
--- a/Source/JavaScriptCore/runtime/ScopedArgumentsTable.cpp
+++ b/Source/JavaScriptCore/runtime/ScopedArgumentsTable.cpp
@@ -68,16 +68,16 @@
{
ScopedArgumentsTable* result = create(vm, m_length);
for (unsigned i = m_length; i--;)
- result->m_arguments[i] = m_arguments[i];
+ result->at(i) = this->at(i);
return result;
}
ScopedArgumentsTable* ScopedArgumentsTable::setLength(VM& vm, uint32_t newLength)
{
if (LIKELY(!m_locked)) {
- ArgumentsPtr newArguments = ArgumentsPtr::create(newLength);
+ ArgumentsPtr newArguments = ArgumentsPtr::create(newLength, newLength);
for (unsigned i = std::min(m_length, newLength); i--;)
- newArguments[i] = m_arguments[i];
+ newArguments.at(i, newLength) = this->at(i);
m_length = newLength;
m_arguments = WTFMove(newArguments);
return this;
@@ -85,10 +85,12 @@
ScopedArgumentsTable* result = create(vm, newLength);
for (unsigned i = std::min(m_length, newLength); i--;)
- result->m_arguments[i] = m_arguments[i];
+ result->at(i) = this->at(i);
return result;
}
+static_assert(std::is_trivially_destructible<ScopeOffset>::value, "");
+
ScopedArgumentsTable* ScopedArgumentsTable::set(VM& vm, uint32_t i, ScopeOffset value)
{
ScopedArgumentsTable* result;
diff --git a/Source/JavaScriptCore/runtime/ScopedArgumentsTable.h b/Source/JavaScriptCore/runtime/ScopedArgumentsTable.h
index e812342..558a9eb 100644
--- a/Source/JavaScriptCore/runtime/ScopedArgumentsTable.h
+++ b/Source/JavaScriptCore/runtime/ScopedArgumentsTable.h
@@ -61,10 +61,7 @@
uint32_t length() const { return m_length; }
ScopedArgumentsTable* setLength(VM&, uint32_t newLength);
- ScopeOffset get(uint32_t i) const
- {
- return const_cast<ScopedArgumentsTable*>(this)->at(i);
- }
+ ScopeOffset get(uint32_t i) const { return at(i); }
void lock()
{
@@ -80,13 +77,13 @@
static ptrdiff_t offsetOfLength() { return OBJECT_OFFSETOF(ScopedArgumentsTable, m_length); }
static ptrdiff_t offsetOfArguments() { return OBJECT_OFFSETOF(ScopedArgumentsTable, m_arguments); }
- typedef CagedUniquePtr<Gigacage::Primitive, ScopeOffset[]> ArgumentsPtr;
+ typedef CagedUniquePtr<Gigacage::Primitive, ScopeOffset> ArgumentsPtr;
private:
- ScopeOffset& at(uint32_t i)
+ ScopeOffset& at(uint32_t i) const
{
ASSERT_WITH_SECURITY_IMPLICATION(i < m_length);
- return m_arguments[i];
+ return m_arguments.get(length())[i];
}
uint32_t m_length;
diff --git a/Source/JavaScriptCore/runtime/SymbolTable.h b/Source/JavaScriptCore/runtime/SymbolTable.h
index 8e80d4b..3fd1c46 100644
--- a/Source/JavaScriptCore/runtime/SymbolTable.h
+++ b/Source/JavaScriptCore/runtime/SymbolTable.h
@@ -631,8 +631,9 @@
void setArgumentsLength(VM& vm, uint32_t length)
{
if (UNLIKELY(!m_arguments))
- m_arguments.set(vm, this, ScopedArgumentsTable::create(vm));
- m_arguments.set(vm, this, m_arguments->setLength(vm, length));
+ m_arguments.set(vm, this, ScopedArgumentsTable::create(vm, length));
+ else
+ m_arguments.set(vm, this, m_arguments->setLength(vm, length));
}
ScopeOffset argumentOffset(uint32_t i) const
diff --git a/Source/JavaScriptCore/wasm/WasmAirIRGenerator.cpp b/Source/JavaScriptCore/wasm/WasmAirIRGenerator.cpp
index 07e219c..26a400d 100644
--- a/Source/JavaScriptCore/wasm/WasmAirIRGenerator.cpp
+++ b/Source/JavaScriptCore/wasm/WasmAirIRGenerator.cpp
@@ -833,6 +833,9 @@
patchpoint->setGenerator([pinnedRegs] (CCallHelpers& jit, const B3::StackmapGenerationParams& params) {
jit.loadPtr(CCallHelpers::Address(params[0].gpr(), Instance::offsetOfCachedMemorySize()), pinnedRegs->sizeRegister);
jit.loadPtr(CCallHelpers::Address(params[0].gpr(), Instance::offsetOfCachedMemory()), pinnedRegs->baseMemoryPointer);
+#if CPU(ARM64E)
+ jit.untagArrayPtr(pinnedRegs->sizeRegister, pinnedRegs->baseMemoryPointer);
+#endif
});
emitPatchpoint(block, patchpoint, Tmp(), instance);
@@ -1856,6 +1859,9 @@
ASSERT(pinnedRegs.sizeRegister != newContextInstance);
jit.loadPtr(CCallHelpers::Address(newContextInstance, Instance::offsetOfCachedMemorySize()), pinnedRegs.sizeRegister); // Memory size.
jit.loadPtr(CCallHelpers::Address(newContextInstance, Instance::offsetOfCachedMemory()), baseMemory); // Memory::void*.
+#if CPU(ARM64E)
+ jit.untagArrayPtr(pinnedRegs.sizeRegister, baseMemory);
+#endif
});
emitPatchpoint(doContextSwitch, patchpoint, Tmp(), newContextInstance, instanceValue());
diff --git a/Source/JavaScriptCore/wasm/WasmB3IRGenerator.cpp b/Source/JavaScriptCore/wasm/WasmB3IRGenerator.cpp
index 7a7bdc3..aee8b4d 100644
--- a/Source/JavaScriptCore/wasm/WasmB3IRGenerator.cpp
+++ b/Source/JavaScriptCore/wasm/WasmB3IRGenerator.cpp
@@ -482,6 +482,9 @@
GPRReg baseMemory = pinnedRegs->baseMemoryPointer;
jit.loadPtr(CCallHelpers::Address(params[0].gpr(), Instance::offsetOfCachedMemorySize()), pinnedRegs->sizeRegister);
jit.loadPtr(CCallHelpers::Address(params[0].gpr(), Instance::offsetOfCachedMemory()), baseMemory);
+#if CPU(ARM64E)
+ jit.untagArrayPtr(pinnedRegs->sizeRegister, baseMemory);
+#endif
});
}
}
@@ -1285,6 +1288,9 @@
ASSERT(pinnedRegs.sizeRegister != newContextInstance);
jit.loadPtr(CCallHelpers::Address(newContextInstance, Instance::offsetOfCachedMemorySize()), pinnedRegs.sizeRegister); // Memory size.
jit.loadPtr(CCallHelpers::Address(newContextInstance, Instance::offsetOfCachedMemory()), baseMemory); // Memory::void*.
+#if CPU(ARM64E)
+ jit.untagArrayPtr(pinnedRegs.sizeRegister, baseMemory);
+#endif
});
doContextSwitch->appendNewControlValue(m_proc, Jump, origin(), continuation);
diff --git a/Source/JavaScriptCore/wasm/WasmBBQPlan.cpp b/Source/JavaScriptCore/wasm/WasmBBQPlan.cpp
index df0c08d..fe09d33 100644
--- a/Source/JavaScriptCore/wasm/WasmBBQPlan.cpp
+++ b/Source/JavaScriptCore/wasm/WasmBBQPlan.cpp
@@ -318,7 +318,7 @@
}
m_wasmInternalFunctions[functionIndex]->entrypoint.compilation = std::make_unique<B3::Compilation>(
- FINALIZE_CODE(linkBuffer, B3CompilationPtrTag, "WebAssembly function[%i] %s", functionIndex, signature.toString().ascii().data()),
+ FINALIZE_CODE(linkBuffer, B3CompilationPtrTag, "WebAssembly BBQ function[%i] %s", functionIndex, signature.toString().ascii().data()),
WTFMove(context.wasmEntrypointByproducts));
}
diff --git a/Source/JavaScriptCore/wasm/WasmBinding.cpp b/Source/JavaScriptCore/wasm/WasmBinding.cpp
index ea13a25..0647a63 100644
--- a/Source/JavaScriptCore/wasm/WasmBinding.cpp
+++ b/Source/JavaScriptCore/wasm/WasmBinding.cpp
@@ -66,7 +66,10 @@
// FIXME the following code assumes that all Wasm::Instance have the same pinned registers. https://bugs.webkit.org/show_bug.cgi?id=162952
// Set up the callee's baseMemory register as well as the memory size registers.
jit.loadPtr(JIT::Address(baseMemory, Wasm::Instance::offsetOfCachedMemorySize()), pinnedRegs.sizeRegister); // Memory size.
- jit.loadPtr(JIT::Address(baseMemory, Wasm::Instance::offsetOfCachedMemory()), baseMemory); // Wasm::Memory::void*.
+ jit.loadPtr(JIT::Address(baseMemory, Wasm::Instance::offsetOfCachedMemory()), baseMemory); // Wasm::Memory::TaggedArrayStoragePtr<void> (void*).
+#if CPU(ARM64E)
+ jit.untagArrayPtr(pinnedRegs.sizeRegister, baseMemory);
+#endif
// Tail call into the callee WebAssembly function.
jit.loadPtr(scratch, scratch);
diff --git a/Source/JavaScriptCore/wasm/WasmInstance.h b/Source/JavaScriptCore/wasm/WasmInstance.h
index 1d389be..387bdd1 100644
--- a/Source/JavaScriptCore/wasm/WasmInstance.h
+++ b/Source/JavaScriptCore/wasm/WasmInstance.h
@@ -64,7 +64,7 @@
Memory* memory() { return m_memory.get(); }
Table* table() { return m_table.get(); }
- void* cachedMemory() const { return m_cachedMemory; }
+ void* cachedMemory() const { return m_cachedMemory.get(cachedMemorySize()); }
size_t cachedMemorySize() const { return m_cachedMemorySize; }
void setMemory(Ref<Memory>&& memory)
@@ -76,7 +76,7 @@
void updateCachedMemory()
{
if (m_memory != nullptr) {
- m_cachedMemory = memory()->memory();
+ m_cachedMemory = TaggedArrayStoragePtr<void>(memory()->memory(), memory()->size());
m_cachedMemorySize = memory()->size();
}
}
@@ -143,7 +143,7 @@
}
void* m_owner { nullptr }; // In a JS embedding, this is a JSWebAssemblyInstance*.
Context* m_context { nullptr };
- void* m_cachedMemory { nullptr };
+ TaggedArrayStoragePtr<void> m_cachedMemory;
size_t m_cachedMemorySize { 0 };
Ref<Module> m_module;
RefPtr<CodeBlock> m_codeBlock;
diff --git a/Source/JavaScriptCore/wasm/WasmMemory.cpp b/Source/JavaScriptCore/wasm/WasmMemory.cpp
index b2da870..4ca54ac 100644
--- a/Source/JavaScriptCore/wasm/WasmMemory.cpp
+++ b/Source/JavaScriptCore/wasm/WasmMemory.cpp
@@ -253,10 +253,11 @@
ASSERT(!initial.bytes());
ASSERT(m_mode == MemoryMode::BoundsChecking);
dataLogLnIf(verbose, "Memory::Memory allocating ", *this);
+ ASSERT(!memory());
}
Memory::Memory(void* memory, PageCount initial, PageCount maximum, size_t mappedCapacity, MemoryMode mode, Function<void(NotifyPressure)>&& notifyMemoryPressure, Function<void(SyncTryToReclaim)>&& syncTryToReclaimMemory, WTF::Function<void(GrowSuccess, PageCount, PageCount)>&& growSuccessCallback)
- : m_memory(memory)
+ : m_memory(memory, initial.bytes())
, m_size(initial.bytes())
, m_initial(initial)
, m_maximum(maximum)
@@ -338,14 +339,14 @@
memoryManager().freePhysicalBytes(m_size);
switch (m_mode) {
case MemoryMode::Signaling:
- if (mprotect(m_memory, Memory::fastMappedBytes(), PROT_READ | PROT_WRITE)) {
+ if (mprotect(memory(), Memory::fastMappedBytes(), PROT_READ | PROT_WRITE)) {
dataLog("mprotect failed: ", strerror(errno), "\n");
RELEASE_ASSERT_NOT_REACHED();
}
- memoryManager().freeFastMemory(m_memory);
+ memoryManager().freeFastMemory(memory());
break;
case MemoryMode::BoundsChecking:
- Gigacage::freeVirtualPages(Gigacage::Primitive, m_memory, m_size);
+ Gigacage::freeVirtualPages(Gigacage::Primitive, memory(), m_size);
break;
}
}
@@ -419,24 +420,26 @@
if (!newMemory)
return makeUnexpected(GrowFailReason::OutOfMemory);
- memcpy(newMemory, m_memory, m_size);
+ memcpy(newMemory, memory(), m_size);
if (m_memory)
- Gigacage::freeVirtualPages(Gigacage::Primitive, m_memory, m_size);
- m_memory = newMemory;
+ Gigacage::freeVirtualPages(Gigacage::Primitive, memory(), m_size);
+ m_memory = TaggedArrayStoragePtr<void>(newMemory, desiredSize);
m_mappedCapacity = desiredSize;
m_size = desiredSize;
+ ASSERT(memory() == newMemory);
return success();
}
case MemoryMode::Signaling: {
- RELEASE_ASSERT(m_memory);
+ RELEASE_ASSERT(memory());
// Signaling memory must have been pre-allocated virtually.
- uint8_t* startAddress = static_cast<uint8_t*>(m_memory) + m_size;
+ uint8_t* startAddress = static_cast<uint8_t*>(memory()) + m_size;
- dataLogLnIf(verbose, "Marking WebAssembly memory's ", RawPointer(m_memory), " as read+write in range [", RawPointer(startAddress), ", ", RawPointer(startAddress + extraBytes), ")");
+ dataLogLnIf(verbose, "Marking WebAssembly memory's ", RawPointer(memory()), " as read+write in range [", RawPointer(startAddress), ", ", RawPointer(startAddress + extraBytes), ")");
if (mprotect(startAddress, extraBytes, PROT_READ | PROT_WRITE)) {
dataLog("mprotect failed: ", strerror(errno), "\n");
RELEASE_ASSERT_NOT_REACHED();
}
+ m_memory.resize(m_size, desiredSize);
m_size = desiredSize;
return success();
}
@@ -460,7 +463,7 @@
void Memory::dump(PrintStream& out) const
{
- out.print("Memory at ", RawPointer(m_memory), ", size ", m_size, "B capacity ", m_mappedCapacity, "B, initial ", m_initial, " maximum ", m_maximum, " mode ", makeString(m_mode));
+ out.print("Memory at ", RawPointer(memory()), ", size ", m_size, "B capacity ", m_mappedCapacity, "B, initial ", m_initial, " maximum ", m_maximum, " mode ", makeString(m_mode));
}
} // namespace JSC
diff --git a/Source/JavaScriptCore/wasm/WasmMemory.h b/Source/JavaScriptCore/wasm/WasmMemory.h
index 00737fc..9adbb11 100644
--- a/Source/JavaScriptCore/wasm/WasmMemory.h
+++ b/Source/JavaScriptCore/wasm/WasmMemory.h
@@ -34,6 +34,7 @@
#include <wtf/Function.h>
#include <wtf/RefCounted.h>
#include <wtf/RefPtr.h>
+#include <wtf/TaggedArrayStoragePtr.h>
#include <wtf/Vector.h>
#include <wtf/WeakPtr.h>
@@ -68,7 +69,7 @@
static size_t fastMappedBytes(); // Includes redzone.
static bool addressIsInActiveFastMemory(void*);
- void* memory() const { return m_memory; }
+ void* memory() const { ASSERT(m_memory.get(size()) == m_memory.getUnsafe()); return m_memory.get(size()); }
size_t size() const { return m_size; }
PageCount sizeInPages() const { return PageCount::fromBytes(m_size); }
@@ -96,7 +97,7 @@
Memory(void* memory, PageCount initial, PageCount maximum, size_t mappedCapacity, MemoryMode, WTF::Function<void(NotifyPressure)>&& notifyMemoryPressure, WTF::Function<void(SyncTryToReclaim)>&& syncTryToReclaimMemory, WTF::Function<void(GrowSuccess, PageCount, PageCount)>&& growSuccessCallback);
Memory(PageCount initial, PageCount maximum, WTF::Function<void(NotifyPressure)>&& notifyMemoryPressure, WTF::Function<void(SyncTryToReclaim)>&& syncTryToReclaimMemory, WTF::Function<void(GrowSuccess, PageCount, PageCount)>&& growSuccessCallback);
- void* m_memory { nullptr };
+ TaggedArrayStoragePtr<void> m_memory;
size_t m_size { 0 };
PageCount m_initial;
PageCount m_maximum;
diff --git a/Source/JavaScriptCore/wasm/js/JSToWasm.cpp b/Source/JavaScriptCore/wasm/js/JSToWasm.cpp
index 20a6e8d..e036cef 100644
--- a/Source/JavaScriptCore/wasm/js/JSToWasm.cpp
+++ b/Source/JavaScriptCore/wasm/js/JSToWasm.cpp
@@ -29,6 +29,7 @@
#if ENABLE(WEBASSEMBLY)
#include "CCallHelpers.h"
+#include "DisallowMacroScratchRegisterUsage.h"
#include "JSCInlines.h"
#include "JSWebAssemblyInstance.h"
#include "JSWebAssemblyRuntimeError.h"
@@ -214,10 +215,23 @@
jit.loadWasmContextInstance(baseMemory);
GPRReg currentInstanceGPR = Context::useFastTLS() ? baseMemory : wasmContextInstanceGPR;
- if (mode != MemoryMode::Signaling)
+ if (mode != MemoryMode::Signaling) {
jit.loadPtr(CCallHelpers::Address(currentInstanceGPR, Wasm::Instance::offsetOfCachedMemorySize()), pinnedRegs.sizeRegister);
+ jit.loadPtr(CCallHelpers::Address(currentInstanceGPR, Wasm::Instance::offsetOfCachedMemory()), baseMemory);
+#if CPU(ARM64E)
+ jit.untagArrayPtr(pinnedRegs.sizeRegister, baseMemory);
+#endif
+ } else {
+#if CPU(ARM64E)
+ GPRReg scratch = wasmCallingConventionAir().prologueScratch(0);
- jit.loadPtr(CCallHelpers::Address(currentInstanceGPR, Wasm::Instance::offsetOfCachedMemory()), baseMemory);
+ jit.loadPtr(CCallHelpers::Address(currentInstanceGPR, Wasm::Instance::offsetOfCachedMemorySize()), scratch);
+ jit.loadPtr(CCallHelpers::Address(currentInstanceGPR, Wasm::Instance::offsetOfCachedMemory()), baseMemory);
+ jit.untagArrayPtr(scratch, baseMemory);
+#else
+ jit.loadPtr(CCallHelpers::Address(currentInstanceGPR, Wasm::Instance::offsetOfCachedMemory()), baseMemory);
+#endif
+ }
}
CCallHelpers::Call call = jit.threadSafePatchableNearCall();
diff --git a/Source/JavaScriptCore/wasm/js/WebAssemblyFunction.cpp b/Source/JavaScriptCore/wasm/js/WebAssemblyFunction.cpp
index 8f6e56d..b344c65 100644
--- a/Source/JavaScriptCore/wasm/js/WebAssemblyFunction.cpp
+++ b/Source/JavaScriptCore/wasm/js/WebAssemblyFunction.cpp
@@ -397,11 +397,20 @@
GPRReg baseMemory = pinnedRegs.baseMemoryPointer;
if (instance()->memoryMode() != Wasm::MemoryMode::Signaling) {
- ASSERT(pinnedRegs.sizeRegister != scratchGPR);
jit.loadPtr(CCallHelpers::Address(scratchGPR, Wasm::Instance::offsetOfCachedMemorySize()), pinnedRegs.sizeRegister);
+ jit.loadPtr(CCallHelpers::Address(scratchGPR, Wasm::Instance::offsetOfCachedMemory()), baseMemory);
+#if CPU(ARM64E)
+ jit.untagArrayPtr(pinnedRegs.sizeRegister, baseMemory);
+#endif
+ } else {
+#if CPU(ARM64E)
+ jit.loadPtr(CCallHelpers::Address(scratchGPR, Wasm::Instance::offsetOfCachedMemorySize()), scratch2GPR);
+ jit.loadPtr(CCallHelpers::Address(scratchGPR, Wasm::Instance::offsetOfCachedMemory()), baseMemory);
+ jit.untagArrayPtr(scratch2GPR, baseMemory);
+#else
+ jit.loadPtr(CCallHelpers::Address(scratchGPR, Wasm::Instance::offsetOfCachedMemory()), baseMemory);
+#endif
}
-
- jit.loadPtr(CCallHelpers::Address(scratchGPR, Wasm::Instance::offsetOfCachedMemory()), baseMemory);
}
// We use this callee to indicate how to unwind past these types of frames:
diff --git a/Source/WTF/ChangeLog b/Source/WTF/ChangeLog
index a020da15..b92d057 100644
--- a/Source/WTF/ChangeLog
+++ b/Source/WTF/ChangeLog
@@ -1,3 +1,55 @@
+2019-05-08 Keith Miller <keith_miller@apple.com>
+
+ Remove Gigacage from arm64 and use PAC for arm64e instead
+ https://bugs.webkit.org/show_bug.cgi?id=197110
+
+ Reviewed by Saam Barati.
+
+ This patch changes the Gigacage to use PAC on arm64e. As part of
+ this process all platforms must provide their length when
+ materializing the caged pointer. Since it would be somewhat
+ confusing to have two parameters for an operator [] those methods
+ have been removed. Lastly, this patch removes the specializations
+ for void* caged pointers, instead opting to use enable_if on the
+ methods that would normally fail on void* e.g. anything that
+ returns a T&.
+
+ * WTF.xcodeproj/project.pbxproj:
+ * wtf/CMakeLists.txt:
+ * wtf/CagedPtr.h:
+ (WTF::CagedPtr::CagedPtr):
+ (WTF::CagedPtr::get const):
+ (WTF::CagedPtr::getMayBeNull const):
+ (WTF::CagedPtr::getUnsafe const):
+ (WTF::CagedPtr::at const):
+ (WTF::CagedPtr::reauthenticate):
+ (WTF::CagedPtr::operator=):
+ (WTF::CagedPtr::operator== const):
+ (WTF::CagedPtr::operator bool const):
+ (WTF::CagedPtr::operator* const): Deleted.
+ (WTF::CagedPtr::operator-> const): Deleted.
+ (WTF::CagedPtr::operator[] const): Deleted.
+ (): Deleted.
+ * wtf/CagedUniquePtr.h:
+ (WTF::CagedUniquePtr::CagedUniquePtr):
+ (WTF::CagedUniquePtr::create):
+ (WTF::CagedUniquePtr::~CagedUniquePtr):
+ (WTF::CagedUniquePtr::destroy):
+ (): Deleted.
+ * wtf/Gigacage.h:
+ (Gigacage::cagedMayBeNull):
+ * wtf/PtrTag.h:
+ (WTF::tagArrayPtr):
+ (WTF::untagArrayPtr):
+ (WTF::removeArrayPtrTag):
+ (WTF::retagArrayPtr):
+ * wtf/TaggedArrayStoragePtr.h: Copied from Source/JavaScriptCore/runtime/ArrayBufferView.cpp.
+ (WTF::TaggedArrayStoragePtr::TaggedArrayStoragePtr):
+ (WTF::TaggedArrayStoragePtr::get const):
+ (WTF::TaggedArrayStoragePtr::getUnsafe const):
+ (WTF::TaggedArrayStoragePtr::resize):
+ (WTF::TaggedArrayStoragePtr::operator bool const):
+
2019-05-08 Robin Morisset <rmorisset@apple.com>
WTF::TimingScope should show the total duration and not just the mean
diff --git a/Source/WTF/WTF.xcodeproj/project.pbxproj b/Source/WTF/WTF.xcodeproj/project.pbxproj
index 0ba9840..8d5efaf 100644
--- a/Source/WTF/WTF.xcodeproj/project.pbxproj
+++ b/Source/WTF/WTF.xcodeproj/project.pbxproj
@@ -650,6 +650,7 @@
DCEE21FC1CEA7551000C2396 /* BlockObjCExceptions.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = BlockObjCExceptions.h; sourceTree = "<group>"; };
DCEE21FD1CEA7551000C2396 /* BlockObjCExceptions.mm */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.objcpp; path = BlockObjCExceptions.mm; sourceTree = "<group>"; };
DCEE22041CEB9869000C2396 /* BackwardsGraph.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = BackwardsGraph.h; sourceTree = "<group>"; };
+ DEF7FE5F22581AC800C15129 /* TaggedArrayStoragePtr.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = TaggedArrayStoragePtr.h; sourceTree = "<group>"; };
E15556F318A0CC18006F48FB /* CryptographicUtilities.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = CryptographicUtilities.cpp; sourceTree = "<group>"; };
E15556F418A0CC18006F48FB /* CryptographicUtilities.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = CryptographicUtilities.h; sourceTree = "<group>"; };
E300E521203D645F00DA79BE /* UniqueArray.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = UniqueArray.h; sourceTree = "<group>"; };
@@ -1154,6 +1155,7 @@
5597F82C1D94B9970066BC21 /* SynchronizedFixedQueue.h */,
E3E158251EADA53C004A079D /* SystemFree.h */,
0FB317C31C488001007E395A /* SystemTracing.h */,
+ DEF7FE5F22581AC800C15129 /* TaggedArrayStoragePtr.h */,
E311FB151F0A568B003C08DE /* ThreadGroup.cpp */,
E311FB161F0A568B003C08DE /* ThreadGroup.h */,
A8A47332151A825B004123FF /* Threading.cpp */,
diff --git a/Source/WTF/wtf/CMakeLists.txt b/Source/WTF/wtf/CMakeLists.txt
index 1990ca9..e60ec0b 100644
--- a/Source/WTF/wtf/CMakeLists.txt
+++ b/Source/WTF/wtf/CMakeLists.txt
@@ -229,6 +229,7 @@
SynchronizedFixedQueue.h
SystemFree.h
SystemTracing.h
+ TaggedArrayStoragePtr.h
ThreadGroup.h
ThreadMessage.h
ThreadSafeRefCounted.h
diff --git a/Source/WTF/wtf/CagedPtr.h b/Source/WTF/wtf/CagedPtr.h
index d4b4ba1..7c80efb 100644
--- a/Source/WTF/wtf/CagedPtr.h
+++ b/Source/WTF/wtf/CagedPtr.h
@@ -27,50 +27,91 @@
#include <wtf/DumbPtrTraits.h>
#include <wtf/Gigacage.h>
+#include <wtf/PtrTag.h>
namespace WTF {
-template<Gigacage::Kind passedKind, typename T, typename PtrTraits = DumbPtrTraits<T>>
+constexpr bool tagCagedPtr = true;
+
+template<Gigacage::Kind passedKind, typename T, bool shouldTag = false, typename PtrTraits = DumbPtrTraits<T>>
class CagedPtr {
public:
static constexpr Gigacage::Kind kind = passedKind;
CagedPtr() : CagedPtr(nullptr) { }
- CagedPtr(std::nullptr_t) : m_ptr(nullptr) { }
+ CagedPtr(std::nullptr_t)
+ : m_ptr(shouldTag ? tagArrayPtr<T>(nullptr, 0) : nullptr)
+ { }
- explicit CagedPtr(T* ptr)
- : m_ptr(ptr)
- {
- }
-
- T* get() const
+ CagedPtr(T* ptr, unsigned size)
+ : m_ptr(shouldTag ? tagArrayPtr(ptr, size) : ptr)
+ { }
+
+
+ T* get(unsigned size) const
{
ASSERT(m_ptr);
- return Gigacage::caged(kind, PtrTraits::unwrap(m_ptr));
- }
-
- T* getMayBeNull() const
- {
- if (!m_ptr)
- return nullptr;
- return get();
+ T* ptr = PtrTraits::unwrap(m_ptr);
+ if (shouldTag)
+ ptr = untagArrayPtr(ptr, size);
+ return Gigacage::caged(kind, ptr);
}
- CagedPtr& operator=(T* ptr)
+ T* getMayBeNull(unsigned size) const
{
- m_ptr = ptr;
+ T* ptr = PtrTraits::unwrap(m_ptr);
+ if (shouldTag)
+ ptr = untagArrayPtr(ptr, size);
+ return Gigacage::cagedMayBeNull(kind, ptr);
+ }
+
+ T* getUnsafe() const
+ {
+ T* ptr = PtrTraits::unwrap(m_ptr);
+ if (shouldTag)
+ ptr = removeArrayPtrTag(ptr);
+ return Gigacage::cagedMayBeNull(kind, ptr);
+ }
+
+ // We need the template here so that the type of U is deduced at usage time rather than class time. U should always be T.
+ template<typename U = T>
+ typename std::enable_if<!std::is_same<void, U>::value, T>::type&
+ /* T& */ at(unsigned index, unsigned size) const { return get(size)[index]; }
+
+ void reauthenticate(unsigned oldSize, unsigned newSize)
+ {
+ auto ptr = get(oldSize);
+ ASSERT(ptr == getUnsafe());
+ *this = CagedPtr(ptr, newSize);
+ }
+
+ CagedPtr(CagedPtr& other)
+ : m_ptr(other.m_ptr)
+ {
+ }
+
+ CagedPtr& operator=(const CagedPtr& ptr)
+ {
+ m_ptr = ptr.m_ptr;
return *this;
}
- CagedPtr& operator=(T*&& ptr)
+ CagedPtr(CagedPtr&& other)
+ : m_ptr(PtrTraits::exchange(other.m_ptr, nullptr))
{
- m_ptr = WTFMove(ptr);
+ }
+
+ CagedPtr& operator=(CagedPtr&& ptr)
+ {
+ m_ptr = PtrTraits::exchange(ptr.m_ptr, nullptr);
return *this;
}
bool operator==(const CagedPtr& other) const
{
- return getMayBeNull() == other.getMayBeNull();
+ bool result = m_ptr == other.m_ptr;
+ ASSERT(result == (getUnsafe() == other.getUnsafe()));
+ return result;
}
bool operator!=(const CagedPtr& other) const
@@ -80,64 +121,7 @@
explicit operator bool() const
{
- return *this != CagedPtr();
- }
-
- T& operator*() const { return *get(); }
- T* operator->() const { return get(); }
-
- template<typename IndexType>
- T& operator[](IndexType index) const { return get()[index]; }
-
-protected:
- typename PtrTraits::StorageType m_ptr;
-};
-
-template<Gigacage::Kind passedKind, typename PtrTraits>
-class CagedPtr<passedKind, void, PtrTraits> {
-public:
- static constexpr Gigacage::Kind kind = passedKind;
-
- CagedPtr() : CagedPtr(nullptr) { }
- CagedPtr(std::nullptr_t) : m_ptr(nullptr) { }
-
- explicit CagedPtr(void* ptr)
- : m_ptr(ptr)
- {
- }
-
- void* get() const
- {
- ASSERT(m_ptr);
- return Gigacage::caged(kind, PtrTraits::unwrap(m_ptr));
- }
-
- void* getMayBeNull() const
- {
- if (!m_ptr)
- return nullptr;
- return get();
- }
-
- CagedPtr& operator=(void* ptr)
- {
- m_ptr = ptr;
- return *this;
- }
-
- bool operator==(const CagedPtr& other) const
- {
- return getMayBeNull() == other.getMayBeNull();
- }
-
- bool operator!=(const CagedPtr& other) const
- {
- return !(*this == other);
- }
-
- explicit operator bool() const
- {
- return *this != CagedPtr();
+ return getUnsafe() != nullptr;
}
protected:
@@ -147,4 +131,5 @@
} // namespace WTF
using WTF::CagedPtr;
+using WTF::tagCagedPtr;
diff --git a/Source/WTF/wtf/CagedUniquePtr.h b/Source/WTF/wtf/CagedUniquePtr.h
index 4aae287..2ed083c 100644
--- a/Source/WTF/wtf/CagedUniquePtr.h
+++ b/Source/WTF/wtf/CagedUniquePtr.h
@@ -29,141 +29,37 @@
namespace WTF {
-template<Gigacage::Kind kind, typename T, typename Enable = void>
-class CagedUniquePtr : public CagedPtr<kind, T> {
+template<Gigacage::Kind kind, typename T, bool shouldTag = false>
+class CagedUniquePtr : public CagedPtr<kind, T, shouldTag> {
+ static_assert(std::is_trivially_destructible<T>::value, "We expect the contents of a caged pointer to be trivially destructable.");
public:
- CagedUniquePtr(T* ptr = nullptr)
- : CagedPtr<kind, T>(ptr)
- {
- }
-
- CagedUniquePtr(CagedUniquePtr&& ptr)
- : CagedPtr<kind, T>(ptr.m_ptr)
- {
- ptr.m_ptr = nullptr;
- }
-
- CagedUniquePtr(const CagedUniquePtr&) = delete;
-
- template<typename... Arguments>
- static CagedUniquePtr create(Arguments&&... arguments)
- {
- T* result = static_cast<T*>(Gigacage::malloc(kind, sizeof(T)));
- new (result) T(std::forward<Arguments>(arguments)...);
- return CagedUniquePtr(result);
- }
-
- CagedUniquePtr& operator=(CagedUniquePtr&& ptr)
- {
- destroy();
- this->m_ptr = ptr.m_ptr;
- ptr.m_ptr = nullptr;
- return *this;
- }
-
- CagedUniquePtr& operator=(const CagedUniquePtr&) = delete;
-
- ~CagedUniquePtr()
- {
- destroy();
- }
-
-private:
- void destroy()
- {
- if (!this->m_ptr)
- return;
- this->m_ptr->~T();
- Gigacage::free(kind, this->m_ptr);
- }
-};
+ using Base = CagedPtr<kind, T, shouldTag>;
+ CagedUniquePtr() = default;
-template<Gigacage::Kind kind, typename T>
-class CagedUniquePtr<kind, T[], typename std::enable_if<std::is_trivially_destructible<T>::value>::type> : public CagedPtr<kind, T> {
-public:
- CagedUniquePtr() : CagedPtr<kind, T>() { }
-
- CagedUniquePtr(T* ptr)
- : CagedPtr<kind, T>(ptr)
- {
- }
-
- CagedUniquePtr(CagedUniquePtr&& ptr)
- : CagedPtr<kind, T>(ptr.m_ptr)
- {
- ptr.m_ptr = nullptr;
- }
-
- CagedUniquePtr(const CagedUniquePtr&) = delete;
-
- template<typename... Arguments>
- static CagedUniquePtr create(size_t count, Arguments&&... arguments)
- {
- T* result = static_cast<T*>(Gigacage::mallocArray(kind, count, sizeof(T)));
- while (count--)
- new (result + count) T(std::forward<Arguments>(arguments)...);
- return CagedUniquePtr(result);
- }
-
- CagedUniquePtr& operator=(CagedUniquePtr&& ptr)
- {
- destroy();
- this->m_ptr = ptr.m_ptr;
- ptr.m_ptr = nullptr;
- return *this;
- }
-
- CagedUniquePtr& operator=(const CagedUniquePtr&) = delete;
-
- ~CagedUniquePtr()
- {
- destroy();
- }
-
-private:
- void destroy()
- {
- if (!this->m_ptr)
- return;
- Gigacage::free(kind, this->m_ptr);
- }
-};
+ CagedUniquePtr(T* ptr, unsigned size)
+ : Base(ptr, size)
+ { }
-template<Gigacage::Kind kind, typename T>
-class CagedUniquePtr<kind, T[], typename std::enable_if<!std::is_trivially_destructible<T>::value>::type> : public CagedPtr<kind, T> {
-public:
- CagedUniquePtr() : CagedPtr<kind, T>() { }
-
- CagedUniquePtr(T* ptr, size_t count)
- : CagedPtr<kind, T>(ptr)
- , m_count(count)
- {
- }
-
CagedUniquePtr(CagedUniquePtr&& ptr)
- : CagedPtr<kind, T>(ptr.m_ptr)
- , m_count(ptr.m_count)
- {
- ptr.clear();
- }
+ : Base(std::forward<CagedUniquePtr&&>(ptr))
+ { }
CagedUniquePtr(const CagedUniquePtr&) = delete;
template<typename... Arguments>
- static CagedUniquePtr create(size_t count, Arguments&&... arguments)
+ static CagedUniquePtr create(unsigned length, Arguments&&... arguments)
{
- T* result = static_cast<T*>(Gigacage::mallocArray(kind, count, sizeof(T)));
- while (count--)
- new (result + count) T(std::forward<Arguments>(arguments)...);
- return CagedUniquePtr(result, count);
+ T* result = static_cast<T*>(Gigacage::malloc(kind, sizeof(T) * length));
+ while (length--)
+ new (result + length) T(arguments...);
+ return CagedUniquePtr(result, length);
}
CagedUniquePtr& operator=(CagedUniquePtr&& ptr)
{
destroy();
this->m_ptr = ptr.m_ptr;
- m_count = ptr.m_count;
- ptr.clear();
+ ptr.m_ptr = nullptr;
return *this;
}
@@ -173,29 +69,16 @@
{
destroy();
}
-
- // FIXME: It's weird that we inherit CagedPtr::operator== and friends, which don't do anything
- // about m_count. It "works" because pointer equality is enough so long as everything is sane, but
- // it seems like a missed opportunity to assert things.
- // https://bugs.webkit.org/show_bug.cgi?id=175541
-
+
private:
void destroy()
{
- if (!this->m_ptr)
+ T* ptr = Base::getUnsafe();
+ if (!ptr)
return;
- while (m_count--)
- this->m_ptr[m_count].~T();
- Gigacage::free(kind, this->m_ptr);
+ ptr->~T();
+ Gigacage::free(kind, ptr);
}
-
- void clear()
- {
- this->m_ptr = nullptr;
- m_count = 0;
- }
-
- size_t m_count { 0 };
};
} // namespace WTF
diff --git a/Source/WTF/wtf/Gigacage.h b/Source/WTF/wtf/Gigacage.h
index 233f56e..227f647 100644
--- a/Source/WTF/wtf/Gigacage.h
+++ b/Source/WTF/wtf/Gigacage.h
@@ -114,6 +114,8 @@
template<typename T>
inline T* caged(Kind, T* ptr) { return ptr; }
+template<typename T>
+inline T* cagedMayBeNull(Kind, T* ptr) { return ptr; }
inline bool isCaged(Kind, const void*) { return false; }
diff --git a/Source/WTF/wtf/PtrTag.h b/Source/WTF/wtf/PtrTag.h
index 342a40f..a0e7856 100644
--- a/Source/WTF/wtf/PtrTag.h
+++ b/Source/WTF/wtf/PtrTag.h
@@ -121,6 +121,39 @@
} \
} while (false)
+
+template<typename T>
+inline T* tagArrayPtr(nullptr_t ptr, size_t length)
+{
+ ASSERT(!length);
+ return ptrauth_sign_unauthenticated(static_cast<T*>(ptr), ptrauth_key_process_dependent_data, length);
+}
+
+
+template<typename T>
+inline T* tagArrayPtr(T* ptr, size_t length)
+{
+ return ptrauth_sign_unauthenticated(ptr, ptrauth_key_process_dependent_data, length);
+}
+
+template<typename T>
+inline T* untagArrayPtr(T* ptr, size_t length)
+{
+ return ptrauth_auth_data(ptr, ptrauth_key_process_dependent_data, length);
+}
+
+template<typename T>
+inline T* removeArrayPtrTag(T* ptr)
+{
+ return ptrauth_strip(ptr, ptrauth_key_process_dependent_data);
+}
+
+template<typename T>
+inline T* retagArrayPtr(T* ptr, size_t oldLength, size_t newLength)
+{
+ return ptrauth_auth_and_resign(ptr, ptrauth_key_process_dependent_data, oldLength, ptrauth_key_process_dependent_data, newLength);
+}
+
template<typename T, typename PtrType, typename = std::enable_if_t<std::is_pointer<PtrType>::value && !std::is_same<T, PtrType>::value>>
inline constexpr T removeCodePtrTag(PtrType ptr)
{
@@ -394,6 +427,38 @@
inline void registerPtrTagLookup(PtrTagLookup*) { }
inline void reportBadTag(const void*, PtrTag) { }
+template<typename T>
+inline T* tagArrayPtr(nullptr_t, size_t size)
+{
+ ASSERT_UNUSED(size, !size);
+ return nullptr;
+}
+
+template<typename T>
+inline T* tagArrayPtr(T* ptr, size_t)
+{
+ return ptr;
+}
+
+template<typename T>
+inline T* untagArrayPtr(T* ptr, size_t)
+{
+ return ptr;
+}
+
+template<typename T>
+inline T* removeArrayPtrTag(T* ptr)
+{
+ return ptr;
+}
+
+template<typename T>
+inline T* retagArrayPtr(T* ptr, size_t, size_t)
+{
+ return ptr;
+}
+
+
template<typename T, typename PtrType, typename = std::enable_if_t<std::is_pointer<PtrType>::value && !std::is_same<T, PtrType>::value>>
constexpr T tagCodePtr(PtrType ptr, PtrTag) { return bitwise_cast<T>(ptr); }
@@ -493,6 +558,11 @@
using WTF::reportBadTag;
+using WTF::tagArrayPtr;
+using WTF::untagArrayPtr;
+using WTF::retagArrayPtr;
+using WTF::removeArrayPtrTag;
+
using WTF::tagCodePtr;
using WTF::untagCodePtr;
using WTF::retagCodePtr;
diff --git a/Source/WTF/wtf/TaggedArrayStoragePtr.h b/Source/WTF/wtf/TaggedArrayStoragePtr.h
new file mode 100644
index 0000000..553e86f
--- /dev/null
+++ b/Source/WTF/wtf/TaggedArrayStoragePtr.h
@@ -0,0 +1,59 @@
+/*
+ * Copyright (C) 2019 Apple Inc. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in the
+ * documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. ``AS IS'' AND ANY
+ * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+ * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR
+ * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
+ * EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
+ * PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
+ * PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
+ * OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#pragma once
+
+#include <wtf/PtrTag.h>
+
+namespace WTF {
+
+template<typename PtrType>
+class TaggedArrayStoragePtr {
+public:
+ TaggedArrayStoragePtr()
+ : m_ptr(tagArrayPtr<PtrType>(nullptr, 0))
+ { }
+
+ TaggedArrayStoragePtr(PtrType* ptr, unsigned length)
+ : m_ptr(tagArrayPtr(ptr, length))
+ { }
+
+ PtrType* get(unsigned length) const { return untagArrayPtr(m_ptr, length); }
+ PtrType* getUnsafe() const { return removeArrayPtrTag(m_ptr); }
+
+ void resize(unsigned oldLength, unsigned newLength)
+ {
+ m_ptr = retagArrayPtr(m_ptr, oldLength, newLength);
+ }
+
+ explicit operator bool() const { return !!getUnsafe(); }
+
+private:
+ PtrType* m_ptr;
+};
+
+}
+
+using WTF::TaggedArrayStoragePtr;
diff --git a/Source/bmalloc/ChangeLog b/Source/bmalloc/ChangeLog
index d265b86..128c1ff 100644
--- a/Source/bmalloc/ChangeLog
+++ b/Source/bmalloc/ChangeLog
@@ -1,3 +1,16 @@
+2019-05-08 Keith Miller <keith_miller@apple.com>
+
+ Remove Gigacage from arm64 and use PAC for arm64e instead
+ https://bugs.webkit.org/show_bug.cgi?id=197110
+
+ Reviewed by Saam Barati.
+
+ Stop using gigacage on arm64 and add a new cage function cagedMayBeNull that is the same as
+ cage but returns a nullptr if the incoming pointer is already null.
+
+ * bmalloc/Gigacage.h:
+ (Gigacage::cagedMayBeNull):
+
2019-04-29 Alex Christensen <achristensen@webkit.org>
<rdar://problem/50299396> Fix internal High Sierra build
diff --git a/Source/bmalloc/bmalloc/Gigacage.h b/Source/bmalloc/bmalloc/Gigacage.h
index 70cce67..bcde37c 100644
--- a/Source/bmalloc/bmalloc/Gigacage.h
+++ b/Source/bmalloc/bmalloc/Gigacage.h
@@ -34,8 +34,7 @@
#include <cstddef>
#include <inttypes.h>
-#if ((BOS(DARWIN) || BOS(LINUX)) && \
-(BCPU(X86_64) || (BCPU(ARM64) && !defined(__ILP32__) && (!BPLATFORM(IOS_FAMILY) || BPLATFORM(IOS)))))
+#if ((BOS(DARWIN) || BOS(LINUX)) && BCPU(X86_64))
#define GIGACAGE_ENABLED 1
#else
#define GIGACAGE_ENABLED 0
@@ -200,6 +199,14 @@
reinterpret_cast<uintptr_t>(ptr) & mask(kind)));
}
+template<typename T>
+BINLINE T* cagedMayBeNull(Kind kind, T* ptr)
+{
+ if (!ptr)
+ return ptr;
+ return caged(kind, ptr);
+}
+
BINLINE bool isCaged(Kind kind, const void* ptr)
{
return caged(kind, ptr) == ptr;
@@ -221,6 +228,7 @@
BINLINE bool isCaged(Kind, const void*) { return true; }
BINLINE bool isEnabled() { return false; }
template<typename T> BINLINE T* caged(Kind, T* ptr) { return ptr; }
+template<typename T> BINLINE T* cagedMayBeNull(Kind, T* ptr) { return ptr; }
BINLINE void disableDisablingPrimitiveGigacageIfShouldBeEnabled() { }
BINLINE bool canPrimitiveGigacageBeDisabled() { return false; }
BINLINE void disablePrimitiveGigacage() { }
