Resource Load Statistics (experimental): Add fast mode for non-cookie website data deletion
https://bugs.webkit.org/show_bug.cgi?id=204858
<rdar://problem/57639851>
Reviewed by Alex Christensen.
Source/WebCore:
This change adds two internal flags:
- "Live-On Testing" with a one hour timeout instead of seven days.
- "Repro Testing" with an instant timeout (bar ITP's regular delays) instead of seven days.
These internal flags should be removed once testing is complete: <rdar://problem/57673418>
No new tests. This change just adds new opt-in settings for manual testing.
* page/Settings.yaml:
* platform/network/NetworkStorageSession.h:
The FirstPartyWebsiteDataRemovalMode enum now has two new values:
- AllButCookiesLiveOnTestingTimeout
- AllButCookiesReproTestingTimeout
Source/WebKit:
The purpose of this change is to allow for dedicated testing of the change in
https://trac.webkit.org/changeset/253082/webkit. Waiting seven days just isn't a good
starting point.
This change adds two internal flags:
- "Live-On Testing" with a one hour timeout instead of seven days.
- "Repro Testing" with an instant timeout (bar ITP's regular delays) instead of seven days.
The change also makes sure that hasHadUnexpiredRecentUserInteraction() in
ResourceLoadStatisticsDatabaseStore and ResourceLoadStatisticsMemoryStore only
age out the user interaction timestamp if the OperatingDatesWindow is Long so
that we don't age out timestamps early with the shorter OperatingDatesWindows.
This change changes the default value of IsFirstPartyWebsiteDataRemovalEnabled to true.
These internal flags should be removed once testing is complete: <rdar://problem/57673418>
* NetworkProcess/Classifier/ResourceLoadStatisticsDatabaseStore.cpp:
* NetworkProcess/Classifier/ResourceLoadStatisticsMemoryStore.cpp:
(WebKit::ResourceLoadStatisticsMemoryStore::hasHadUnexpiredRecentUserInteraction const):
(WebKit::ResourceLoadStatisticsMemoryStore::shouldRemoveAllButCookiesFor const):
* NetworkProcess/Classifier/ResourceLoadStatisticsStore.cpp:
(WebKit::ResourceLoadStatisticsStore::hasStatisticsExpired const):
* NetworkProcess/Classifier/ResourceLoadStatisticsStore.h:
* NetworkProcess/Classifier/WebResourceLoadStatisticsStore.cpp:
(WebKit::WebResourceLoadStatisticsStore::setFirstPartyWebsiteDataRemovalMode):
* NetworkProcess/NetworkProcess.messages.in:
* Shared/WebPreferences.yaml:
* UIProcess/WebsiteData/Cocoa/WebsiteDataStoreCocoa.mm:
(WebKit::WebsiteDataStore::parameters):
git-svn-id: http://svn.webkit.org/repository/webkit/trunk@253185 268f45cc-cd09-0410-ab3c-d52691b4dbfc
diff --git a/Source/WebCore/ChangeLog b/Source/WebCore/ChangeLog
index f7ff228..c475a12 100644
--- a/Source/WebCore/ChangeLog
+++ b/Source/WebCore/ChangeLog
@@ -1,3 +1,25 @@
+2019-12-05 John Wilander <wilander@apple.com>
+
+ Resource Load Statistics (experimental): Add fast mode for non-cookie website data deletion
+ https://bugs.webkit.org/show_bug.cgi?id=204858
+ <rdar://problem/57639851>
+
+ Reviewed by Alex Christensen.
+
+ This change adds two internal flags:
+ - "Live-On Testing" with a one hour timeout instead of seven days.
+ - "Repro Testing" with an instant timeout (bar ITP's regular delays) instead of seven days.
+
+ These internal flags should be removed once testing is complete: <rdar://problem/57673418>
+
+ No new tests. This change just adds new opt-in settings for manual testing.
+
+ * page/Settings.yaml:
+ * platform/network/NetworkStorageSession.h:
+ The FirstPartyWebsiteDataRemovalMode enum now has two new values:
+ - AllButCookiesLiveOnTestingTimeout
+ - AllButCookiesReproTestingTimeout
+
2019-12-05 Chris Dumez <cdumez@apple.com>
PageConfiguration::dragClient should use a smart pointer
diff --git a/Source/WebCore/page/Settings.yaml b/Source/WebCore/page/Settings.yaml
index 6122882..a3e3526 100644
--- a/Source/WebCore/page/Settings.yaml
+++ b/Source/WebCore/page/Settings.yaml
@@ -887,6 +887,12 @@
initial: true
isFirstPartyWebsiteDataRemovalEnabled:
+ initial: true
+
+isFirstPartyWebsiteDataRemovalLiveOnTestingEnabled:
+ initial: false
+
+isFirstPartyWebsiteDataRemovalReproTestingEnabled:
initial: false
isLoggedInAPIEnabled:
diff --git a/Source/WebCore/platform/network/NetworkStorageSession.h b/Source/WebCore/platform/network/NetworkStorageSession.h
index 1ba7bf9..eefb870 100644
--- a/Source/WebCore/platform/network/NetworkStorageSession.h
+++ b/Source/WebCore/platform/network/NetworkStorageSession.h
@@ -75,7 +75,7 @@
enum class IncludeSecureCookies : bool;
enum class IncludeHttpOnlyCookies : bool;
enum class ThirdPartyCookieBlockingMode : uint8_t { All, AllOnSitesWithoutUserInteraction, OnlyAccordingToPerDomainPolicy };
-enum class FirstPartyWebsiteDataRemovalMode : bool { AllButCookies, None };
+enum class FirstPartyWebsiteDataRemovalMode : uint8_t { AllButCookies, None, AllButCookiesLiveOnTestingTimeout, AllButCookiesReproTestingTimeout };
class NetworkStorageSession {
WTF_MAKE_NONCOPYABLE(NetworkStorageSession); WTF_MAKE_FAST_ALLOCATED;
@@ -233,4 +233,14 @@
>;
};
+template<> struct EnumTraits<WebCore::FirstPartyWebsiteDataRemovalMode> {
+ using values = EnumValues<
+ WebCore::FirstPartyWebsiteDataRemovalMode,
+ WebCore::FirstPartyWebsiteDataRemovalMode::AllButCookies,
+ WebCore::FirstPartyWebsiteDataRemovalMode::None,
+ WebCore::FirstPartyWebsiteDataRemovalMode::AllButCookiesLiveOnTestingTimeout,
+ WebCore::FirstPartyWebsiteDataRemovalMode::AllButCookiesReproTestingTimeout
+ >;
+};
+
}
diff --git a/Source/WebKit/ChangeLog b/Source/WebKit/ChangeLog
index 5d086a6..6366c6b 100644
--- a/Source/WebKit/ChangeLog
+++ b/Source/WebKit/ChangeLog
@@ -1,3 +1,42 @@
+2019-12-05 John Wilander <wilander@apple.com>
+
+ Resource Load Statistics (experimental): Add fast mode for non-cookie website data deletion
+ https://bugs.webkit.org/show_bug.cgi?id=204858
+ <rdar://problem/57639851>
+
+ Reviewed by Alex Christensen.
+
+ The purpose of this change is to allow for dedicated testing of the change in
+ https://trac.webkit.org/changeset/253082/webkit. Waiting seven days just isn't a good
+ starting point.
+
+ This change adds two internal flags:
+ - "Live-On Testing" with a one hour timeout instead of seven days.
+ - "Repro Testing" with an instant timeout (bar ITP's regular delays) instead of seven days.
+
+ The change also makes sure that hasHadUnexpiredRecentUserInteraction() in
+ ResourceLoadStatisticsDatabaseStore and ResourceLoadStatisticsMemoryStore only
+ age out the user interaction timestamp if the OperatingDatesWindow is Long so
+ that we don't age out timestamps early with the shorter OperatingDatesWindows.
+
+ This change changes the default value of IsFirstPartyWebsiteDataRemovalEnabled to true.
+
+ These internal flags should be removed once testing is complete: <rdar://problem/57673418>
+
+ * NetworkProcess/Classifier/ResourceLoadStatisticsDatabaseStore.cpp:
+ * NetworkProcess/Classifier/ResourceLoadStatisticsMemoryStore.cpp:
+ (WebKit::ResourceLoadStatisticsMemoryStore::hasHadUnexpiredRecentUserInteraction const):
+ (WebKit::ResourceLoadStatisticsMemoryStore::shouldRemoveAllButCookiesFor const):
+ * NetworkProcess/Classifier/ResourceLoadStatisticsStore.cpp:
+ (WebKit::ResourceLoadStatisticsStore::hasStatisticsExpired const):
+ * NetworkProcess/Classifier/ResourceLoadStatisticsStore.h:
+ * NetworkProcess/Classifier/WebResourceLoadStatisticsStore.cpp:
+ (WebKit::WebResourceLoadStatisticsStore::setFirstPartyWebsiteDataRemovalMode):
+ * NetworkProcess/NetworkProcess.messages.in:
+ * Shared/WebPreferences.yaml:
+ * UIProcess/WebsiteData/Cocoa/WebsiteDataStoreCocoa.mm:
+ (WebKit::WebsiteDataStore::parameters):
+
2019-12-05 Chris Dumez <cdumez@apple.com>
Optimize IPC::Connection::SyncMessageState methods
diff --git a/Source/WebKit/NetworkProcess/Classifier/ResourceLoadStatisticsDatabaseStore.cpp b/Source/WebKit/NetworkProcess/Classifier/ResourceLoadStatisticsDatabaseStore.cpp
index 8f89531..dbd0058 100644
--- a/Source/WebKit/NetworkProcess/Classifier/ResourceLoadStatisticsDatabaseStore.cpp
+++ b/Source/WebKit/NetworkProcess/Classifier/ResourceLoadStatisticsDatabaseStore.cpp
@@ -2143,7 +2143,11 @@
bool ResourceLoadStatisticsDatabaseStore::hasHadUnexpiredRecentUserInteraction(const DomainData& resourceStatistic, OperatingDatesWindow operatingDatesWindow)
{
if (resourceStatistic.hadUserInteraction && hasStatisticsExpired(resourceStatistic.mostRecentUserInteractionTime, operatingDatesWindow)) {
- clearUserInteraction(resourceStatistic.registrableDomain, [] { });
+
+ // Drop privacy sensitive data if we no longer need it.
+ if (operatingDatesWindow == OperatingDatesWindow::Long)
+ clearUserInteraction(resourceStatistic.registrableDomain, [] { });
+
return false;
}
@@ -2160,7 +2164,21 @@
bool isRemovalEnabled = firstPartyWebsiteDataRemovalMode() != FirstPartyWebsiteDataRemovalMode::None || resourceStatistic.isScheduledForAllButCookieDataRemoval;
bool isResourceGrandfathered = shouldCheckForGrandfathering && resourceStatistic.grandfathered;
- return isRemovalEnabled && !isResourceGrandfathered && !hasHadUnexpiredRecentUserInteraction(resourceStatistic, OperatingDatesWindow::Short);
+ OperatingDatesWindow window;
+ switch (firstPartyWebsiteDataRemovalMode()) {
+ case FirstPartyWebsiteDataRemovalMode::AllButCookies:
+ FALLTHROUGH;
+ case FirstPartyWebsiteDataRemovalMode::None:
+ window = OperatingDatesWindow::Short;
+ break;
+ case FirstPartyWebsiteDataRemovalMode::AllButCookiesLiveOnTestingTimeout:
+ window = OperatingDatesWindow::ForLiveOnTesting;
+ break;
+ case FirstPartyWebsiteDataRemovalMode::AllButCookiesReproTestingTimeout:
+ window = OperatingDatesWindow::ForReproTesting;
+ }
+
+ return isRemovalEnabled && !isResourceGrandfathered && !hasHadUnexpiredRecentUserInteraction(resourceStatistic, window);
}
Vector<std::pair<RegistrableDomain, WebsiteDataToRemove>> ResourceLoadStatisticsDatabaseStore::registrableDomainsToRemoveWebsiteDataFor()
diff --git a/Source/WebKit/NetworkProcess/Classifier/ResourceLoadStatisticsMemoryStore.cpp b/Source/WebKit/NetworkProcess/Classifier/ResourceLoadStatisticsMemoryStore.cpp
index 340df93..8192853 100644
--- a/Source/WebKit/NetworkProcess/Classifier/ResourceLoadStatisticsMemoryStore.cpp
+++ b/Source/WebKit/NetworkProcess/Classifier/ResourceLoadStatisticsMemoryStore.cpp
@@ -863,12 +863,16 @@
ASSERT(!RunLoop::isMain());
if (resourceStatistic.hadUserInteraction && hasStatisticsExpired(resourceStatistic, operatingDatesWindow)) {
- // Drop privacy sensitive data because we no longer need it.
- // Set timestamp to 0 so that statistics merge will know
- // it has been reset as opposed to its default -1.
- resourceStatistic.mostRecentUserInteractionTime = { };
- resourceStatistic.storageAccessUnderTopFrameDomains.clear();
- resourceStatistic.hadUserInteraction = false;
+ if (operatingDatesWindow == OperatingDatesWindow::Long) {
+ // Drop privacy sensitive data because we no longer need it.
+ // Set timestamp to 0 so that statistics merge will know
+ // it has been reset as opposed to its default -1.
+ resourceStatistic.mostRecentUserInteractionTime = { };
+ resourceStatistic.storageAccessUnderTopFrameDomains.clear();
+ resourceStatistic.hadUserInteraction = false;
+ }
+
+ return false;
}
return resourceStatistic.hadUserInteraction;
@@ -883,8 +887,22 @@
{
bool isRemovalEnabled = firstPartyWebsiteDataRemovalMode() != FirstPartyWebsiteDataRemovalMode::None || resourceStatistic.gotLinkDecorationFromPrevalentResource;
bool isResourceGrandfathered = shouldCheckForGrandfathering && resourceStatistic.grandfathered;
+
+ OperatingDatesWindow window;
+ switch (firstPartyWebsiteDataRemovalMode()) {
+ case FirstPartyWebsiteDataRemovalMode::AllButCookies:
+ FALLTHROUGH;
+ case FirstPartyWebsiteDataRemovalMode::None:
+ window = OperatingDatesWindow::Short;
+ break;
+ case FirstPartyWebsiteDataRemovalMode::AllButCookiesLiveOnTestingTimeout:
+ window = OperatingDatesWindow::ForLiveOnTesting;
+ break;
+ case FirstPartyWebsiteDataRemovalMode::AllButCookiesReproTestingTimeout:
+ window = OperatingDatesWindow::ForReproTesting;
+ }
- return isRemovalEnabled && !isResourceGrandfathered && !hasHadUnexpiredRecentUserInteraction(resourceStatistic, OperatingDatesWindow::Short);
+ return isRemovalEnabled && !isResourceGrandfathered && !hasHadUnexpiredRecentUserInteraction(resourceStatistic, window);
}
Vector<std::pair<RegistrableDomain, WebsiteDataToRemove>> ResourceLoadStatisticsMemoryStore::registrableDomainsToRemoveWebsiteDataFor()
diff --git a/Source/WebKit/NetworkProcess/Classifier/ResourceLoadStatisticsStore.cpp b/Source/WebKit/NetworkProcess/Classifier/ResourceLoadStatisticsStore.cpp
index d93545c..7846b54 100644
--- a/Source/WebKit/NetworkProcess/Classifier/ResourceLoadStatisticsStore.cpp
+++ b/Source/WebKit/NetworkProcess/Classifier/ResourceLoadStatisticsStore.cpp
@@ -53,6 +53,7 @@
constexpr Seconds minimumStatisticsProcessingInterval { 5_s };
constexpr unsigned operatingDatesWindowLong { 30 };
constexpr unsigned operatingDatesWindowShort { 7 };
+constexpr Seconds operatingTimeWindowForLiveOnTesting { 1_h };
#if !RELEASE_LOG_DISABLED
static String domainsToString(const Vector<RegistrableDomain>& domains)
@@ -503,7 +504,20 @@
{
ASSERT(!RunLoop::isMain());
- unsigned operatingDatesWindowInDays = (operatingDatesWindow == OperatingDatesWindow::Long ? operatingDatesWindowLong : operatingDatesWindowShort);
+ unsigned operatingDatesWindowInDays;
+ switch (operatingDatesWindow) {
+ case OperatingDatesWindow::Long:
+ operatingDatesWindowInDays = operatingDatesWindowLong;
+ break;
+ case OperatingDatesWindow::Short:
+ operatingDatesWindowInDays = operatingDatesWindowShort;
+ break;
+ case OperatingDatesWindow::ForLiveOnTesting:
+ return WallTime::now() > mostRecentUserInteractionTime + operatingTimeWindowForLiveOnTesting;
+ case OperatingDatesWindow::ForReproTesting:
+ return true;
+ }
+
if (m_operatingDates.size() >= operatingDatesWindowInDays) {
if (OperatingDate::fromWallTime(mostRecentUserInteractionTime) < m_operatingDates.first())
return true;
diff --git a/Source/WebKit/NetworkProcess/Classifier/ResourceLoadStatisticsStore.h b/Source/WebKit/NetworkProcess/Classifier/ResourceLoadStatisticsStore.h
index 9a5f89a..ae3c8ee 100644
--- a/Source/WebKit/NetworkProcess/Classifier/ResourceLoadStatisticsStore.h
+++ b/Source/WebKit/NetworkProcess/Classifier/ResourceLoadStatisticsStore.h
@@ -74,7 +74,7 @@
int m_monthDay { 0 }; // [1, 31].
};
-enum class OperatingDatesWindow : bool { Long, Short };
+enum class OperatingDatesWindow : uint8_t { Long, Short, ForLiveOnTesting, ForReproTesting };
enum class CookieAccess : uint8_t { CannotRequest, BasedOnCookiePolicy, OnlyIfGranted };
// This is always constructed / used / destroyed on the WebResourceLoadStatisticsStore's statistics queue.
diff --git a/Source/WebKit/NetworkProcess/Classifier/WebResourceLoadStatisticsStore.cpp b/Source/WebKit/NetworkProcess/Classifier/WebResourceLoadStatisticsStore.cpp
index 3ff0a6b..d4586d5 100644
--- a/Source/WebKit/NetworkProcess/Classifier/WebResourceLoadStatisticsStore.cpp
+++ b/Source/WebKit/NetworkProcess/Classifier/WebResourceLoadStatisticsStore.cpp
@@ -486,8 +486,11 @@
ASSERT(RunLoop::isMain());
postTask([this, mode, completionHandler = WTFMove(completionHandler)]() mutable {
- if (m_statisticsStore)
+ if (m_statisticsStore) {
m_statisticsStore->setFirstPartyWebsiteDataRemovalMode(mode);
+ if (mode == FirstPartyWebsiteDataRemovalMode::AllButCookiesReproTestingTimeout)
+ m_statisticsStore->setIsRunningTest(true);
+ }
postTaskReply([completionHandler = WTFMove(completionHandler)]() mutable {
completionHandler();
});
diff --git a/Source/WebKit/NetworkProcess/NetworkProcess.messages.in b/Source/WebKit/NetworkProcess/NetworkProcess.messages.in
index 0257d8d..ee086c0 100644
--- a/Source/WebKit/NetworkProcess/NetworkProcess.messages.in
+++ b/Source/WebKit/NetworkProcess/NetworkProcess.messages.in
@@ -138,7 +138,7 @@
HasIsolatedSession(PAL::SessionID sessionID, WebCore::RegistrableDomain domain) -> (bool hasIsolatedSession) Async
SetShouldDowngradeReferrerForTesting(bool enabled) -> () Async
SetShouldBlockThirdPartyCookiesForTesting(PAL::SessionID sessionID, enum:uint8_t WebCore::ThirdPartyCookieBlockingMode blockingMode) -> () Async
- SetFirstPartyWebsiteDataRemovalModeForTesting(PAL::SessionID sessionID, enum:bool WebCore::FirstPartyWebsiteDataRemovalMode mode) -> () Async
+ SetFirstPartyWebsiteDataRemovalModeForTesting(PAL::SessionID sessionID, enum:uint8_t WebCore::FirstPartyWebsiteDataRemovalMode mode) -> () Async
#endif
SetSessionIsControlledByAutomation(PAL::SessionID sessionID, bool controlled);
diff --git a/Source/WebKit/Shared/WebPreferences.yaml b/Source/WebKit/Shared/WebPreferences.yaml
index c53d664..835fdc8 100644
--- a/Source/WebKit/Shared/WebPreferences.yaml
+++ b/Source/WebKit/Shared/WebPreferences.yaml
@@ -1819,11 +1819,25 @@
IsFirstPartyWebsiteDataRemovalEnabled:
type: bool
- defaultValue: false
+ defaultValue: true
humanReadableName: "Remove Non-Cookie Data After 7 Days of No User Interaction (ITP)"
humanReadableDescription: "Remove all non-cookie website data after seven days of no user interaction when Intelligent Tracking Prevention is enabled"
category: experimental
+IsFirstPartyWebsiteDataRemovalLiveOnTestingEnabled:
+ type: bool
+ defaultValue: false
+ humanReadableName: "[ITP Live-On] 1 Hour Timeout For Non-Cookie Data Removal"
+ humanReadableDescription: "Remove all non-cookie website data after just one hour of no user interaction when Intelligent Tracking Prevention is enabled"
+ category: internal
+
+IsFirstPartyWebsiteDataRemovalReproTestingEnabled:
+ type: bool
+ defaultValue: false
+ humanReadableName: "[ITP Repro] 30 Second Timeout For Non-Cookie Data Removal"
+ humanReadableDescription: "Remove all non-cookie website data after just 30 seconds of no user interaction when Intelligent Tracking Prevention is enabled"
+ category: internal
+
IsLoggedInAPIEnabled:
type: bool
defaultValue: false
diff --git a/Source/WebKit/UIProcess/WebsiteData/Cocoa/WebsiteDataStoreCocoa.mm b/Source/WebKit/UIProcess/WebsiteData/Cocoa/WebsiteDataStoreCocoa.mm
index 3e4b154..1888c8d 100644
--- a/Source/WebKit/UIProcess/WebsiteData/Cocoa/WebsiteDataStoreCocoa.mm
+++ b/Source/WebKit/UIProcess/WebsiteData/Cocoa/WebsiteDataStoreCocoa.mm
@@ -82,8 +82,14 @@
thirdPartyCookieBlockingMode = WebCore::ThirdPartyCookieBlockingMode::All;
else
thirdPartyCookieBlockingMode = WebCore::ThirdPartyCookieBlockingMode::AllOnSitesWithoutUserInteraction;
- if ([defaults boolForKey:[NSString stringWithFormat:@"Experimental%@", WebPreferencesKey::isFirstPartyWebsiteDataRemovalEnabledKey().createCFString().get()]])
- firstPartyWebsiteDataRemovalMode = WebCore::FirstPartyWebsiteDataRemovalMode::AllButCookies;
+ if ([defaults boolForKey:[NSString stringWithFormat:@"Experimental%@", WebPreferencesKey::isFirstPartyWebsiteDataRemovalEnabledKey().createCFString().get()]]) {
+ if ([defaults boolForKey:[NSString stringWithFormat:@"InternalDebug%@", WebPreferencesKey::isFirstPartyWebsiteDataRemovalReproTestingEnabledKey().createCFString().get()]])
+ firstPartyWebsiteDataRemovalMode = WebCore::FirstPartyWebsiteDataRemovalMode::AllButCookiesReproTestingTimeout;
+ else if ([defaults boolForKey:[NSString stringWithFormat:@"InternalDebug%@", WebPreferencesKey::isFirstPartyWebsiteDataRemovalLiveOnTestingEnabledKey().createCFString().get()]])
+ firstPartyWebsiteDataRemovalMode = WebCore::FirstPartyWebsiteDataRemovalMode::AllButCookiesLiveOnTestingTimeout;
+ else
+ firstPartyWebsiteDataRemovalMode = WebCore::FirstPartyWebsiteDataRemovalMode::AllButCookies;
+ }
auto* manualPrevalentResource = [defaults stringForKey:@"ITPManualPrevalentResource"];
if (manualPrevalentResource) {
URL url { URL(), manualPrevalentResource };
