Resource Load Statistics: Grandfather domains for existing data records
https://bugs.webkit.org/show_bug.cgi?id=172155
<rdar://problem/24913532>

Reviewed by Alex Christensen.

Source/WebCore:

Test: http/tests/loading/resourceLoadStatistics/grandfathering.html

* loader/ResourceLoadObserver.cpp:
(WebCore::ResourceLoadObserver::setGrandfathered):
(WebCore::ResourceLoadObserver::isGrandfathered):
(WebCore::ResourceLoadObserver::setMinimumTimeBetweeenDataRecordsRemoval):
(WebCore::ResourceLoadObserver::setGrandfatheringTime):
    Functions for testing and configuration.
    ResourceLoadObserver::setMinimumTimeBetweeenDataRecordsRemoval() changed as a result of moving
    WebKit::WebResourceLoadStatisticsStore::setMinimumTimeBetweeenDataRecordsRemoval() here.
* loader/ResourceLoadObserver.h:
* loader/ResourceLoadStatisticsStore.cpp:
(WebCore::ResourceLoadStatisticsStore::createEncoderFromData):
(WebCore::ResourceLoadStatisticsStore::readDataFromDecoder):
    Now contains endOfGrandfatheringTimestamp.
(WebCore::ResourceLoadStatisticsStore::clearInMemoryAndPersistent):
    Now makes a call to m_grandfatherExistingWebsiteDataHandler().
(WebCore::ResourceLoadStatisticsStore::setGrandfatherExistingWebsiteDataCallback):
(WebCore::ResourceLoadStatisticsStore::setMinimumTimeBetweeenDataRecordsRemoval):
    Changed as a result of moving
    WebKit::WebResourceLoadStatisticsStore::setMinimumTimeBetweeenDataRecordsRemoval() here.
(WebCore::ResourceLoadStatisticsStore::setGrandfatheringTime):
(WebCore::ResourceLoadStatisticsStore::topPrivatelyControlledDomainsToRemoveWebsiteDataFor):
    Renamed since it now also takes grandfathering into account.
(WebCore::ResourceLoadStatisticsStore::updateStatisticsForRemovedDataRecords):
    Fixed typo in local variable name.
(WebCore::ResourceLoadStatisticsStore::handleFreshStartWithEmptyOrNoStore):
(WebCore::ResourceLoadStatisticsStore::shouldRemoveDataRecords):
    Convenience function added.
(WebCore::ResourceLoadStatisticsStore::dataRecordsBeingRemoved):
    Convenience function added.
(WebCore::ResourceLoadStatisticsStore::dataRecordsWereRemoved):
    Convenience function added.
(WebCore::ResourceLoadStatisticsStore::prevalentResourceDomainsWithoutUserInteraction): Deleted.
    Replaced by ResourceLoadStatisticsStore::topPrivatelyControlledDomainsToRemoveWebsiteDataFor().
* loader/ResourceLoadStatisticsStore.h:

Source/WebKit2:

When WebResourceLoadStatisticsStore starts fresh it needs
to scan existing website data records and 'grandfather' them to
allow ample time to capture user interaction.

* Shared/WebPreferencesDefinitions.h:
* UIProcess/API/C/WKResourceLoadStatisticsManager.cpp:
(WKResourceLoadStatisticsManagerSetGrandfathered):
(WKResourceLoadStatisticsManagerIsGrandfathered):
(WKResourceLoadStatisticsManagerSetMinimumTimeBetweeenDataRecordsRemoval):
(WKResourceLoadStatisticsManagerSetGrandfatheringTime):
    Functions for testing and configuration.
* UIProcess/API/C/WKResourceLoadStatisticsManager.h:
* UIProcess/Cocoa/WebResourceLoadStatisticsManagerCocoa.mm:
(WebKit::WebResourceLoadStatisticsManager::registerUserDefaultsIfNeeded):
    Added grandfathering configuration.
* UIProcess/WebProcessProxy.cpp:
(WebKit::WebProcessProxy::topPrivatelyControlledDomainsWithWebiteData):
    New function to get all top privately controlled domains that
    have website data.
* UIProcess/WebProcessProxy.h:
* UIProcess/WebResourceLoadStatisticsManager.cpp:
(WebKit::WebResourceLoadStatisticsManager::setGrandfathered):
(WebKit::WebResourceLoadStatisticsManager::isGrandfathered):
(WebKit::WebResourceLoadStatisticsManager::setMinimumTimeBetweeenDataRecordsRemoval):
(WebKit::WebResourceLoadStatisticsManager::setGrandfatheringTime):
(WebKit::WebResourceLoadStatisticsManager::resetToConsistentState):
    Functions for testing and configuration.
    WebResourceLoadStatisticsManager::setMinimumTimeBetweeenDataRecordsRemoval() changed
    as a result of WebResourceLoadStatisticsStore::setMinimumTimeBetweeenDataRecordsRemoval()
    moving to WebCore::ResourceLoadObserver::setMinimumTimeBetweeenDataRecordsRemoval().
* UIProcess/WebResourceLoadStatisticsManager.h:
* UIProcess/WebResourceLoadStatisticsStore.cpp:
(WebKit::initializeDataTypesToRemove):
(WebKit::WebResourceLoadStatisticsStore::removeDataRecords):
    Moved handling of pending removal into WebCore::ResourceLoadStatisticsStore since
    that's where grandfathering happens.
(WebKit::WebResourceLoadStatisticsStore::registerSharedResourceLoadObserver):
    Moved registration of write persistent store callback and reading of Cocoa defaults to
    WebResourceLoadStatisticsStore::registerSharedResourceLoadObserver without parameters
    so they are called for platforms without CFNETWORK_STORAGE_PARTITIONING.
    Now includes registering WebResourceLoadStatisticsStore::grandfatherExistingWebsiteData()
    as handler for grandfathering since it involves reading of the website data store.
(WebKit::WebResourceLoadStatisticsStore::grandfatherExistingWebsiteData):
(WebKit::WebResourceLoadStatisticsStore::readDataFromDiskIfNeeded):
(WebKit::WebResourceLoadStatisticsStore::setMinimumTimeBetweeenDataRecordsRemoval): Deleted.
    Now happens in WebCore::ResourceLoadObserver::setMinimumTimeBetweeenDataRecordsRemoval().
* UIProcess/WebResourceLoadStatisticsStore.h:
* UIProcess/WebsiteData/WebsiteDataRecord.cpp:
(WebKit::WebsiteDataRecord::topPrivatelyControlledDomain):
    New function to ask a WebsiteDataRecord for its top privately controlled domain.
* UIProcess/WebsiteData/WebsiteDataRecord.h:
* UIProcess/WebsiteData/WebsiteDataStore.cpp:
(WebKit::WebsiteDataStore::topPrivatelyControlledDomainsWithWebsiteData):
    New function to get all top privately controlled domains that
    have website data.
* UIProcess/WebsiteData/WebsiteDataStore.h:

Tools:

Adds test infrastructure needed for the added functionality.

* WebKitTestRunner/InjectedBundle/Bindings/TestRunner.idl:
* WebKitTestRunner/InjectedBundle/InjectedBundle.cpp:
(WTR::InjectedBundle::didReceiveMessageToPage):
* WebKitTestRunner/InjectedBundle/TestRunner.cpp:
(WTR::TestRunner::setStatisticsGrandfathered):
(WTR::TestRunner::isStatisticsGrandfathered):
(WTR::TestRunner::installStatisticsDidScanDataRecordsCallback):
(WTR::TestRunner::statisticsDidScanDataRecordsCallback):
(WTR::TestRunner::setStatisticsGrandfatheringTime):
* WebKitTestRunner/InjectedBundle/TestRunner.h:
* WebKitTestRunner/TestController.cpp:
(WTR::TestController::setStatisticsGrandfathered):
(WTR::TestController::isStatisticsGrandfathered):
(WTR::TestController::setStatisticsGrandfatheringTime):
* WebKitTestRunner/TestController.h:
* WebKitTestRunner/TestInvocation.cpp:
(WTR::TestInvocation::didReceiveSynchronousMessageFromInjectedBundle):

LayoutTests:

* http/tests/loading/resourceLoadStatistics/grandfathering-expected.txt: Added.
* http/tests/loading/resourceLoadStatistics/grandfathering.html: Added.
* platform/wk2/TestExpectations:
    Marked it Pass for WebKit2.


git-svn-id: http://svn.webkit.org/repository/webkit/trunk@217068 268f45cc-cd09-0410-ab3c-d52691b4dbfc
diff --git a/LayoutTests/ChangeLog b/LayoutTests/ChangeLog
index b7750d5..c1a1545 100644
--- a/LayoutTests/ChangeLog
+++ b/LayoutTests/ChangeLog
@@ -1,3 +1,16 @@
+2017-05-18  John Wilander  <wilander@apple.com>
+
+        Resource Load Statistics: Grandfather domains for existing data records
+        https://bugs.webkit.org/show_bug.cgi?id=172155
+        <rdar://problem/24913532>
+
+        Reviewed by Alex Christensen.
+
+        * http/tests/loading/resourceLoadStatistics/grandfathering-expected.txt: Added.
+        * http/tests/loading/resourceLoadStatistics/grandfathering.html: Added.
+        * platform/wk2/TestExpectations:
+            Marked it Pass for WebKit2.
+
 2017-05-18  Keith Miller  <keith_miller@apple.com>
 
         Fix wasm-mem-post-message.html test expectations