Change the decoding for some animated images to be asynchronous
https://bugs.webkit.org/show_bug.cgi?id=161566
Patch by Said Abou-Hallawa <sabouhallawa@apple.com> on 2016-11-09
Reviewed by Simon Fraser.
Source/WebCore:
Tests: fast/images/slower-animation-than-decoding-image.html
fast/images/slower-decoding-than-animation-image.html
fast/images/stopped-animation-deleted-image.html
Request the next frame before firing the animation timer. The asynchronous
image decoding work queue notifies the BitmapImage when the frame finishes
decoding. If the timer fires before the frame is decoded, no repaint will
be requested. Only when the image frame is ready, the animation will be
advanced and the image will be repainted.
* loader/cache/CachedImage.cpp:
(WebCore::CachedImage::load): Cache the image settings in CachedImage.
(WebCore::CachedImage::createImage): No need to pass allowSubsampling to BitmapImage. It can be retrieved through Image::imageObserver().
(WebCore::CachedImage::changedInRect): Change the parameter to notifyObservers() to be a pointer.
* loader/cache/CachedImage.h: Cache the settings: allowSubsampling, allowAsyncImageDecoding and showDebugBackground through m_loader.
* platform/graphics/BitmapImage.cpp:
(WebCore::BitmapImage::dataChanged): Fix a logging message.
(WebCore::BitmapImage::draw): Store the current SubsamplingLevel to be used when requesting decoding the image of the next frame.
Draw a debug rectangle if the next frame is missed because it is being decoded and the setting showDebugBackground is on.
(WebCore::BitmapImage::startAnimation): Deleted. Moved to the header file.
(WebCore::BitmapImage::internalStartAnimation): Added. Request asynchronous image decoding for the next frame if required. Return the
result of starting the animation.
(WebCore::BitmapImage::advanceAnimation): Call internalAdvanceAnimation() if the frame image is not being decoded. If it is being decoded
and the setting showDebugBackground is on, force repaint so the debug rectangle is drawn.
(WebCore::BitmapImage::internalAdvanceAnimation): This is the old body of advanceAnimation().
(WebCore::BitmapImage::stopAnimation): Stop the asynchronous image decoding if it is started.
(WebCore::BitmapImage::newFrameNativeImageAvailableAtIndex): This function is called from the async image decoding work queue when finishing decoding a native image frame.
* platform/graphics/BitmapImage.h:
(WebCore::BitmapImage::startAnimation): Added. It is now calls internalStartAnimation().
* platform/graphics/Color.h: Define a constant for the yellow color.
* platform/graphics/ImageFrameCache.cpp:
(WebCore::ImageFrameCache::clearMetadata): Delete unreferenced member.
(WebCore::ImageFrameCache::requestFrameAsyncDecodingAtIndex): Return true if the frame is requested for async decoding.
* platform/graphics/ImageFrameCache.h:
* platform/graphics/ImageObserver.h: Add virtual functions for allowSubsampling, allowAsyncImageDecoding and showDebugBackground.
* platform/graphics/ImageSource.cpp:
(WebCore::ImageSource::maximumSubsamplingLevel): Move checking allowSubsampling() to the caller BitmapImage::draw().
* platform/graphics/ImageSource.h: Remove the setting allowSubsampling(); it can be retrieved from imageObserver().
(WebCore::ImageSource::setAllowSubsampling): Deleted.
* rendering/RenderImageResource.cpp:
(WebCore::RenderImageResource::shutdown): Stop the animation of an image when shutting down the resource.
* rendering/RenderImageResourceStyleImage.cpp:
(WebCore::RenderImageResourceStyleImage::shutdown): Ditto.
svg/graphics/SVGImageClients.h: Change the parameter to ImageObserver::changedInRect() to be a pointer.
(WebCore::SVGImageChromeClient::invalidateContentsAndRootView):
* testing/Internals.cpp:
(WebCore::Internals::setImageFrameDecodingDuration): Sets a fixed frame decoding duration for testing.
* testing/Internals.h:
* testing/Internals.idl: Adds an internal option for ImageFrameDecodingDuration.
LayoutTests:
* fast/images/slower-animation-than-decoding-image-expected.txt: Added.
* fast/images/slower-animation-than-decoding-image.html: Added.
* fast/images/slower-decoding-than-animation-image-expected.txt: Added.
* fast/images/slower-decoding-than-animation-image.html: Added.
In these tests, CanvasRenderingContext2D.drawImage() is used to better
control advancing the animation of an animated image. A setTimeout() is
used instead of the frame duration to schedule when the drawing happens.
The first test ensures that faster decoding does not overrule the frame
duration; the setTimeout interval in this case. The second test ensures
the animation is not advanced unless decoding the next frame has finished.
* fast/images/stopped-animation-deleted-image-expected.txt: Added.
* fast/images/stopped-animation-deleted-image.html: Added.
This test ensures that if an animated image is removed from the document,
its draw() method won't be called even if the animation timer fires or the
decoding new frame availability notification is received.
git-svn-id: http://svn.webkit.org/repository/webkit/trunk@208511 268f45cc-cd09-0410-ab3c-d52691b4dbfc
diff --git a/LayoutTests/ChangeLog b/LayoutTests/ChangeLog
index 94ef6c8..93db0c1 100644
--- a/LayoutTests/ChangeLog
+++ b/LayoutTests/ChangeLog
@@ -1,3 +1,27 @@
+2016-11-09 Said Abou-Hallawa <sabouhallawa@apple.com>
+
+ Change the decoding for some animated images to be asynchronous
+ https://bugs.webkit.org/show_bug.cgi?id=161566
+
+ Reviewed by Simon Fraser.
+
+ * fast/images/slower-animation-than-decoding-image-expected.txt: Added.
+ * fast/images/slower-animation-than-decoding-image.html: Added.
+ * fast/images/slower-decoding-than-animation-image-expected.txt: Added.
+ * fast/images/slower-decoding-than-animation-image.html: Added.
+ In these tests, CanvasRenderingContext2D.drawImage() is used to better
+ control advancing the animation of an animated image. A setTimeout() is
+ used instead of the frame duration to schedule when the drawing happens.
+ The first test ensures that faster decoding does not overrule the frame
+ duration; the setTimeout interval in this case. The second test ensures
+ the animation is not advanced unless decoding the next frame has finished.
+
+ * fast/images/stopped-animation-deleted-image-expected.txt: Added.
+ * fast/images/stopped-animation-deleted-image.html: Added.
+ This test ensures that if an animated image is removed from the document,
+ its draw() method won't be called even if the animation timer fires or the
+ decoding new frame availability notification is received.
+
2016-11-04 Brent Fulgham <bfulgham@apple.com>
Local HTML should be blocked from localStorage access unless "Disable Local File Restrictions" is checked
diff --git a/LayoutTests/fast/images/slower-animation-than-decoding-image-expected.txt b/LayoutTests/fast/images/slower-animation-than-decoding-image-expected.txt
new file mode 100644
index 0000000..ebeac66
--- /dev/null
+++ b/LayoutTests/fast/images/slower-animation-than-decoding-image-expected.txt
@@ -0,0 +1,10 @@
+Ensure the image frame duration is respected even if the frame finishes decoding early.
+
+On success, you will see a series of "PASS" messages, followed by "TEST COMPLETE".
+
+
+PASS internals.imageFrameIndex(image) is 2
+PASS successfullyParsed is true
+
+TEST COMPLETE
+
diff --git a/LayoutTests/fast/images/slower-animation-than-decoding-image.html b/LayoutTests/fast/images/slower-animation-than-decoding-image.html
new file mode 100644
index 0000000..11c246d
--- /dev/null
+++ b/LayoutTests/fast/images/slower-animation-than-decoding-image.html
@@ -0,0 +1,53 @@
+<!DOCTYPE html>
+<html>
+<head>
+ <script src="../../resources/js-test-pre.js"></script>
+</head>
+<body>
+ <canvas id="canvas"></canvas>
+ <script>
+ description("Ensure the image frame duration is respected even if the frame finishes decoding early.");
+ jsTestIsAsync = true;
+
+ internals.clearMemoryCache();
+
+ var image = new Image;
+ image.onload = imageLoaded;
+ image.src = "resources/animated-red-green-blue.gif";
+
+ function imageLoaded()
+ {
+ if (!window.internals)
+ return;
+ internals.setImageFrameDecodingDuration(image, 0.040);
+ drawImage();
+ drawLoop();
+ }
+
+ function drawImage()
+ {
+ if (drawImage.count == undefined)
+ drawImage.count = 0;
+ var canvas = document.getElementById("canvas");
+ var ctx = canvas.getContext("2d");
+ ctx.drawImage(image, 0, 0, canvas.width, canvas.height);
+ return ++drawImage.count;
+ }
+
+ function drawLoop()
+ {
+ // 1st time the image is drawn, time = 0, current_frame = 0
+ // 2nd time the image is drawn, time = 50, current_frame = 1
+ // 3rd time the image is drawn, time = 100, current_frame = 2
+ setTimeout(function() {
+ if (drawImage() == 3) {
+ shouldBe("internals.imageFrameIndex(image)", "2");
+ finishJSTest();
+ } else
+ drawLoop();
+ }, 50);
+ }
+ </script>
+ <script src="../../resources/js-test-post.js"></script>
+</body>
+</html>
diff --git a/LayoutTests/fast/images/slower-decoding-than-animation-image-expected.txt b/LayoutTests/fast/images/slower-decoding-than-animation-image-expected.txt
new file mode 100644
index 0000000..ba4bae2
--- /dev/null
+++ b/LayoutTests/fast/images/slower-decoding-than-animation-image-expected.txt
@@ -0,0 +1,10 @@
+Ensure the image frame is drawn when it finishes decoding even if it takes more than the previous frame duration.
+
+On success, you will see a series of "PASS" messages, followed by "TEST COMPLETE".
+
+
+PASS internals.imageFrameIndex(image) is 1
+PASS successfullyParsed is true
+
+TEST COMPLETE
+
diff --git a/LayoutTests/fast/images/slower-decoding-than-animation-image.html b/LayoutTests/fast/images/slower-decoding-than-animation-image.html
new file mode 100644
index 0000000..ce6d9b8
--- /dev/null
+++ b/LayoutTests/fast/images/slower-decoding-than-animation-image.html
@@ -0,0 +1,53 @@
+<!DOCTYPE html>
+<html>
+<head>
+ <script src="../../resources/js-test-pre.js"></script>
+</head>
+<body>
+ <canvas id="canvas"></canvas>
+ <script>
+ description("Ensure the image frame is drawn when it finishes decoding even if it takes more than the previous frame duration.");
+ jsTestIsAsync = true;
+
+ internals.clearMemoryCache();
+
+ var image = new Image;
+ image.onload = imageLoaded;
+ image.src = "resources/animated-red-green-blue.gif";
+
+ function imageLoaded()
+ {
+ if (!window.internals)
+ return;
+ internals.setImageFrameDecodingDuration(image, 0.050);
+ drawImage();
+ drawLoop();
+ }
+
+ function drawImage()
+ {
+ if (drawImage.count == undefined)
+ drawImage.count = 0;
+ var canvas = document.getElementById("canvas");
+ var ctx = canvas.getContext("2d");
+ ctx.drawImage(image, 0, 0, canvas.width, canvas.height);
+ return ++drawImage.count;
+ }
+
+ function drawLoop()
+ {
+ // 1st time the image is drawn, time = 0, current_frame = 0
+ // 2nd time the image is drawn, time = 40, current_frame = 0
+ // 3rd time the image is drawn, time = 80, current_frame = 1
+ setTimeout(function() {
+ if (drawImage() == 3) {
+ shouldBe("internals.imageFrameIndex(image)", "1");
+ finishJSTest();
+ } else
+ drawLoop();
+ }, 40);
+ }
+ </script>
+ <script src="../../resources/js-test-post.js"></script>
+</body>
+</html>
diff --git a/LayoutTests/fast/images/stopped-animation-deleted-image-expected.txt b/LayoutTests/fast/images/stopped-animation-deleted-image-expected.txt
new file mode 100644
index 0000000..e377891
--- /dev/null
+++ b/LayoutTests/fast/images/stopped-animation-deleted-image-expected.txt
@@ -0,0 +1,10 @@
+Ensure the image stops animating if it is removed from the document before finishing decoding the current frame.
+
+On success, you will see a series of "PASS" messages, followed by "TEST COMPLETE".
+
+
+PASS internals.imageFrameIndex(image) is frameIndex
+PASS successfullyParsed is true
+
+TEST COMPLETE
+
diff --git a/LayoutTests/fast/images/stopped-animation-deleted-image.html b/LayoutTests/fast/images/stopped-animation-deleted-image.html
new file mode 100644
index 0000000..02a6dd4
--- /dev/null
+++ b/LayoutTests/fast/images/stopped-animation-deleted-image.html
@@ -0,0 +1,40 @@
+<!DOCTYPE html>
+<html>
+<head>
+ <script src="../../resources/js-test-pre.js"></script>
+</head>
+<body>
+ <img src="resources/animated-red-green-blue.gif" onload="imageLoaded()">
+ <script>
+ description("Ensure the image stops animating if it is removed from the document before finishing decoding the current frame.");
+ jsTestIsAsync = true;
+
+ internals.clearMemoryCache();
+
+ var image = document.getElementsByTagName("img")[0];
+ var loopCount = 0;
+ var frameIndex;
+
+ function imageLoaded()
+ {
+ if (!window.internals)
+ return;
+ internals.setImageFrameDecodingDuration(image, 0.040);
+ imageRemove();
+ }
+
+ function imageRemove()
+ {
+ setTimeout(function() {
+ frameIndex = internals.imageFrameIndex(image);
+ image.remove();
+ setTimeout(function() {
+ shouldBe("internals.imageFrameIndex(image)", "frameIndex");
+ finishJSTest();
+ }, 50);
+ }, 50);
+ }
+ </script>
+ <script src="../../resources/js-test-post.js"></script>
+</body>
+</html>
diff --git a/Source/WebCore/ChangeLog b/Source/WebCore/ChangeLog
index 5e72a58..bbdb28b 100644
--- a/Source/WebCore/ChangeLog
+++ b/Source/WebCore/ChangeLog
@@ -1,3 +1,60 @@
+2016-11-09 Said Abou-Hallawa <sabouhallawa@apple.com>
+
+ Change the decoding for some animated images to be asynchronous
+ https://bugs.webkit.org/show_bug.cgi?id=161566
+
+ Reviewed by Simon Fraser.
+
+ Tests: fast/images/slower-animation-than-decoding-image.html
+ fast/images/slower-decoding-than-animation-image.html
+ fast/images/stopped-animation-deleted-image.html
+
+ Request the next frame before firing the animation timer. The asynchronous
+ image decoding work queue notifies the BitmapImage when the frame finishes
+ decoding. If the timer fires before the frame is decoded, no repaint will
+ be requested. Only when the image frame is ready, the animation will be
+ advanced and the image will be repainted.
+
+ * loader/cache/CachedImage.cpp:
+ (WebCore::CachedImage::load): Cache the image settings in CachedImage.
+ (WebCore::CachedImage::createImage): No need to pass allowSubsampling to BitmapImage. It can be retrieved through Image::imageObserver().
+ (WebCore::CachedImage::changedInRect): Change the parameter to notifyObservers() to be a pointer.
+ * loader/cache/CachedImage.h: Cache the settings: allowSubsampling, allowAsyncImageDecoding and showDebugBackground through m_loader.
+ * platform/graphics/BitmapImage.cpp:
+ (WebCore::BitmapImage::dataChanged): Fix a logging message.
+ (WebCore::BitmapImage::draw): Store the current SubsamplingLevel to be used when requesting decoding the image of the next frame.
+ Draw a debug rectangle if the next frame is missed because it is being decoded and the setting showDebugBackground is on.
+ (WebCore::BitmapImage::startAnimation): Deleted. Moved to the header file.
+ (WebCore::BitmapImage::internalStartAnimation): Added. Request asynchronous image decoding for the next frame if required. Return the
+ result of starting the animation.
+ (WebCore::BitmapImage::advanceAnimation): Call internalAdvanceAnimation() if the frame image is not being decoded. If it is being decoded
+ and the setting showDebugBackground is on, force repaint so the debug rectangle is drawn.
+ (WebCore::BitmapImage::internalAdvanceAnimation): This is the old body of advanceAnimation().
+ (WebCore::BitmapImage::stopAnimation): Stop the asynchronous image decoding if it is started.
+ (WebCore::BitmapImage::newFrameNativeImageAvailableAtIndex): This function is called from the async image decoding work queue when finishing decoding a native image frame.
+ * platform/graphics/BitmapImage.h:
+ (WebCore::BitmapImage::startAnimation): Added. It is now calls internalStartAnimation().
+ * platform/graphics/Color.h: Define a constant for the yellow color.
+ * platform/graphics/ImageFrameCache.cpp:
+ (WebCore::ImageFrameCache::clearMetadata): Delete unreferenced member.
+ (WebCore::ImageFrameCache::requestFrameAsyncDecodingAtIndex): Return true if the frame is requested for async decoding.
+ * platform/graphics/ImageFrameCache.h:
+ * platform/graphics/ImageObserver.h: Add virtual functions for allowSubsampling, allowAsyncImageDecoding and showDebugBackground.
+ * platform/graphics/ImageSource.cpp:
+ (WebCore::ImageSource::maximumSubsamplingLevel): Move checking allowSubsampling() to the caller BitmapImage::draw().
+ * platform/graphics/ImageSource.h: Remove the setting allowSubsampling(); it can be retrieved from imageObserver().
+ (WebCore::ImageSource::setAllowSubsampling): Deleted.
+ * rendering/RenderImageResource.cpp:
+ (WebCore::RenderImageResource::shutdown): Stop the animation of an image when shutting down the resource.
+ * rendering/RenderImageResourceStyleImage.cpp:
+ (WebCore::RenderImageResourceStyleImage::shutdown): Ditto.
+ svg/graphics/SVGImageClients.h: Change the parameter to ImageObserver::changedInRect() to be a pointer.
+ (WebCore::SVGImageChromeClient::invalidateContentsAndRootView):
+ * testing/Internals.cpp:
+ (WebCore::Internals::setImageFrameDecodingDuration): Sets a fixed frame decoding duration for testing.
+ * testing/Internals.h:
+ * testing/Internals.idl: Adds an internal option for ImageFrameDecodingDuration.
+
2016-11-04 Brent Fulgham <bfulgham@apple.com>
Local HTML should be blocked from localStorage access unless "Disable Local File Restrictions" is checked
diff --git a/Source/WebCore/loader/cache/CachedImage.cpp b/Source/WebCore/loader/cache/CachedImage.cpp
index 1ddefe3..2939e4d 100644
--- a/Source/WebCore/loader/cache/CachedImage.cpp
+++ b/Source/WebCore/loader/cache/CachedImage.cpp
@@ -89,6 +89,12 @@
CachedResource::load(loader);
else
setLoading(false);
+
+ if (m_loader) {
+ m_allowSubsampling = m_loader->frameLoader()->frame().settings().imageSubsamplingEnabled();
+ m_allowAsyncImageDecoding = m_loader->frameLoader()->frame().settings().asyncImageDecodingEnabled();
+ m_showDebugBackground = m_loader->frameLoader()->frame().settings().showDebugBorders();
+ }
}
void CachedImage::setBodyDataFrom(const CachedResource& resource)
@@ -325,10 +331,8 @@
auto svgImage = SVGImage::create(*this, url());
m_svgImageCache = std::make_unique<SVGImageCache>(svgImage.ptr());
m_image = WTFMove(svgImage);
- } else {
+ } else
m_image = BitmapImage::create(this);
- downcast<BitmapImage>(*m_image).setAllowSubsampling(m_loader && m_loader->frameLoader()->frame().settings().imageSubsamplingEnabled());
- }
if (m_image) {
// Send queued container size requests.
@@ -481,11 +485,11 @@
client->newImageAnimationFrameAvailable(*this);
}
-void CachedImage::changedInRect(const Image* image, const IntRect& rect)
+void CachedImage::changedInRect(const Image* image, const IntRect* rect)
{
if (!image || image != m_image)
return;
- notifyObservers(&rect);
+ notifyObservers(rect);
}
bool CachedImage::currentFrameKnownToBeOpaque(const RenderElement* renderer)
diff --git a/Source/WebCore/loader/cache/CachedImage.h b/Source/WebCore/loader/cache/CachedImage.h
index f793bc9..6347750 100644
--- a/Source/WebCore/loader/cache/CachedImage.h
+++ b/Source/WebCore/loader/cache/CachedImage.h
@@ -118,11 +118,14 @@
bool stillNeedsLoad() const override { return !errorOccurred() && status() == Unknown && !isLoading(); }
// ImageObserver
+ bool allowSubsampling() const override { return m_allowSubsampling; }
+ bool allowAsyncImageDecoding() const override { return m_allowAsyncImageDecoding; }
+ bool showDebugBackground() const override { return m_showDebugBackground; }
void decodedSizeChanged(const Image*, long long delta) override;
void didDraw(const Image*) override;
void animationAdvanced(const Image*) override;
- void changedInRect(const Image*, const IntRect&) override;
+ void changedInRect(const Image*, const IntRect* changeRect = nullptr) override;
void addIncrementalDataBuffer(SharedBuffer&);
@@ -136,6 +139,15 @@
std::unique_ptr<SVGImageCache> m_svgImageCache;
bool m_isManuallyCached { false };
bool m_shouldPaintBrokenImage { true };
+
+ // The default value of m_allowSubsampling should be the same as defaultImageSubsamplingEnabled in Settings.cpp
+#if PLATFORM(IOS)
+ bool m_allowSubsampling { true };
+#else
+ bool m_allowSubsampling { false };
+#endif
+ bool m_allowAsyncImageDecoding { true };
+ bool m_showDebugBackground { false };
};
} // namespace WebCore
diff --git a/Source/WebCore/platform/graphics/BitmapImage.cpp b/Source/WebCore/platform/graphics/BitmapImage.cpp
index 1fa0244..3dba562 100644
--- a/Source/WebCore/platform/graphics/BitmapImage.cpp
+++ b/Source/WebCore/platform/graphics/BitmapImage.cpp
@@ -83,7 +83,7 @@
NativeImagePtr BitmapImage::frameImageAtIndex(size_t index, SubsamplingLevel subsamplingLevel, const GraphicsContext* targetContext)
{
if (!frameHasValidNativeImageAtIndex(index, subsamplingLevel)) {
- LOG(Images, "BitmapImage %p frameImageAtIndex - subsamplingLevel was %d, resampling", this, static_cast<int>(frameSubsamplingLevelAtIndex(index)));
+ LOG(Images, "BitmapImage %p %s - subsamplingLevel was %d, resampling", this, __FUNCTION__, static_cast<int>(frameSubsamplingLevelAtIndex(index)));
invalidatePlatformData();
}
@@ -141,19 +141,24 @@
if (destRect.isEmpty() || srcRect.isEmpty())
return;
- startAnimation();
+ StartAnimationResult result = internalStartAnimation();
- Color color = singlePixelSolidColor();
+ Color color;
+ if (result == StartAnimationResult::DecodingActive && showDebugBackground())
+ color = Color::yellow;
+ else
+ color = singlePixelSolidColor();
+
if (color.isValid()) {
fillWithSolidColor(context, destRect, color, op);
return;
}
float scale = subsamplingScale(context, destRect, srcRect);
- SubsamplingLevel subsamplingLevel = m_source.subsamplingLevelForScale(scale);
- LOG(Images, "BitmapImage %p draw - subsamplingLevel %d at scale %.4f", this, static_cast<int>(subsamplingLevel), scale);
+ m_currentSubsamplingLevel = allowSubsampling() ? m_source.subsamplingLevelForScale(scale) : SubsamplingLevel::Default;
+ LOG(Images, "BitmapImage %p draw - subsamplingLevel %d at scale %.4f", this, static_cast<int>(m_currentSubsamplingLevel), scale);
- auto image = frameImageAtIndex(m_currentFrame, subsamplingLevel, &context);
+ auto image = frameImageAtIndex(m_currentFrame, m_currentSubsamplingLevel, &context);
if (!image) // If it's too early we won't have an image yet.
return;
@@ -223,10 +228,18 @@
m_frameTimer->startOneShot(delay);
}
-void BitmapImage::startAnimation()
+BitmapImage::StartAnimationResult BitmapImage::internalStartAnimation()
{
- if (m_frameTimer || !shouldAnimate() || frameCount() <= 1)
- return;
+ if (!canAnimate())
+ return StartAnimationResult::CannotStart;
+
+ if (m_frameTimer)
+ return StartAnimationResult::TimerActive;
+
+ // Don't start a new animation until we draw the frame that is currently being decoded.
+ size_t nextFrame = (m_currentFrame + 1) % frameCount();
+ if (frameIsBeingDecodedAtIndex(nextFrame))
+ return StartAnimationResult::DecodingActive;
if (m_currentFrame >= frameCount() - 1) {
// Don't advance past the last frame if we haven't decoded the whole image
@@ -234,7 +247,7 @@
// in a GIF can potentially come after all the rest of the image data, so
// wait on it.
if (!m_source.isAllDataReceived() && repetitionCount() == RepetitionCountOnce)
- return;
+ return StartAnimationResult::IncompleteData;
++m_repetitionsComplete;
@@ -242,16 +255,15 @@
if (repetitionCount() != RepetitionCountInfinite && m_repetitionsComplete > repetitionCount()) {
m_animationFinished = true;
destroyDecodedDataIfNecessary(false);
- return;
+ return StartAnimationResult::CannotStart;
}
destroyDecodedDataIfNecessary(true);
}
// Don't advance the animation to an incomplete frame.
- size_t nextFrame = (m_currentFrame + 1) % frameCount();
if (!m_source.isAllDataReceived() && !frameIsCompleteAtIndex(nextFrame))
- return;
+ return StartAnimationResult::IncompleteData;
double time = monotonicallyIncreasingTime();
@@ -262,15 +274,53 @@
// Setting 'm_desiredFrameStartTime' to 'time' means we are late; otherwise we are early.
m_desiredFrameStartTime = std::max(time, m_desiredFrameStartTime + frameDurationAtIndex(m_currentFrame));
+ // Request async decoding for nextFrame only if this is required. If nextFrame is not in the frameCache,
+ // it will be decoded on a separate work queue. When decoding nextFrame finishes, we will be notified
+ // through the callback newFrameNativeImageAvailableAtIndex(). Otherwise, advanceAnimation() will be called
+ // when the timer fires and m_currentFrame will be advanced to nextFrame since it is not being decoded.
+ if ((allowAsyncImageDecoding() && m_source.isAsyncDecodingRequired()) || isAsyncDecodingForcedForTesting()) {
+ if (!m_source.requestFrameAsyncDecodingAtIndex(nextFrame, m_currentSubsamplingLevel))
+ LOG(Images, "BitmapImage %p %s - cachedFrameCount %ld nextFrame %ld", this, __FUNCTION__, ++m_cachedFrameCount, nextFrame);
+ m_desiredFrameDecodeTimeForTesting = time + std::max(m_frameDecodingDurationForTesting, 0.0f);
+ }
+
ASSERT(!m_frameTimer);
startTimer(m_desiredFrameStartTime - time);
+ return StartAnimationResult::Started;
}
void BitmapImage::advanceAnimation()
{
clearTimer();
+ // Pretend as if decoding nextFrame has taken m_frameDecodingDurationForTesting from
+ // the time this decoding was requested.
+ if (isAsyncDecodingForcedForTesting()) {
+ double time = monotonicallyIncreasingTime();
+ // Start a timer with the remaining time from now till the m_desiredFrameDecodeTime.
+ if (m_desiredFrameDecodeTimeForTesting > std::max(time, m_desiredFrameStartTime)) {
+ startTimer(m_desiredFrameDecodeTimeForTesting - time);
+ return;
+ }
+ }
+
+ // Don't advance to nextFrame unless its decoding has finished or was not required.
+ size_t nextFrame = (m_currentFrame + 1) % frameCount();
+ if (!frameIsBeingDecodedAtIndex(nextFrame))
+ internalAdvanceAnimation();
+ else {
+ // Force repaint if showDebugBackground() is on.
+ if (showDebugBackground())
+ imageObserver()->changedInRect(this);
+ LOG(Images, "BitmapImage %p %s - lateFrameCount %ld nextFrame %ld", this, __FUNCTION__, ++m_lateFrameCount, nextFrame);
+ }
+}
+
+void BitmapImage::internalAdvanceAnimation()
+{
m_currentFrame = (m_currentFrame + 1) % frameCount();
+ ASSERT(!frameIsBeingDecodedAtIndex(m_currentFrame));
+
destroyDecodedDataIfNecessary(false);
if (imageObserver())
@@ -282,6 +332,7 @@
// This timer is used to animate all occurrences of this image. Don't invalidate
// the timer unless all renderers have stopped drawing.
clearTimer();
+ m_source.stopAsyncDecodingQueue();
}
void BitmapImage::resetAnimation()
@@ -296,6 +347,18 @@
destroyDecodedDataIfNecessary(true);
}
+void BitmapImage::newFrameNativeImageAvailableAtIndex(size_t index)
+{
+ UNUSED_PARAM(index);
+ ASSERT(index == (m_currentFrame + 1) % frameCount());
+
+ // Don't advance to nextFrame unless the timer was fired before its decoding finishes.
+ if (canAnimate() && !m_frameTimer)
+ internalAdvanceAnimation();
+ else
+ LOG(Images, "BitmapImage %p %s - earlyFrameCount %ld nextFrame %ld", this, __FUNCTION__, ++m_earlyFrameCount, index);
+}
+
void BitmapImage::dump(TextStream& ts) const
{
Image::dump(ts);
diff --git a/Source/WebCore/platform/graphics/BitmapImage.h b/Source/WebCore/platform/graphics/BitmapImage.h
index 369a8aa..89032dc 100644
--- a/Source/WebCore/platform/graphics/BitmapImage.h
+++ b/Source/WebCore/platform/graphics/BitmapImage.h
@@ -29,6 +29,7 @@
#include "Image.h"
#include "Color.h"
+#include "ImageObserver.h"
#include "ImageOrientation.h"
#include "ImageSource.h"
#include "IntSize.h"
@@ -80,8 +81,6 @@
IntSize sizeRespectingOrientation() const { return m_source.sizeRespectingOrientation(); }
Color singlePixelSolidColor() const override { return m_source.singlePixelSolidColor(); }
- void setAllowSubsampling(bool allowSubsampling) { m_source.setAllowSubsampling(allowSubsampling); }
-
bool frameIsBeingDecodedAtIndex(size_t index) const { return m_source.frameIsBeingDecodedAtIndex(index); }
bool frameIsCompleteAtIndex(size_t index) const { return m_source.frameIsCompleteAtIndex(index); }
bool frameHasAlphaAtIndex(size_t index) const { return m_source.frameHasAlphaAtIndex(index); }
@@ -95,6 +94,9 @@
bool currentFrameKnownToBeOpaque() const override { return !frameHasAlphaAtIndex(currentFrame()); }
ImageOrientation orientationForCurrentFrame() const override { return frameOrientationAtIndex(currentFrame()); }
+ bool isAsyncDecodingForcedForTesting() const { return m_frameDecodingDurationForTesting > 0; }
+ void setFrameDecodingDurationForTesting(float duration) { m_frameDecodingDurationForTesting = duration; }
+
// Accessors for native image formats.
#if USE(APPKIT)
NSImage *nsImage() override;
@@ -131,6 +133,10 @@
NativeImagePtr frameImageAtIndex(size_t, SubsamplingLevel = SubsamplingLevel::Default, const GraphicsContext* = nullptr);
+ bool allowSubsampling() const { return imageObserver() && imageObserver()->allowSubsampling(); }
+ bool allowAsyncImageDecoding() const { return imageObserver() && imageObserver()->allowAsyncImageDecoding(); }
+ bool showDebugBackground() const { return imageObserver() && imageObserver()->showDebugBackground(); }
+
// Called to invalidate cached data. When |destroyAll| is true, we wipe out
// the entire frame buffer cache and tell the image source to destroy
// everything; this is used when e.g. we want to free some room in the image
@@ -150,17 +156,21 @@
#endif
// Animation.
+ enum class StartAnimationResult { CannotStart, IncompleteData, TimerActive, DecodingActive, Started };
bool isAnimated() const override { return m_source.frameCount() > 1; }
bool shouldAnimate();
bool canAnimate();
- void startAnimation() override;
+ void startAnimation() override { internalStartAnimation(); }
+ StartAnimationResult internalStartAnimation();
void advanceAnimation();
+ void internalAdvanceAnimation();
// It may look unusual that there is no start animation call as public API. This is because
// we start and stop animating lazily. Animation begins whenever someone draws the image. It will
// automatically pause once all observers no longer want to render the image anywhere.
void stopAnimation() override;
void resetAnimation() override;
+ void newFrameNativeImageAvailableAtIndex(size_t) override;
// Handle platform-specific data
void invalidatePlatformData();
@@ -182,11 +192,20 @@
mutable ImageSource m_source;
size_t m_currentFrame { 0 }; // The index of the current frame of animation.
+ SubsamplingLevel m_currentSubsamplingLevel { SubsamplingLevel::Default };
std::unique_ptr<Timer> m_frameTimer;
RepetitionCount m_repetitionsComplete { RepetitionCountNone }; // How many repetitions we've finished.
double m_desiredFrameStartTime { 0 }; // The system time at which we hope to see the next call to startAnimation().
bool m_animationFinished { false };
+ float m_frameDecodingDurationForTesting { 0 };
+ double m_desiredFrameDecodeTimeForTesting { 0 };
+#if !LOG_DISABLED
+ size_t m_lateFrameCount { 0 };
+ size_t m_earlyFrameCount { 0 };
+ size_t m_cachedFrameCount { 0 };
+#endif
+
#if USE(APPKIT)
mutable RetainPtr<NSImage> m_nsImage; // A cached NSImage of all the frames. Only built lazily if someone actually queries for one.
#endif
diff --git a/Source/WebCore/platform/graphics/Color.h b/Source/WebCore/platform/graphics/Color.h
index 97f95ed..e994ba9 100644
--- a/Source/WebCore/platform/graphics/Color.h
+++ b/Source/WebCore/platform/graphics/Color.h
@@ -264,6 +264,7 @@
static const RGBA32 lightGray = 0xFFC0C0C0;
WEBCORE_EXPORT static const RGBA32 transparent = 0x00000000;
static const RGBA32 cyan = 0xFF00FFFF;
+ static const RGBA32 yellow = 0xFFFFFF00;
#if PLATFORM(IOS)
static const RGBA32 compositionFill = 0x3CAFC0E3;
diff --git a/Source/WebCore/platform/graphics/ImageFrameCache.cpp b/Source/WebCore/platform/graphics/ImageFrameCache.cpp
index e639909..1c418e2 100644
--- a/Source/WebCore/platform/graphics/ImageFrameCache.cpp
+++ b/Source/WebCore/platform/graphics/ImageFrameCache.cpp
@@ -284,10 +284,10 @@
});
}
-void ImageFrameCache::requestFrameAsyncDecodingAtIndex(size_t index, SubsamplingLevel subsamplingLevel)
+bool ImageFrameCache::requestFrameAsyncDecodingAtIndex(size_t index, SubsamplingLevel subsamplingLevel)
{
if (!isDecoderAvailable())
- return;
+ return false;
if (!hasDecodingQueue())
startAsyncDecodingQueue();
@@ -299,10 +299,11 @@
subsamplingLevel = frame.subsamplingLevel();
if (frame.hasValidNativeImage(subsamplingLevel))
- return;
+ return false;
frame.setDecoding(ImageFrame::Decoding::BeingDecoded);
m_frameRequestQueue.enqueue({ index, subsamplingLevel });
+ return true;
}
void ImageFrameCache::stopAsyncDecodingQueue()
@@ -336,7 +337,6 @@
{
m_frameCount = Nullopt;
m_singlePixelSolidColor = Nullopt;
- m_maximumSubsamplingLevel = Nullopt;
}
template<typename T, T (ImageDecoder::*functor)() const>
diff --git a/Source/WebCore/platform/graphics/ImageFrameCache.h b/Source/WebCore/platform/graphics/ImageFrameCache.h
index 2e30aee..6f905a1 100644
--- a/Source/WebCore/platform/graphics/ImageFrameCache.h
+++ b/Source/WebCore/platform/graphics/ImageFrameCache.h
@@ -68,7 +68,7 @@
// Asynchronous image decoding
void startAsyncDecodingQueue();
- void requestFrameAsyncDecodingAtIndex(size_t, SubsamplingLevel);
+ bool requestFrameAsyncDecodingAtIndex(size_t, SubsamplingLevel);
void stopAsyncDecodingQueue();
bool hasDecodingQueue() { return m_decodingQueue; }
@@ -162,7 +162,6 @@
// Image metadata which is calculated from the first ImageFrame.
Optional<IntSize> m_size;
Optional<IntSize> m_sizeRespectingOrientation;
- Optional<SubsamplingLevel> m_maximumSubsamplingLevel;
Optional<Color> m_singlePixelSolidColor;
};
diff --git a/Source/WebCore/platform/graphics/ImageObserver.h b/Source/WebCore/platform/graphics/ImageObserver.h
index 8fa5983..55317da 100644
--- a/Source/WebCore/platform/graphics/ImageObserver.h
+++ b/Source/WebCore/platform/graphics/ImageObserver.h
@@ -37,12 +37,16 @@
protected:
virtual ~ImageObserver() {}
public:
+ virtual bool allowSubsampling() const = 0;
+ virtual bool allowAsyncImageDecoding() const = 0;
+ virtual bool showDebugBackground() const = 0;
virtual void decodedSizeChanged(const Image*, long long delta) = 0;
+
virtual void didDraw(const Image*) = 0;
virtual void animationAdvanced(const Image*) = 0;
- virtual void changedInRect(const Image*, const IntRect&) = 0;
+ virtual void changedInRect(const Image*, const IntRect* changeRect = nullptr) = 0;
};
}
diff --git a/Source/WebCore/platform/graphics/ImageSource.cpp b/Source/WebCore/platform/graphics/ImageSource.cpp
index 3adc763..910d61d 100644
--- a/Source/WebCore/platform/graphics/ImageSource.cpp
+++ b/Source/WebCore/platform/graphics/ImageSource.cpp
@@ -186,7 +186,7 @@
if (m_maximumSubsamplingLevel)
return m_maximumSubsamplingLevel.value();
- if (!m_allowSubsampling || !isDecoderAvailable() || !m_decoder->frameAllowSubsamplingAtIndex(0))
+ if (!isDecoderAvailable() || !m_decoder->frameAllowSubsamplingAtIndex(0))
return SubsamplingLevel::Default;
// FIXME: this value was chosen to be appropriate for iOS since the image
diff --git a/Source/WebCore/platform/graphics/ImageSource.h b/Source/WebCore/platform/graphics/ImageSource.h
index c920b25..3e24126 100644
--- a/Source/WebCore/platform/graphics/ImageSource.h
+++ b/Source/WebCore/platform/graphics/ImageSource.h
@@ -66,7 +66,7 @@
bool isAllDataReceived();
bool isAsyncDecodingRequired();
- void requestFrameAsyncDecodingAtIndex(size_t index, SubsamplingLevel subsamplingLevel) { m_frameCache->requestFrameAsyncDecodingAtIndex(index, subsamplingLevel); }
+ bool requestFrameAsyncDecodingAtIndex(size_t index, SubsamplingLevel subsamplingLevel) { return m_frameCache->requestFrameAsyncDecodingAtIndex(index, subsamplingLevel); }
void stopAsyncDecodingQueue() { m_frameCache->stopAsyncDecodingQueue(); }
// Image metadata which is calculated by the decoder or can deduced by the case of the memory NativeImage.
@@ -98,7 +98,6 @@
SubsamplingLevel maximumSubsamplingLevel();
SubsamplingLevel subsamplingLevelForScale(float);
- void setAllowSubsampling(bool allowSubsampling) { m_allowSubsampling = allowSubsampling; }
NativeImagePtr createFrameImageAtIndex(size_t, SubsamplingLevel = SubsamplingLevel::Default);
private:
@@ -113,13 +112,6 @@
Optional<SubsamplingLevel> m_maximumSubsamplingLevel;
- // The default value of m_allowSubsampling should be the same as defaultImageSubsamplingEnabled in Settings.cpp
-#if PLATFORM(IOS)
- bool m_allowSubsampling { true };
-#else
- bool m_allowSubsampling { false };
-#endif
-
#if PLATFORM(IOS)
// FIXME: We should expose a setting to enable/disable progressive loading so that we can remove the PLATFORM(IOS)-guard.
double m_progressiveLoadChunkTime { 0 };
diff --git a/Source/WebCore/rendering/RenderImageResource.cpp b/Source/WebCore/rendering/RenderImageResource.cpp
index 0c60052..b94f159 100644
--- a/Source/WebCore/rendering/RenderImageResource.cpp
+++ b/Source/WebCore/rendering/RenderImageResource.cpp
@@ -56,8 +56,10 @@
{
ASSERT(m_renderer);
- if (m_cachedImage)
+ if (m_cachedImage) {
+ image()->stopAnimation();
m_cachedImage->removeClient(*m_renderer);
+ }
}
void RenderImageResource::setCachedImage(CachedImage* newImage)
diff --git a/Source/WebCore/rendering/RenderImageResourceStyleImage.cpp b/Source/WebCore/rendering/RenderImageResourceStyleImage.cpp
index 4046d54..28e36b1 100644
--- a/Source/WebCore/rendering/RenderImageResourceStyleImage.cpp
+++ b/Source/WebCore/rendering/RenderImageResourceStyleImage.cpp
@@ -57,7 +57,10 @@
{
ASSERT(m_renderer);
m_styleImage->removeClient(m_renderer);
- m_cachedImage = nullptr;
+ if (m_cachedImage) {
+ image()->stopAnimation();
+ m_cachedImage = nullptr;
+ }
}
RefPtr<Image> RenderImageResourceStyleImage::image(int width, int height) const
diff --git a/Source/WebCore/svg/graphics/SVGImageClients.h b/Source/WebCore/svg/graphics/SVGImageClients.h
index fe6f2e4..5dc4041 100644
--- a/Source/WebCore/svg/graphics/SVGImageClients.h
+++ b/Source/WebCore/svg/graphics/SVGImageClients.h
@@ -54,7 +54,7 @@
{
// If m_image->m_page is null, we're being destructed, don't fire changedInRect() in that case.
if (m_image && m_image->imageObserver() && m_image->m_page)
- m_image->imageObserver()->changedInRect(m_image, r);
+ m_image->imageObserver()->changedInRect(m_image, &r);
}
SVGImage* m_image;
diff --git a/Source/WebCore/testing/Internals.cpp b/Source/WebCore/testing/Internals.cpp
index e899f93..35d971c 100644
--- a/Source/WebCore/testing/Internals.cpp
+++ b/Source/WebCore/testing/Internals.cpp
@@ -649,6 +649,19 @@
return is<BitmapImage>(image) ? downcast<BitmapImage>(*image).currentFrame() : 0;
}
+void Internals::setImageFrameDecodingDuration(HTMLImageElement& element, float duration)
+{
+ auto* cachedImage = element.cachedImage();
+ if (!cachedImage)
+ return;
+
+ auto* image = cachedImage->image();
+ if (!is<BitmapImage>(image))
+ return;
+
+ downcast<BitmapImage>(*image).setFrameDecodingDurationForTesting(duration);
+}
+
void Internals::clearPageCache()
{
PageCache::singleton().pruneToSizeNow(0, PruningReason::None);
diff --git a/Source/WebCore/testing/Internals.h b/Source/WebCore/testing/Internals.h
index 4d20f9f..9421cd4 100644
--- a/Source/WebCore/testing/Internals.h
+++ b/Source/WebCore/testing/Internals.h
@@ -103,6 +103,7 @@
unsigned memoryCacheSize() const;
unsigned imageFrameIndex(HTMLImageElement&);
+ void setImageFrameDecodingDuration(HTMLImageElement&, float duration);
void clearPageCache();
unsigned pageCacheSize() const;
diff --git a/Source/WebCore/testing/Internals.idl b/Source/WebCore/testing/Internals.idl
index 89612d2..266b91e 100644
--- a/Source/WebCore/testing/Internals.idl
+++ b/Source/WebCore/testing/Internals.idl
@@ -221,6 +221,7 @@
[MayThrowException] boolean isPageBoxVisible(long pageNumber);
unsigned long imageFrameIndex(HTMLImageElement element);
+ void setImageFrameDecodingDuration(HTMLImageElement element, unrestricted float duration);
readonly attribute InternalSettings settings;
readonly attribute unsigned long workerThreadCount;
