| commit | 57ab0f4d46b63593a17a86f0b67979abab293625 | [log] [tgz] |
|---|---|---|
| author | jer.noble@apple.com <jer.noble@apple.com@268f45cc-cd09-0410-ab3c-d52691b4dbfc> | Sat May 20 16:55:01 2017 +0000 |
| committer | jer.noble@apple.com <jer.noble@apple.com@268f45cc-cd09-0410-ab3c-d52691b4dbfc> | Sat May 20 16:55:01 2017 +0000 |
| tree | ca41e0175f1a7765c2257ad80cf1ec970a7f26c0 | |
| parent | 3196d094f4ba9c97d2b8d8033f080eb533a40435 [diff] |
[MSE][Mac] Support painting MSE video-element to canvas https://bugs.webkit.org/show_bug.cgi?id=125157 <rdar://problem/23062016> Reviewed by Eric Carlson. Source/WebCore: Test: media/media-source/media-source-paint-to-canvas.html In order to have access to decoded video data for painting, decode the encoded samples manually instead of adding them to the AVSampleBufferDisplayLayer. To facilitate doing so, add a new utility class WebCoreDecompressionSession, which can decode samples and store them. For the purposes of this patch, to avoid double-decoding of video data and to avoid severe complication of our sample delivery pipeline, we will only support painting of decoded video samples when the video is not displayed in the DOM. * Modules/mediasource/MediaSource.cpp: (WebCore::MediaSource::seekToTime): Always send waitForSeekCompleted() to give private a chance to delay seek completion. * Modules/mediasource/SourceBuffer.cpp: (WebCore::SourceBuffer::sourceBufferPrivateReenqueSamples): Added. * Modules/mediasource/SourceBuffer.h: * WebCore.xcodeproj/project.pbxproj: * platform/cf/CoreMediaSoftLink.cpp: Added new soft link macros. * platform/cf/CoreMediaSoftLink.h: Ditto. * platform/cocoa/CoreVideoSoftLink.cpp: Ditto. * platform/cocoa/CoreVideoSoftLink.h: Ditto. * platform/graphics/SourceBufferPrivateClient.h: * platform/graphics/avfoundation/objc/MediaPlayerPrivateMediaSourceAVFObjC.h: (WebCore::MediaPlayerPrivateMediaSourceAVFObjC::sampleBufferDisplayLayer): Simple accessor. (WebCore::MediaPlayerPrivateMediaSourceAVFObjC::decompressionSession): Ditto. * platform/graphics/avfoundation/objc/MediaPlayerPrivateMediaSourceAVFObjC.mm: (WebCore::MediaPlayerPrivateMediaSourceAVFObjC::MediaPlayerPrivateMediaSourceAVFObjC): (WebCore::MediaPlayerPrivateMediaSourceAVFObjC::load): Update whether we should be displaying in a layer or decompression session.. (WebCore::MediaPlayerPrivateMediaSourceAVFObjC::setVisible): Ditto. (WebCore::MediaPlayerPrivateMediaSourceAVFObjC::waitForSeekCompleted): m_seeking is now an enum. (WebCore::MediaPlayerPrivateMediaSourceAVFObjC::seeking): Ditto. (WebCore::MediaPlayerPrivateMediaSourceAVFObjC::seekCompleted): Ditto. If waiting for a video frame, delay completing seek. (WebCore::MediaPlayerPrivateMediaSourceAVFObjC::nativeImageForCurrentTime): Call updateLastImage() and return result. (WebCore::MediaPlayerPrivateMediaSourceAVFObjC::updateLastImage): Fetch the image for the current time. (WebCore::MediaPlayerPrivateMediaSourceAVFObjC::paint): Pass to paintCurrentFrameInCanvas. (WebCore::MediaPlayerPrivateMediaSourceAVFObjC::paintCurrentFrameInContext): Get a native image, and render it. (WebCore::MediaPlayerPrivateMediaSourceAVFObjC::acceleratedRenderingStateChanged): Create or destroy a layer or decompression session as appropriate. (WebCore::MediaPlayerPrivateMediaSourceAVFObjC::ensureLayer): Creates a layer. (WebCore::MediaPlayerPrivateMediaSourceAVFObjC::destroyLayer): Destroys a layer. (WebCore::MediaPlayerPrivateMediaSourceAVFObjC::ensureDecompressionSession): Creates a decompression session. (WebCore::MediaPlayerPrivateMediaSourceAVFObjC::destroyDecompressionSession): Destroys a decompression session. (WebCore::MediaPlayerPrivateMediaSourceAVFObjC::setHasAvailableVideoFrame): If seek completion delayed, complete now. Ditto for ready state change. (WebCore::MediaPlayerPrivateMediaSourceAVFObjC::setReadyState): If waiting for a video frame, delay ready state change. (WebCore::MediaPlayerPrivateMediaSourceAVFObjC::addDisplayLayer): Deleted. (WebCore::MediaPlayerPrivateMediaSourceAVFObjC::removeDisplayLayer): Deleted. * platform/graphics/avfoundation/objc/MediaSourcePrivateAVFObjC.h: * platform/graphics/avfoundation/objc/MediaSourcePrivateAVFObjC.mm: (WebCore::MediaSourcePrivateAVFObjC::hasVideo): Promote to a class function. (WebCore::MediaSourcePrivateAVFObjC::hasSelectedVideo): Return whether any of the active source buffers have video and are selected. (WebCore::MediaSourcePrivateAVFObjC::hasSelectedVideoChanged): Call setSourceBufferWithSelectedVideo(). (WebCore::MediaSourcePrivateAVFObjC::setVideoLayer): Set (or clear) the layer on the selected buffer. (WebCore::MediaSourcePrivateAVFObjC::setDecompressionSession): Ditto for decompression session. (WebCore::MediaSourcePrivateAVFObjC::setSourceBufferWithSelectedVideo): Remove the layer and decompression session from the unselected buffer and add the decompression session or layer to the newly selected buffer. (WebCore::MediaSourcePrivateAVFObjCHasVideo): Deleted. * platform/graphics/avfoundation/objc/SourceBufferPrivateAVFObjC.h: * platform/graphics/avfoundation/objc/SourceBufferPrivateAVFObjC.mm: (WebCore::SourceBufferPrivateAVFObjC::destroyRenderers): Clear the videoLayer and decompressionSession. (WebCore::SourceBufferPrivateAVFObjC::hasSelectedVideo): Return whether the buffer has a selected video track. (WebCore::SourceBufferPrivateAVFObjC::trackDidChangeEnabled): The media player now manages the video layer and decompression session lifetimes. (WebCore::SourceBufferPrivateAVFObjC::flush): Flush the decompression session, if it exists. (WebCore::SourceBufferPrivateAVFObjC::enqueueSample): Enqueue to the decompression session, if it exists. (WebCore::SourceBufferPrivateAVFObjC::isReadyForMoreSamples): As the decompression session, if it exists. (WebCore::SourceBufferPrivateAVFObjC::didBecomeReadyForMoreSamples): Tell the decompression session to stop requesting data, if it exists. (WebCore::SourceBufferPrivateAVFObjC::notifyClientWhenReadyForMoreSamples): Request media data from the decompression session, if it exists. (WebCore::SourceBufferPrivateAVFObjC::setVideoLayer): Added. (WebCore::SourceBufferPrivateAVFObjC::setDecompressionSession): Added. * platform/graphics/cocoa/WebCoreDecompressionSession.h: Added. (WebCore::WebCoreDecompressionSession::create): (WebCore::WebCoreDecompressionSession::isInvalidated): (WebCore::WebCoreDecompressionSession::createWeakPtr): * platform/graphics/cocoa/WebCoreDecompressionSession.mm: Added. (WebCore::WebCoreDecompressionSession::WebCoreDecompressionSession): Register for media data requests. (WebCore::WebCoreDecompressionSession::invalidate): Unregister for same. (WebCore::WebCoreDecompressionSession::maybeBecomeReadyForMoreMediaDataCallback): Pass to maybeBecomeReadyForMoreMediaData. (WebCore::WebCoreDecompressionSession::maybeBecomeReadyForMoreMediaData): Check in-flight decodes, and decoded frame counts. (WebCore::WebCoreDecompressionSession::enqueueSample): Pass the sample to be decoded on a background queue. (WebCore::WebCoreDecompressionSession::decodeSample): Decode the sample. (WebCore::WebCoreDecompressionSession::decompressionOutputCallback): Call handleDecompressionOutput. (WebCore::WebCoreDecompressionSession::handleDecompressionOutput): Pass decoded sample to be enqueued on the main thread. (WebCore::WebCoreDecompressionSession::getFirstVideoFrame): (WebCore::WebCoreDecompressionSession::enqueueDecodedSample): Enqueue the frame (if it's a displayed frame). (WebCore::WebCoreDecompressionSession::isReadyForMoreMediaData): Return whether we've hit our high water sample count. (WebCore::WebCoreDecompressionSession::requestMediaDataWhenReady): (WebCore::WebCoreDecompressionSession::stopRequestingMediaData): Unset the same. (WebCore::WebCoreDecompressionSession::notifyWhenHasAvailableVideoFrame): Set a callback to notify when a decoded frame has been enqueued. (WebCore::WebCoreDecompressionSession::imageForTime): Successively dequeue images until reaching one at or beyond the requested time. (WebCore::WebCoreDecompressionSession::flush): Synchronously empty the producer and consumer queues. (WebCore::WebCoreDecompressionSession::getDecodeTime): Utility method. (WebCore::WebCoreDecompressionSession::getPresentationTime): Ditto. (WebCore::WebCoreDecompressionSession::getDuration): Ditto. (WebCore::WebCoreDecompressionSession::compareBuffers): Ditto. * platform/cocoa/VideoToolboxSoftLink.cpp: Added. * platform/cocoa/VideoToolboxSoftLink.h: Added. LayoutTests: * media/media-source/content/test-fragmented.mp4: Add a 'edts' atom to move the presentation time for the first sample to 0:00. * media/media-source/content/test-fragmented-manifest.json: * media/media-source/media-source-paint-to-canvas-expected.txt: Added. * media/media-source/media-source-paint-to-canvas.html: Added. git-svn-id: http://svn.webkit.org/repository/webkit/trunk@217185 268f45cc-cd09-0410-ab3c-d52691b4dbfc
WebKit is a cross-platform web browser engine. On iOS and macOS, it powers Safari, Mail, iBooks, and many other applications.
Visit WebKit Feature Status page to see which Web API has been implemented, in development, or under consideration.
Downloading Safari Technology Preview to test the latest version of WebKit.
Once your bug is filed, you will receive email when it is updated at each stage in the bug life cycle. After the bug is considered fixed, you may be asked to download the latest nightly and confirm that the fix works for you.
On Windows, follow the instructions on our website.
Run the following command to clone WebKit's Git SVN repository:
git clone git://git.webkit.org/WebKit.git WebKit
If you want to be able to commit changes to the repository, or just want to check out branches that aren’t contained in WebKit.git, you will need track WebKit's Subversion repository. You can run the following command to configure this and other options of the new Git clone for WebKit development.
Tools/Scripts/webkit-patch setup-git-clone
For information about this, and other aspects of using Git with WebKit, read the wiki page.
Run the following command to check out WebKit's subversion repository:
svn checkout https://svn.webkit.org/repository/webkit/trunk WebKit
Install Xcode and its command line tools if you haven't done so already:
xcode-select --installRun the following command to build a debug build with debugging symbols and assertions:
Tools/Scripts/build-webkit --debug
For performance testing, and other purposes, use --release instead.
You can open WebKit.xcworkspace to build and debug WebKit within WebKit.
If you don't use a custom build location in Xcode preferences, you have to update the workspace settings to use WebKitBuild directory. In menu bar, choose File > Workspace Settings, then click the Advanced button, select “Custom”, “Relative to Workspace”, and enter WebKitBuild for both Products and Intermediates.
The first time after you install a new Xcode, you will need to run the following command to enable Xcode to build command line tools for iOS Simulator:
sudo Tools/Scripts/configure-xcode-for-ios-development
Without this step, you will see the error message: “target specifies product type ‘com.apple.product-type.tool’, but there’s no such product type for the ‘iphonesimulator’ platform.” when building target JSCLLIntOffsetsExtractor of project JavaScriptCore.
Run the following command to build a debug build with debugging symbols and assertions for iOS:
Tools/Scripts/build-webkit --debug --ios-simulator.
Install the dependencies by running the following command:
Tools/gtk/install-dependencies
Then run the following command to build additional dependencies:
Tools/Scripts/update-webkitgtk-libs
Run the following command to build WebKit with debugging symbols for GTK+ port:
Tools/Scripts/build-webkit --debug --gtk
Note that the procedure for building a release tarball is different. For more information, see the wiki page.
For building WebKit on Windows, see the wiki page.
Run the following command to launch Safari with your local build of WebKit:
Tools/Scripts/run-safari --debug
The run-safari script sets the DYLD_FRAMEWORK_PATH environment variable to point to your build products, and then launches /Applications/Safari.app. DYLD_FRAMEWORK_PATH tells the system loader to prefer your build products over the frameworks installed in /System/Library/Frameworks.
To run other applications with your local build of WebKit, run the following command:
Tools/Scripts/run-webkit-app <application-path>
Run the following command to launch iOS simulator with your local build of WebKit:
run-safari --debug --ios-simulator
In both cases, if you have built release builds instead, use --release instead of --debug.
Congratulations! You’re up and running. Now you can begin coding in WebKit and contribute your fixes and new features to the project. For details on submitting your code to the project, read Contributing Code.