haikuwebkit/LayoutTests/fast/mediastream/MediaStream-video-element-d...

32 lines
992 B
Plaintext
Raw Permalink Normal View History

HTMLVideoElement frames do not update on iOS when src is a MediaStream blob https://bugs.webkit.org/show_bug.cgi?id=159833 <rdar://problem/27379487> Patch by George Ruan <gruan@apple.com> on 2016-07-26 Reviewed by Eric Carlson. Source/WebCore: Test: fast/mediastream/MediaStream-video-element-displays-buffer.html * WebCore.xcodeproj/project.pbxproj: * platform/cf/CoreMediaSoftLink.cpp: Add CMSampleBufferCreateReadyWithImageBuffer and CMVideoFormatDescriptionCreateForImageBuffer softlink. * platform/cf/CoreMediaSoftLink.h: Ditto. * platform/cocoa/CoreVideoSoftLink.cpp: Add CVPixelBufferCreate, kCVPixelBufferCGBitmapContextCompatibilityKey, and kCVPixelBufferCGImageCompatibilityKey. * platform/cocoa/CoreVideoSoftLink.h: Ditto. * platform/graphics/avfoundation/MediaSampleAVFObjC.h: Change create to return a Ref<T> instead of RefPtr<T>. * platform/graphics/avfoundation/objc/MediaPlayerPrivateMediaStreamAVFObjC.h: Make observer of MediaStreamTrackPrivate and make MediaPlayer use an AVSampleBufferDisplayLayer instead of CALayer. * platform/graphics/avfoundation/objc/MediaPlayerPrivateMediaStreamAVFObjC.mm: Ditto. (WebCore::MediaPlayerPrivateMediaStreamAVFObjC::~MediaPlayerPrivateMediaStreamAVFObjC): Clean up observers and AVSampleBufferDisplayLayer. (WebCore::MediaPlayerPrivateMediaStreamAVFObjC::isAvailable): Ensures AVSampleBufferDisplayLayer is available. (WebCore::MediaPlayerPrivateMediaStreamAVFObjC::enqueueAudioSampleBufferFromTrack): Placeholder. (WebCore::MediaPlayerPrivateMediaStreamAVFObjC::enqueueVideoSampleBufferFromTrack): Responsible for enqueuing sample buffers to the active video track. (WebCore::MediaPlayerPrivateMediaStreamAVFObjC::ensureLayer): Ensures that an AVSampleBufferDisplayLayer exists. (WebCore::MediaPlayerPrivateMediaStreamAVFObjC::destroyLayer): Destroys the AVSampleBufferDisplayLayer. (WebCore::MediaPlayerPrivateMediaStreamAVFObjC::platformLayer): Replace CALayer with AVSampleBufferDisplayLayer. (WebCore::MediaPlayerPrivateMediaStreamAVFObjC::currentDisplayMode): Ditto. (WebCore::MediaPlayerPrivateMediaStreamAVFObjC::play): Call updateReadyState as a deferred task. (WebCore::MediaPlayerPrivateMediaStreamAVFObjC::currentReadyState): readyState is bumped to HAVE_ENOUGH_DATA only when the MediaPlayerPrivateMediaStreamAVFObjC has received a media sample. (WebCore::MediaPlayerPrivateMediaStreamAVFObjC::sampleBufferUpdated): Called from MediaStreamTrackPrivate when a new SampleBuffer is available. (WebCore::updateTracksOfType): Manage adding and removing self as observer from tracks. (WebCore::MediaPlayerPrivateMediaStreamAVFObjC::updateTracks): Replace CALayer with AVSampleBufferDisplayLayer (WebCore::MediaPlayerPrivateMediaStreamAVFObjC::acceleratedRenderingStateChanged): Copied from MediaPlayerPrivateMediaSourceAVFObjC.mm (WebCore::MediaPlayerPrivateMediaStreamAVFObjC::load): Deleted CALayer. (WebCore::MediaPlayerPrivateMediaStreamAVFObjC::updateDisplayMode): Deleted process of updating CALayer. (WebCore::MediaPlayerPrivateMediaStreamAVFObjC::updateIntrinsicSize): Deleted CALayer. (WebCore::MediaPlayerPrivateMediaStreamAVFObjC::createPreviewLayers): Deleted. * platform/mediastream/MediaStreamPrivate.cpp: (WebCore::MediaStreamPrivate::updateActiveVideoTrack): Remove redundant check. * platform/mediastream/MediaStreamTrackPrivate.cpp: (WebCore::MediaStreamTrackPrivate::sourceHasMoreMediaData): Called from RealtimeMediaSource when a new SampleBuffer is available. * platform/mediastream/MediaStreamTrackPrivate.h: (WebCore::MediaStreamTrackPrivate::Observer::sampleBufferUpdated): Relays to MediaPlayerPrivateMediaStreamAVFObjC that a new SampleBuffer is available to enqueue to the AVSampleBufferDisplayLayer. * platform/mediastream/RealtimeMediaSource.cpp: (WebCore::RealtimeMediaSource::settingsDidChange): Fix grammatical mistake in function name settingsDidChanged(). (WebCore::RealtimeMediaSource::mediaDataUpdated): Relays to all observers that a new SampleBuffer is available. (WebCore::RealtimeMediaSource::settingsDidChanged): Deleted. * platform/mediastream/RealtimeMediaSource.h: * platform/mediastream/mac/AVVideoCaptureSource.mm: (WebCore::AVVideoCaptureSource::processNewFrame): Calls mediaDataUpdated when a new SampleBuffer is captured. * platform/mediastream/mac/MockRealtimeVideoSourceMac.h: * platform/mediastream/mac/MockRealtimeVideoSourceMac.mm: (WebCore::MockRealtimeVideoSourceMac::CMSampleBufferFromPixelBuffer): Convert CVPixelBuffer to CMSampleBuffer. (WebCore::MockRealtimeVideoSourceMac::pixelBufferFromCGImage): Convert CGImage to CVPixelBuffer. (WebCore::MockRealtimeVideoSourceMac::updateSampleBuffer): Creates a CMSampleBuffer from current imageBuffer and sends the CMSampleBuffer to MediaPlayerPrivateMediaStreamAVFObjC * platform/mock/MockRealtimeVideoSource.cpp: (WebCore::MockRealtimeVideoSource::setFrameRate): Fix grammar of settingsDidChanged() to settingsDidChange(). (WebCore::MockRealtimeVideoSource::setSize): Ditto. (WebCore::MockRealtimeVideoSource::generateFrame): Call updateSampleBuffer(). * platform/mock/MockRealtimeVideoSource.h: Change elapsedTime() from private to protected. (WebCore::MockRealtimeVideoSource::updateSampleBuffer): Overriden by MockRealtimeVideoSourceMac. LayoutTests: * fast/mediastream/MediaStream-video-element-displays-buffer-expected.txt: Added. * fast/mediastream/MediaStream-video-element-displays-buffer.html: Added. Checks that a video element with a mediastream source displays frames that are neither black or transparent. * fast/mediastream/resources/getUserMedia-helper.js: (setupVideoElementWithStream): Sets up video element with global variable mediastream. Canonical link: https://commits.webkit.org/178368@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@203739 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2016-07-26 21:52:27 +00:00
Tests that the stream displays captured buffers to the video element.
On success, you will see a series of "PASS" messages, followed by "TEST COMPLETE".
[MediaStream] "ideal" constraints passed to getUserMedia should affect fitness score https://bugs.webkit.org/show_bug.cgi?id=170056 Reviewed by Youenn Fablet. Source/WebCore: Include the fitness score calculated for ideal constraints in the calculation of a capture overall device fitness score. No new tests, existing tests updated. * platform/mediastream/MediaConstraints.cpp: (WebCore::StringConstraint::fitnessDistance): Drive-by fix: return early if ideal is empty, not exact. * platform/mediastream/RealtimeMediaSource.cpp: (WebCore::RealtimeMediaSource::supportsSizeAndFrameRate): Return fitness distance. (WebCore::RealtimeMediaSource::selectSettings): Include the fitness distance of supported ideal constraints. (WebCore::RealtimeMediaSource::supportsConstraint): New. (WebCore::RealtimeMediaSource::applyConstraints): * platform/mediastream/RealtimeMediaSource.h: * platform/mock/MockRealtimeMediaSourceCenter.cpp: (WebCore::MockRealtimeMediaSourceCenter::validateRequestConstraints): Sort candidate sources by their fitness score. * platform/mock/MockRealtimeVideoSource.cpp: (WebCore::MockRealtimeVideoSource::initializeCapabilities): Each video source should support one facing mode, not both. Source/WebKit2: * UIProcess/UserMediaPermissionRequestManagerProxy.cpp: (WebKit::UserMediaPermissionRequestManagerProxy::requestUserMediaPermissionForFrame): When short-circuiting the user prompt because the page is already authorized, return the first audio and/or video device because so the page gets the one with the best fitness distance. LayoutTests: * fast/mediastream/MediaStream-video-element-displays-buffer-expected.txt: * fast/mediastream/MediaStream-video-element-displays-buffer.html: * fast/mediastream/MediaStreamTrack-getCapabilities-expected.txt: * fast/mediastream/apply-constraints-advanced-expected.txt: * fast/mediastream/apply-constraints-advanced.html: * fast/mediastream/apply-constraints-video-expected.txt: * fast/mediastream/apply-constraints-video.html: Canonical link: https://commits.webkit.org/187010@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@214385 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-03-24 23:27:14 +00:00
=== checking pixels from front camera ===
HTMLVideoElement frames do not update on iOS when src is a MediaStream blob https://bugs.webkit.org/show_bug.cgi?id=159833 <rdar://problem/27379487> Patch by George Ruan <gruan@apple.com> on 2016-07-26 Reviewed by Eric Carlson. Source/WebCore: Test: fast/mediastream/MediaStream-video-element-displays-buffer.html * WebCore.xcodeproj/project.pbxproj: * platform/cf/CoreMediaSoftLink.cpp: Add CMSampleBufferCreateReadyWithImageBuffer and CMVideoFormatDescriptionCreateForImageBuffer softlink. * platform/cf/CoreMediaSoftLink.h: Ditto. * platform/cocoa/CoreVideoSoftLink.cpp: Add CVPixelBufferCreate, kCVPixelBufferCGBitmapContextCompatibilityKey, and kCVPixelBufferCGImageCompatibilityKey. * platform/cocoa/CoreVideoSoftLink.h: Ditto. * platform/graphics/avfoundation/MediaSampleAVFObjC.h: Change create to return a Ref<T> instead of RefPtr<T>. * platform/graphics/avfoundation/objc/MediaPlayerPrivateMediaStreamAVFObjC.h: Make observer of MediaStreamTrackPrivate and make MediaPlayer use an AVSampleBufferDisplayLayer instead of CALayer. * platform/graphics/avfoundation/objc/MediaPlayerPrivateMediaStreamAVFObjC.mm: Ditto. (WebCore::MediaPlayerPrivateMediaStreamAVFObjC::~MediaPlayerPrivateMediaStreamAVFObjC): Clean up observers and AVSampleBufferDisplayLayer. (WebCore::MediaPlayerPrivateMediaStreamAVFObjC::isAvailable): Ensures AVSampleBufferDisplayLayer is available. (WebCore::MediaPlayerPrivateMediaStreamAVFObjC::enqueueAudioSampleBufferFromTrack): Placeholder. (WebCore::MediaPlayerPrivateMediaStreamAVFObjC::enqueueVideoSampleBufferFromTrack): Responsible for enqueuing sample buffers to the active video track. (WebCore::MediaPlayerPrivateMediaStreamAVFObjC::ensureLayer): Ensures that an AVSampleBufferDisplayLayer exists. (WebCore::MediaPlayerPrivateMediaStreamAVFObjC::destroyLayer): Destroys the AVSampleBufferDisplayLayer. (WebCore::MediaPlayerPrivateMediaStreamAVFObjC::platformLayer): Replace CALayer with AVSampleBufferDisplayLayer. (WebCore::MediaPlayerPrivateMediaStreamAVFObjC::currentDisplayMode): Ditto. (WebCore::MediaPlayerPrivateMediaStreamAVFObjC::play): Call updateReadyState as a deferred task. (WebCore::MediaPlayerPrivateMediaStreamAVFObjC::currentReadyState): readyState is bumped to HAVE_ENOUGH_DATA only when the MediaPlayerPrivateMediaStreamAVFObjC has received a media sample. (WebCore::MediaPlayerPrivateMediaStreamAVFObjC::sampleBufferUpdated): Called from MediaStreamTrackPrivate when a new SampleBuffer is available. (WebCore::updateTracksOfType): Manage adding and removing self as observer from tracks. (WebCore::MediaPlayerPrivateMediaStreamAVFObjC::updateTracks): Replace CALayer with AVSampleBufferDisplayLayer (WebCore::MediaPlayerPrivateMediaStreamAVFObjC::acceleratedRenderingStateChanged): Copied from MediaPlayerPrivateMediaSourceAVFObjC.mm (WebCore::MediaPlayerPrivateMediaStreamAVFObjC::load): Deleted CALayer. (WebCore::MediaPlayerPrivateMediaStreamAVFObjC::updateDisplayMode): Deleted process of updating CALayer. (WebCore::MediaPlayerPrivateMediaStreamAVFObjC::updateIntrinsicSize): Deleted CALayer. (WebCore::MediaPlayerPrivateMediaStreamAVFObjC::createPreviewLayers): Deleted. * platform/mediastream/MediaStreamPrivate.cpp: (WebCore::MediaStreamPrivate::updateActiveVideoTrack): Remove redundant check. * platform/mediastream/MediaStreamTrackPrivate.cpp: (WebCore::MediaStreamTrackPrivate::sourceHasMoreMediaData): Called from RealtimeMediaSource when a new SampleBuffer is available. * platform/mediastream/MediaStreamTrackPrivate.h: (WebCore::MediaStreamTrackPrivate::Observer::sampleBufferUpdated): Relays to MediaPlayerPrivateMediaStreamAVFObjC that a new SampleBuffer is available to enqueue to the AVSampleBufferDisplayLayer. * platform/mediastream/RealtimeMediaSource.cpp: (WebCore::RealtimeMediaSource::settingsDidChange): Fix grammatical mistake in function name settingsDidChanged(). (WebCore::RealtimeMediaSource::mediaDataUpdated): Relays to all observers that a new SampleBuffer is available. (WebCore::RealtimeMediaSource::settingsDidChanged): Deleted. * platform/mediastream/RealtimeMediaSource.h: * platform/mediastream/mac/AVVideoCaptureSource.mm: (WebCore::AVVideoCaptureSource::processNewFrame): Calls mediaDataUpdated when a new SampleBuffer is captured. * platform/mediastream/mac/MockRealtimeVideoSourceMac.h: * platform/mediastream/mac/MockRealtimeVideoSourceMac.mm: (WebCore::MockRealtimeVideoSourceMac::CMSampleBufferFromPixelBuffer): Convert CVPixelBuffer to CMSampleBuffer. (WebCore::MockRealtimeVideoSourceMac::pixelBufferFromCGImage): Convert CGImage to CVPixelBuffer. (WebCore::MockRealtimeVideoSourceMac::updateSampleBuffer): Creates a CMSampleBuffer from current imageBuffer and sends the CMSampleBuffer to MediaPlayerPrivateMediaStreamAVFObjC * platform/mock/MockRealtimeVideoSource.cpp: (WebCore::MockRealtimeVideoSource::setFrameRate): Fix grammar of settingsDidChanged() to settingsDidChange(). (WebCore::MockRealtimeVideoSource::setSize): Ditto. (WebCore::MockRealtimeVideoSource::generateFrame): Call updateSampleBuffer(). * platform/mock/MockRealtimeVideoSource.h: Change elapsedTime() from private to protected. (WebCore::MockRealtimeVideoSource::updateSampleBuffer): Overriden by MockRealtimeVideoSourceMac. LayoutTests: * fast/mediastream/MediaStream-video-element-displays-buffer-expected.txt: Added. * fast/mediastream/MediaStream-video-element-displays-buffer.html: Added. Checks that a video element with a mediastream source displays frames that are neither black or transparent. * fast/mediastream/resources/getUserMedia-helper.js: (setupVideoElementWithStream): Sets up video element with global variable mediastream. Canonical link: https://commits.webkit.org/178368@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@203739 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2016-07-26 21:52:27 +00:00
PASS mediaDevices.getUserMedia generated a stream successfully.
Deprecate and remove URL.createObjectURL(mediastream) https://bugs.webkit.org/show_bug.cgi?id=167518 <rdar://problem/31149607> Patch by Youenn Fablet <youenn@apple.com> on 2017-04-05 Reviewed by Eric Carlson. Source/WebCore: Covered by updated tests. * CMakeLists.txt: * DerivedSources.cpp: * DerivedSources.make: * Modules/mediastream/DOMURLMediaStream.cpp: Removed. * Modules/mediastream/DOMURLMediaStream.h: Removed. * Modules/mediastream/DOMURLMediaStream.idl: Removed. * WebCore.xcodeproj/project.pbxproj: * html/DOMURL.idl: LayoutTests: * fast/mediacapturefromelement/CanvasCaptureMediaStream-2d-events.html: * fast/mediacapturefromelement/CanvasCaptureMediaStream-request-frame-events.html: * fast/mediacapturefromelement/CanvasCaptureMediaStream-webgl-events.html: * fast/mediastream/MediaStream-video-element-displays-buffer-expected.txt: * fast/mediastream/MediaStream-video-element-displays-buffer.html: * fast/mediastream/MediaStream-video-element-expected.txt: * fast/mediastream/MediaStream-video-element-track-stop-expected.txt: * fast/mediastream/MediaStream-video-element-track-stop.html: * fast/mediastream/MediaStream-video-element-video-tracks-disabled-then-enabled-expected.txt: * fast/mediastream/MediaStream-video-element-video-tracks-disabled.html: * fast/mediastream/MediaStream-video-element.html: * fast/mediastream/resources/getUserMedia-helper.js: (setupVideoElementWithStream): Canonical link: https://commits.webkit.org/187433@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@214951 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-04-05 17:31:37 +00:00
videos[0].srcObject = mediaStream
[MediaStream] "ideal" constraints passed to getUserMedia should affect fitness score https://bugs.webkit.org/show_bug.cgi?id=170056 Reviewed by Youenn Fablet. Source/WebCore: Include the fitness score calculated for ideal constraints in the calculation of a capture overall device fitness score. No new tests, existing tests updated. * platform/mediastream/MediaConstraints.cpp: (WebCore::StringConstraint::fitnessDistance): Drive-by fix: return early if ideal is empty, not exact. * platform/mediastream/RealtimeMediaSource.cpp: (WebCore::RealtimeMediaSource::supportsSizeAndFrameRate): Return fitness distance. (WebCore::RealtimeMediaSource::selectSettings): Include the fitness distance of supported ideal constraints. (WebCore::RealtimeMediaSource::supportsConstraint): New. (WebCore::RealtimeMediaSource::applyConstraints): * platform/mediastream/RealtimeMediaSource.h: * platform/mock/MockRealtimeMediaSourceCenter.cpp: (WebCore::MockRealtimeMediaSourceCenter::validateRequestConstraints): Sort candidate sources by their fitness score. * platform/mock/MockRealtimeVideoSource.cpp: (WebCore::MockRealtimeVideoSource::initializeCapabilities): Each video source should support one facing mode, not both. Source/WebKit2: * UIProcess/UserMediaPermissionRequestManagerProxy.cpp: (WebKit::UserMediaPermissionRequestManagerProxy::requestUserMediaPermissionForFrame): When short-circuiting the user prompt because the page is already authorized, return the first audio and/or video device because so the page gets the one with the best fitness distance. LayoutTests: * fast/mediastream/MediaStream-video-element-displays-buffer-expected.txt: * fast/mediastream/MediaStream-video-element-displays-buffer.html: * fast/mediastream/MediaStreamTrack-getCapabilities-expected.txt: * fast/mediastream/apply-constraints-advanced-expected.txt: * fast/mediastream/apply-constraints-advanced.html: * fast/mediastream/apply-constraints-video-expected.txt: * fast/mediastream/apply-constraints-video.html: Canonical link: https://commits.webkit.org/187010@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@214385 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-03-24 23:27:14 +00:00
videos[0].play()
PASS isPixelTransparent(buffer) is true
context.drawImage(videos[0], 0, 0, 680, 360)
PASS isPixelTransparent(buffer) is false
PASS isPixelBlack(buffer) is false
PASS isPixelTransparent(buffer) is false
PASS isPixelBlack(buffer) is true
HTMLVideoElement frames do not update on iOS when src is a MediaStream blob https://bugs.webkit.org/show_bug.cgi?id=159833 <rdar://problem/27379487> Patch by George Ruan <gruan@apple.com> on 2016-07-26 Reviewed by Eric Carlson. Source/WebCore: Test: fast/mediastream/MediaStream-video-element-displays-buffer.html * WebCore.xcodeproj/project.pbxproj: * platform/cf/CoreMediaSoftLink.cpp: Add CMSampleBufferCreateReadyWithImageBuffer and CMVideoFormatDescriptionCreateForImageBuffer softlink. * platform/cf/CoreMediaSoftLink.h: Ditto. * platform/cocoa/CoreVideoSoftLink.cpp: Add CVPixelBufferCreate, kCVPixelBufferCGBitmapContextCompatibilityKey, and kCVPixelBufferCGImageCompatibilityKey. * platform/cocoa/CoreVideoSoftLink.h: Ditto. * platform/graphics/avfoundation/MediaSampleAVFObjC.h: Change create to return a Ref<T> instead of RefPtr<T>. * platform/graphics/avfoundation/objc/MediaPlayerPrivateMediaStreamAVFObjC.h: Make observer of MediaStreamTrackPrivate and make MediaPlayer use an AVSampleBufferDisplayLayer instead of CALayer. * platform/graphics/avfoundation/objc/MediaPlayerPrivateMediaStreamAVFObjC.mm: Ditto. (WebCore::MediaPlayerPrivateMediaStreamAVFObjC::~MediaPlayerPrivateMediaStreamAVFObjC): Clean up observers and AVSampleBufferDisplayLayer. (WebCore::MediaPlayerPrivateMediaStreamAVFObjC::isAvailable): Ensures AVSampleBufferDisplayLayer is available. (WebCore::MediaPlayerPrivateMediaStreamAVFObjC::enqueueAudioSampleBufferFromTrack): Placeholder. (WebCore::MediaPlayerPrivateMediaStreamAVFObjC::enqueueVideoSampleBufferFromTrack): Responsible for enqueuing sample buffers to the active video track. (WebCore::MediaPlayerPrivateMediaStreamAVFObjC::ensureLayer): Ensures that an AVSampleBufferDisplayLayer exists. (WebCore::MediaPlayerPrivateMediaStreamAVFObjC::destroyLayer): Destroys the AVSampleBufferDisplayLayer. (WebCore::MediaPlayerPrivateMediaStreamAVFObjC::platformLayer): Replace CALayer with AVSampleBufferDisplayLayer. (WebCore::MediaPlayerPrivateMediaStreamAVFObjC::currentDisplayMode): Ditto. (WebCore::MediaPlayerPrivateMediaStreamAVFObjC::play): Call updateReadyState as a deferred task. (WebCore::MediaPlayerPrivateMediaStreamAVFObjC::currentReadyState): readyState is bumped to HAVE_ENOUGH_DATA only when the MediaPlayerPrivateMediaStreamAVFObjC has received a media sample. (WebCore::MediaPlayerPrivateMediaStreamAVFObjC::sampleBufferUpdated): Called from MediaStreamTrackPrivate when a new SampleBuffer is available. (WebCore::updateTracksOfType): Manage adding and removing self as observer from tracks. (WebCore::MediaPlayerPrivateMediaStreamAVFObjC::updateTracks): Replace CALayer with AVSampleBufferDisplayLayer (WebCore::MediaPlayerPrivateMediaStreamAVFObjC::acceleratedRenderingStateChanged): Copied from MediaPlayerPrivateMediaSourceAVFObjC.mm (WebCore::MediaPlayerPrivateMediaStreamAVFObjC::load): Deleted CALayer. (WebCore::MediaPlayerPrivateMediaStreamAVFObjC::updateDisplayMode): Deleted process of updating CALayer. (WebCore::MediaPlayerPrivateMediaStreamAVFObjC::updateIntrinsicSize): Deleted CALayer. (WebCore::MediaPlayerPrivateMediaStreamAVFObjC::createPreviewLayers): Deleted. * platform/mediastream/MediaStreamPrivate.cpp: (WebCore::MediaStreamPrivate::updateActiveVideoTrack): Remove redundant check. * platform/mediastream/MediaStreamTrackPrivate.cpp: (WebCore::MediaStreamTrackPrivate::sourceHasMoreMediaData): Called from RealtimeMediaSource when a new SampleBuffer is available. * platform/mediastream/MediaStreamTrackPrivate.h: (WebCore::MediaStreamTrackPrivate::Observer::sampleBufferUpdated): Relays to MediaPlayerPrivateMediaStreamAVFObjC that a new SampleBuffer is available to enqueue to the AVSampleBufferDisplayLayer. * platform/mediastream/RealtimeMediaSource.cpp: (WebCore::RealtimeMediaSource::settingsDidChange): Fix grammatical mistake in function name settingsDidChanged(). (WebCore::RealtimeMediaSource::mediaDataUpdated): Relays to all observers that a new SampleBuffer is available. (WebCore::RealtimeMediaSource::settingsDidChanged): Deleted. * platform/mediastream/RealtimeMediaSource.h: * platform/mediastream/mac/AVVideoCaptureSource.mm: (WebCore::AVVideoCaptureSource::processNewFrame): Calls mediaDataUpdated when a new SampleBuffer is captured. * platform/mediastream/mac/MockRealtimeVideoSourceMac.h: * platform/mediastream/mac/MockRealtimeVideoSourceMac.mm: (WebCore::MockRealtimeVideoSourceMac::CMSampleBufferFromPixelBuffer): Convert CVPixelBuffer to CMSampleBuffer. (WebCore::MockRealtimeVideoSourceMac::pixelBufferFromCGImage): Convert CGImage to CVPixelBuffer. (WebCore::MockRealtimeVideoSourceMac::updateSampleBuffer): Creates a CMSampleBuffer from current imageBuffer and sends the CMSampleBuffer to MediaPlayerPrivateMediaStreamAVFObjC * platform/mock/MockRealtimeVideoSource.cpp: (WebCore::MockRealtimeVideoSource::setFrameRate): Fix grammar of settingsDidChanged() to settingsDidChange(). (WebCore::MockRealtimeVideoSource::setSize): Ditto. (WebCore::MockRealtimeVideoSource::generateFrame): Call updateSampleBuffer(). * platform/mock/MockRealtimeVideoSource.h: Change elapsedTime() from private to protected. (WebCore::MockRealtimeVideoSource::updateSampleBuffer): Overriden by MockRealtimeVideoSourceMac. LayoutTests: * fast/mediastream/MediaStream-video-element-displays-buffer-expected.txt: Added. * fast/mediastream/MediaStream-video-element-displays-buffer.html: Added. Checks that a video element with a mediastream source displays frames that are neither black or transparent. * fast/mediastream/resources/getUserMedia-helper.js: (setupVideoElementWithStream): Sets up video element with global variable mediastream. Canonical link: https://commits.webkit.org/178368@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@203739 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2016-07-26 21:52:27 +00:00
[MediaStream] "ideal" constraints passed to getUserMedia should affect fitness score https://bugs.webkit.org/show_bug.cgi?id=170056 Reviewed by Youenn Fablet. Source/WebCore: Include the fitness score calculated for ideal constraints in the calculation of a capture overall device fitness score. No new tests, existing tests updated. * platform/mediastream/MediaConstraints.cpp: (WebCore::StringConstraint::fitnessDistance): Drive-by fix: return early if ideal is empty, not exact. * platform/mediastream/RealtimeMediaSource.cpp: (WebCore::RealtimeMediaSource::supportsSizeAndFrameRate): Return fitness distance. (WebCore::RealtimeMediaSource::selectSettings): Include the fitness distance of supported ideal constraints. (WebCore::RealtimeMediaSource::supportsConstraint): New. (WebCore::RealtimeMediaSource::applyConstraints): * platform/mediastream/RealtimeMediaSource.h: * platform/mock/MockRealtimeMediaSourceCenter.cpp: (WebCore::MockRealtimeMediaSourceCenter::validateRequestConstraints): Sort candidate sources by their fitness score. * platform/mock/MockRealtimeVideoSource.cpp: (WebCore::MockRealtimeVideoSource::initializeCapabilities): Each video source should support one facing mode, not both. Source/WebKit2: * UIProcess/UserMediaPermissionRequestManagerProxy.cpp: (WebKit::UserMediaPermissionRequestManagerProxy::requestUserMediaPermissionForFrame): When short-circuiting the user prompt because the page is already authorized, return the first audio and/or video device because so the page gets the one with the best fitness distance. LayoutTests: * fast/mediastream/MediaStream-video-element-displays-buffer-expected.txt: * fast/mediastream/MediaStream-video-element-displays-buffer.html: * fast/mediastream/MediaStreamTrack-getCapabilities-expected.txt: * fast/mediastream/apply-constraints-advanced-expected.txt: * fast/mediastream/apply-constraints-advanced.html: * fast/mediastream/apply-constraints-video-expected.txt: * fast/mediastream/apply-constraints-video.html: Canonical link: https://commits.webkit.org/187010@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@214385 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-03-24 23:27:14 +00:00
=== checking pixels from back camera ===
PASS mediaDevices.getUserMedia generated a stream successfully.
Deprecate and remove URL.createObjectURL(mediastream) https://bugs.webkit.org/show_bug.cgi?id=167518 <rdar://problem/31149607> Patch by Youenn Fablet <youenn@apple.com> on 2017-04-05 Reviewed by Eric Carlson. Source/WebCore: Covered by updated tests. * CMakeLists.txt: * DerivedSources.cpp: * DerivedSources.make: * Modules/mediastream/DOMURLMediaStream.cpp: Removed. * Modules/mediastream/DOMURLMediaStream.h: Removed. * Modules/mediastream/DOMURLMediaStream.idl: Removed. * WebCore.xcodeproj/project.pbxproj: * html/DOMURL.idl: LayoutTests: * fast/mediacapturefromelement/CanvasCaptureMediaStream-2d-events.html: * fast/mediacapturefromelement/CanvasCaptureMediaStream-request-frame-events.html: * fast/mediacapturefromelement/CanvasCaptureMediaStream-webgl-events.html: * fast/mediastream/MediaStream-video-element-displays-buffer-expected.txt: * fast/mediastream/MediaStream-video-element-displays-buffer.html: * fast/mediastream/MediaStream-video-element-expected.txt: * fast/mediastream/MediaStream-video-element-track-stop-expected.txt: * fast/mediastream/MediaStream-video-element-track-stop.html: * fast/mediastream/MediaStream-video-element-video-tracks-disabled-then-enabled-expected.txt: * fast/mediastream/MediaStream-video-element-video-tracks-disabled.html: * fast/mediastream/MediaStream-video-element.html: * fast/mediastream/resources/getUserMedia-helper.js: (setupVideoElementWithStream): Canonical link: https://commits.webkit.org/187433@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@214951 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-04-05 17:31:37 +00:00
videos[1].srcObject = mediaStream
[MediaStream] "ideal" constraints passed to getUserMedia should affect fitness score https://bugs.webkit.org/show_bug.cgi?id=170056 Reviewed by Youenn Fablet. Source/WebCore: Include the fitness score calculated for ideal constraints in the calculation of a capture overall device fitness score. No new tests, existing tests updated. * platform/mediastream/MediaConstraints.cpp: (WebCore::StringConstraint::fitnessDistance): Drive-by fix: return early if ideal is empty, not exact. * platform/mediastream/RealtimeMediaSource.cpp: (WebCore::RealtimeMediaSource::supportsSizeAndFrameRate): Return fitness distance. (WebCore::RealtimeMediaSource::selectSettings): Include the fitness distance of supported ideal constraints. (WebCore::RealtimeMediaSource::supportsConstraint): New. (WebCore::RealtimeMediaSource::applyConstraints): * platform/mediastream/RealtimeMediaSource.h: * platform/mock/MockRealtimeMediaSourceCenter.cpp: (WebCore::MockRealtimeMediaSourceCenter::validateRequestConstraints): Sort candidate sources by their fitness score. * platform/mock/MockRealtimeVideoSource.cpp: (WebCore::MockRealtimeVideoSource::initializeCapabilities): Each video source should support one facing mode, not both. Source/WebKit2: * UIProcess/UserMediaPermissionRequestManagerProxy.cpp: (WebKit::UserMediaPermissionRequestManagerProxy::requestUserMediaPermissionForFrame): When short-circuiting the user prompt because the page is already authorized, return the first audio and/or video device because so the page gets the one with the best fitness distance. LayoutTests: * fast/mediastream/MediaStream-video-element-displays-buffer-expected.txt: * fast/mediastream/MediaStream-video-element-displays-buffer.html: * fast/mediastream/MediaStreamTrack-getCapabilities-expected.txt: * fast/mediastream/apply-constraints-advanced-expected.txt: * fast/mediastream/apply-constraints-advanced.html: * fast/mediastream/apply-constraints-video-expected.txt: * fast/mediastream/apply-constraints-video.html: Canonical link: https://commits.webkit.org/187010@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@214385 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-03-24 23:27:14 +00:00
videos[1].play()
HTMLVideoElement frames do not update on iOS when src is a MediaStream blob https://bugs.webkit.org/show_bug.cgi?id=159833 <rdar://problem/27379487> Patch by George Ruan <gruan@apple.com> on 2016-07-26 Reviewed by Eric Carlson. Source/WebCore: Test: fast/mediastream/MediaStream-video-element-displays-buffer.html * WebCore.xcodeproj/project.pbxproj: * platform/cf/CoreMediaSoftLink.cpp: Add CMSampleBufferCreateReadyWithImageBuffer and CMVideoFormatDescriptionCreateForImageBuffer softlink. * platform/cf/CoreMediaSoftLink.h: Ditto. * platform/cocoa/CoreVideoSoftLink.cpp: Add CVPixelBufferCreate, kCVPixelBufferCGBitmapContextCompatibilityKey, and kCVPixelBufferCGImageCompatibilityKey. * platform/cocoa/CoreVideoSoftLink.h: Ditto. * platform/graphics/avfoundation/MediaSampleAVFObjC.h: Change create to return a Ref<T> instead of RefPtr<T>. * platform/graphics/avfoundation/objc/MediaPlayerPrivateMediaStreamAVFObjC.h: Make observer of MediaStreamTrackPrivate and make MediaPlayer use an AVSampleBufferDisplayLayer instead of CALayer. * platform/graphics/avfoundation/objc/MediaPlayerPrivateMediaStreamAVFObjC.mm: Ditto. (WebCore::MediaPlayerPrivateMediaStreamAVFObjC::~MediaPlayerPrivateMediaStreamAVFObjC): Clean up observers and AVSampleBufferDisplayLayer. (WebCore::MediaPlayerPrivateMediaStreamAVFObjC::isAvailable): Ensures AVSampleBufferDisplayLayer is available. (WebCore::MediaPlayerPrivateMediaStreamAVFObjC::enqueueAudioSampleBufferFromTrack): Placeholder. (WebCore::MediaPlayerPrivateMediaStreamAVFObjC::enqueueVideoSampleBufferFromTrack): Responsible for enqueuing sample buffers to the active video track. (WebCore::MediaPlayerPrivateMediaStreamAVFObjC::ensureLayer): Ensures that an AVSampleBufferDisplayLayer exists. (WebCore::MediaPlayerPrivateMediaStreamAVFObjC::destroyLayer): Destroys the AVSampleBufferDisplayLayer. (WebCore::MediaPlayerPrivateMediaStreamAVFObjC::platformLayer): Replace CALayer with AVSampleBufferDisplayLayer. (WebCore::MediaPlayerPrivateMediaStreamAVFObjC::currentDisplayMode): Ditto. (WebCore::MediaPlayerPrivateMediaStreamAVFObjC::play): Call updateReadyState as a deferred task. (WebCore::MediaPlayerPrivateMediaStreamAVFObjC::currentReadyState): readyState is bumped to HAVE_ENOUGH_DATA only when the MediaPlayerPrivateMediaStreamAVFObjC has received a media sample. (WebCore::MediaPlayerPrivateMediaStreamAVFObjC::sampleBufferUpdated): Called from MediaStreamTrackPrivate when a new SampleBuffer is available. (WebCore::updateTracksOfType): Manage adding and removing self as observer from tracks. (WebCore::MediaPlayerPrivateMediaStreamAVFObjC::updateTracks): Replace CALayer with AVSampleBufferDisplayLayer (WebCore::MediaPlayerPrivateMediaStreamAVFObjC::acceleratedRenderingStateChanged): Copied from MediaPlayerPrivateMediaSourceAVFObjC.mm (WebCore::MediaPlayerPrivateMediaStreamAVFObjC::load): Deleted CALayer. (WebCore::MediaPlayerPrivateMediaStreamAVFObjC::updateDisplayMode): Deleted process of updating CALayer. (WebCore::MediaPlayerPrivateMediaStreamAVFObjC::updateIntrinsicSize): Deleted CALayer. (WebCore::MediaPlayerPrivateMediaStreamAVFObjC::createPreviewLayers): Deleted. * platform/mediastream/MediaStreamPrivate.cpp: (WebCore::MediaStreamPrivate::updateActiveVideoTrack): Remove redundant check. * platform/mediastream/MediaStreamTrackPrivate.cpp: (WebCore::MediaStreamTrackPrivate::sourceHasMoreMediaData): Called from RealtimeMediaSource when a new SampleBuffer is available. * platform/mediastream/MediaStreamTrackPrivate.h: (WebCore::MediaStreamTrackPrivate::Observer::sampleBufferUpdated): Relays to MediaPlayerPrivateMediaStreamAVFObjC that a new SampleBuffer is available to enqueue to the AVSampleBufferDisplayLayer. * platform/mediastream/RealtimeMediaSource.cpp: (WebCore::RealtimeMediaSource::settingsDidChange): Fix grammatical mistake in function name settingsDidChanged(). (WebCore::RealtimeMediaSource::mediaDataUpdated): Relays to all observers that a new SampleBuffer is available. (WebCore::RealtimeMediaSource::settingsDidChanged): Deleted. * platform/mediastream/RealtimeMediaSource.h: * platform/mediastream/mac/AVVideoCaptureSource.mm: (WebCore::AVVideoCaptureSource::processNewFrame): Calls mediaDataUpdated when a new SampleBuffer is captured. * platform/mediastream/mac/MockRealtimeVideoSourceMac.h: * platform/mediastream/mac/MockRealtimeVideoSourceMac.mm: (WebCore::MockRealtimeVideoSourceMac::CMSampleBufferFromPixelBuffer): Convert CVPixelBuffer to CMSampleBuffer. (WebCore::MockRealtimeVideoSourceMac::pixelBufferFromCGImage): Convert CGImage to CVPixelBuffer. (WebCore::MockRealtimeVideoSourceMac::updateSampleBuffer): Creates a CMSampleBuffer from current imageBuffer and sends the CMSampleBuffer to MediaPlayerPrivateMediaStreamAVFObjC * platform/mock/MockRealtimeVideoSource.cpp: (WebCore::MockRealtimeVideoSource::setFrameRate): Fix grammar of settingsDidChanged() to settingsDidChange(). (WebCore::MockRealtimeVideoSource::setSize): Ditto. (WebCore::MockRealtimeVideoSource::generateFrame): Call updateSampleBuffer(). * platform/mock/MockRealtimeVideoSource.h: Change elapsedTime() from private to protected. (WebCore::MockRealtimeVideoSource::updateSampleBuffer): Overriden by MockRealtimeVideoSourceMac. LayoutTests: * fast/mediastream/MediaStream-video-element-displays-buffer-expected.txt: Added. * fast/mediastream/MediaStream-video-element-displays-buffer.html: Added. Checks that a video element with a mediastream source displays frames that are neither black or transparent. * fast/mediastream/resources/getUserMedia-helper.js: (setupVideoElementWithStream): Sets up video element with global variable mediastream. Canonical link: https://commits.webkit.org/178368@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@203739 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2016-07-26 21:52:27 +00:00
PASS isPixelTransparent(buffer) is true
[MediaStream] "ideal" constraints passed to getUserMedia should affect fitness score https://bugs.webkit.org/show_bug.cgi?id=170056 Reviewed by Youenn Fablet. Source/WebCore: Include the fitness score calculated for ideal constraints in the calculation of a capture overall device fitness score. No new tests, existing tests updated. * platform/mediastream/MediaConstraints.cpp: (WebCore::StringConstraint::fitnessDistance): Drive-by fix: return early if ideal is empty, not exact. * platform/mediastream/RealtimeMediaSource.cpp: (WebCore::RealtimeMediaSource::supportsSizeAndFrameRate): Return fitness distance. (WebCore::RealtimeMediaSource::selectSettings): Include the fitness distance of supported ideal constraints. (WebCore::RealtimeMediaSource::supportsConstraint): New. (WebCore::RealtimeMediaSource::applyConstraints): * platform/mediastream/RealtimeMediaSource.h: * platform/mock/MockRealtimeMediaSourceCenter.cpp: (WebCore::MockRealtimeMediaSourceCenter::validateRequestConstraints): Sort candidate sources by their fitness score. * platform/mock/MockRealtimeVideoSource.cpp: (WebCore::MockRealtimeVideoSource::initializeCapabilities): Each video source should support one facing mode, not both. Source/WebKit2: * UIProcess/UserMediaPermissionRequestManagerProxy.cpp: (WebKit::UserMediaPermissionRequestManagerProxy::requestUserMediaPermissionForFrame): When short-circuiting the user prompt because the page is already authorized, return the first audio and/or video device because so the page gets the one with the best fitness distance. LayoutTests: * fast/mediastream/MediaStream-video-element-displays-buffer-expected.txt: * fast/mediastream/MediaStream-video-element-displays-buffer.html: * fast/mediastream/MediaStreamTrack-getCapabilities-expected.txt: * fast/mediastream/apply-constraints-advanced-expected.txt: * fast/mediastream/apply-constraints-advanced.html: * fast/mediastream/apply-constraints-video-expected.txt: * fast/mediastream/apply-constraints-video.html: Canonical link: https://commits.webkit.org/187010@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@214385 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-03-24 23:27:14 +00:00
context.drawImage(videos[1], 0, 0, 680, 360)
HTMLVideoElement frames do not update on iOS when src is a MediaStream blob https://bugs.webkit.org/show_bug.cgi?id=159833 <rdar://problem/27379487> Patch by George Ruan <gruan@apple.com> on 2016-07-26 Reviewed by Eric Carlson. Source/WebCore: Test: fast/mediastream/MediaStream-video-element-displays-buffer.html * WebCore.xcodeproj/project.pbxproj: * platform/cf/CoreMediaSoftLink.cpp: Add CMSampleBufferCreateReadyWithImageBuffer and CMVideoFormatDescriptionCreateForImageBuffer softlink. * platform/cf/CoreMediaSoftLink.h: Ditto. * platform/cocoa/CoreVideoSoftLink.cpp: Add CVPixelBufferCreate, kCVPixelBufferCGBitmapContextCompatibilityKey, and kCVPixelBufferCGImageCompatibilityKey. * platform/cocoa/CoreVideoSoftLink.h: Ditto. * platform/graphics/avfoundation/MediaSampleAVFObjC.h: Change create to return a Ref<T> instead of RefPtr<T>. * platform/graphics/avfoundation/objc/MediaPlayerPrivateMediaStreamAVFObjC.h: Make observer of MediaStreamTrackPrivate and make MediaPlayer use an AVSampleBufferDisplayLayer instead of CALayer. * platform/graphics/avfoundation/objc/MediaPlayerPrivateMediaStreamAVFObjC.mm: Ditto. (WebCore::MediaPlayerPrivateMediaStreamAVFObjC::~MediaPlayerPrivateMediaStreamAVFObjC): Clean up observers and AVSampleBufferDisplayLayer. (WebCore::MediaPlayerPrivateMediaStreamAVFObjC::isAvailable): Ensures AVSampleBufferDisplayLayer is available. (WebCore::MediaPlayerPrivateMediaStreamAVFObjC::enqueueAudioSampleBufferFromTrack): Placeholder. (WebCore::MediaPlayerPrivateMediaStreamAVFObjC::enqueueVideoSampleBufferFromTrack): Responsible for enqueuing sample buffers to the active video track. (WebCore::MediaPlayerPrivateMediaStreamAVFObjC::ensureLayer): Ensures that an AVSampleBufferDisplayLayer exists. (WebCore::MediaPlayerPrivateMediaStreamAVFObjC::destroyLayer): Destroys the AVSampleBufferDisplayLayer. (WebCore::MediaPlayerPrivateMediaStreamAVFObjC::platformLayer): Replace CALayer with AVSampleBufferDisplayLayer. (WebCore::MediaPlayerPrivateMediaStreamAVFObjC::currentDisplayMode): Ditto. (WebCore::MediaPlayerPrivateMediaStreamAVFObjC::play): Call updateReadyState as a deferred task. (WebCore::MediaPlayerPrivateMediaStreamAVFObjC::currentReadyState): readyState is bumped to HAVE_ENOUGH_DATA only when the MediaPlayerPrivateMediaStreamAVFObjC has received a media sample. (WebCore::MediaPlayerPrivateMediaStreamAVFObjC::sampleBufferUpdated): Called from MediaStreamTrackPrivate when a new SampleBuffer is available. (WebCore::updateTracksOfType): Manage adding and removing self as observer from tracks. (WebCore::MediaPlayerPrivateMediaStreamAVFObjC::updateTracks): Replace CALayer with AVSampleBufferDisplayLayer (WebCore::MediaPlayerPrivateMediaStreamAVFObjC::acceleratedRenderingStateChanged): Copied from MediaPlayerPrivateMediaSourceAVFObjC.mm (WebCore::MediaPlayerPrivateMediaStreamAVFObjC::load): Deleted CALayer. (WebCore::MediaPlayerPrivateMediaStreamAVFObjC::updateDisplayMode): Deleted process of updating CALayer. (WebCore::MediaPlayerPrivateMediaStreamAVFObjC::updateIntrinsicSize): Deleted CALayer. (WebCore::MediaPlayerPrivateMediaStreamAVFObjC::createPreviewLayers): Deleted. * platform/mediastream/MediaStreamPrivate.cpp: (WebCore::MediaStreamPrivate::updateActiveVideoTrack): Remove redundant check. * platform/mediastream/MediaStreamTrackPrivate.cpp: (WebCore::MediaStreamTrackPrivate::sourceHasMoreMediaData): Called from RealtimeMediaSource when a new SampleBuffer is available. * platform/mediastream/MediaStreamTrackPrivate.h: (WebCore::MediaStreamTrackPrivate::Observer::sampleBufferUpdated): Relays to MediaPlayerPrivateMediaStreamAVFObjC that a new SampleBuffer is available to enqueue to the AVSampleBufferDisplayLayer. * platform/mediastream/RealtimeMediaSource.cpp: (WebCore::RealtimeMediaSource::settingsDidChange): Fix grammatical mistake in function name settingsDidChanged(). (WebCore::RealtimeMediaSource::mediaDataUpdated): Relays to all observers that a new SampleBuffer is available. (WebCore::RealtimeMediaSource::settingsDidChanged): Deleted. * platform/mediastream/RealtimeMediaSource.h: * platform/mediastream/mac/AVVideoCaptureSource.mm: (WebCore::AVVideoCaptureSource::processNewFrame): Calls mediaDataUpdated when a new SampleBuffer is captured. * platform/mediastream/mac/MockRealtimeVideoSourceMac.h: * platform/mediastream/mac/MockRealtimeVideoSourceMac.mm: (WebCore::MockRealtimeVideoSourceMac::CMSampleBufferFromPixelBuffer): Convert CVPixelBuffer to CMSampleBuffer. (WebCore::MockRealtimeVideoSourceMac::pixelBufferFromCGImage): Convert CGImage to CVPixelBuffer. (WebCore::MockRealtimeVideoSourceMac::updateSampleBuffer): Creates a CMSampleBuffer from current imageBuffer and sends the CMSampleBuffer to MediaPlayerPrivateMediaStreamAVFObjC * platform/mock/MockRealtimeVideoSource.cpp: (WebCore::MockRealtimeVideoSource::setFrameRate): Fix grammar of settingsDidChanged() to settingsDidChange(). (WebCore::MockRealtimeVideoSource::setSize): Ditto. (WebCore::MockRealtimeVideoSource::generateFrame): Call updateSampleBuffer(). * platform/mock/MockRealtimeVideoSource.h: Change elapsedTime() from private to protected. (WebCore::MockRealtimeVideoSource::updateSampleBuffer): Overriden by MockRealtimeVideoSourceMac. LayoutTests: * fast/mediastream/MediaStream-video-element-displays-buffer-expected.txt: Added. * fast/mediastream/MediaStream-video-element-displays-buffer.html: Added. Checks that a video element with a mediastream source displays frames that are neither black or transparent. * fast/mediastream/resources/getUserMedia-helper.js: (setupVideoElementWithStream): Sets up video element with global variable mediastream. Canonical link: https://commits.webkit.org/178368@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@203739 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2016-07-26 21:52:27 +00:00
PASS isPixelTransparent(buffer) is false
PASS isPixelBlack(buffer) is false
[MediaStream] "ideal" constraints passed to getUserMedia should affect fitness score https://bugs.webkit.org/show_bug.cgi?id=170056 Reviewed by Youenn Fablet. Source/WebCore: Include the fitness score calculated for ideal constraints in the calculation of a capture overall device fitness score. No new tests, existing tests updated. * platform/mediastream/MediaConstraints.cpp: (WebCore::StringConstraint::fitnessDistance): Drive-by fix: return early if ideal is empty, not exact. * platform/mediastream/RealtimeMediaSource.cpp: (WebCore::RealtimeMediaSource::supportsSizeAndFrameRate): Return fitness distance. (WebCore::RealtimeMediaSource::selectSettings): Include the fitness distance of supported ideal constraints. (WebCore::RealtimeMediaSource::supportsConstraint): New. (WebCore::RealtimeMediaSource::applyConstraints): * platform/mediastream/RealtimeMediaSource.h: * platform/mock/MockRealtimeMediaSourceCenter.cpp: (WebCore::MockRealtimeMediaSourceCenter::validateRequestConstraints): Sort candidate sources by their fitness score. * platform/mock/MockRealtimeVideoSource.cpp: (WebCore::MockRealtimeVideoSource::initializeCapabilities): Each video source should support one facing mode, not both. Source/WebKit2: * UIProcess/UserMediaPermissionRequestManagerProxy.cpp: (WebKit::UserMediaPermissionRequestManagerProxy::requestUserMediaPermissionForFrame): When short-circuiting the user prompt because the page is already authorized, return the first audio and/or video device because so the page gets the one with the best fitness distance. LayoutTests: * fast/mediastream/MediaStream-video-element-displays-buffer-expected.txt: * fast/mediastream/MediaStream-video-element-displays-buffer.html: * fast/mediastream/MediaStreamTrack-getCapabilities-expected.txt: * fast/mediastream/apply-constraints-advanced-expected.txt: * fast/mediastream/apply-constraints-advanced.html: * fast/mediastream/apply-constraints-video-expected.txt: * fast/mediastream/apply-constraints-video.html: Canonical link: https://commits.webkit.org/187010@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@214385 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-03-24 23:27:14 +00:00
PASS isPixelTransparent(buffer) is false
PASS isPixelGray(buffer) is true
HTMLVideoElement frames do not update on iOS when src is a MediaStream blob https://bugs.webkit.org/show_bug.cgi?id=159833 <rdar://problem/27379487> Patch by George Ruan <gruan@apple.com> on 2016-07-26 Reviewed by Eric Carlson. Source/WebCore: Test: fast/mediastream/MediaStream-video-element-displays-buffer.html * WebCore.xcodeproj/project.pbxproj: * platform/cf/CoreMediaSoftLink.cpp: Add CMSampleBufferCreateReadyWithImageBuffer and CMVideoFormatDescriptionCreateForImageBuffer softlink. * platform/cf/CoreMediaSoftLink.h: Ditto. * platform/cocoa/CoreVideoSoftLink.cpp: Add CVPixelBufferCreate, kCVPixelBufferCGBitmapContextCompatibilityKey, and kCVPixelBufferCGImageCompatibilityKey. * platform/cocoa/CoreVideoSoftLink.h: Ditto. * platform/graphics/avfoundation/MediaSampleAVFObjC.h: Change create to return a Ref<T> instead of RefPtr<T>. * platform/graphics/avfoundation/objc/MediaPlayerPrivateMediaStreamAVFObjC.h: Make observer of MediaStreamTrackPrivate and make MediaPlayer use an AVSampleBufferDisplayLayer instead of CALayer. * platform/graphics/avfoundation/objc/MediaPlayerPrivateMediaStreamAVFObjC.mm: Ditto. (WebCore::MediaPlayerPrivateMediaStreamAVFObjC::~MediaPlayerPrivateMediaStreamAVFObjC): Clean up observers and AVSampleBufferDisplayLayer. (WebCore::MediaPlayerPrivateMediaStreamAVFObjC::isAvailable): Ensures AVSampleBufferDisplayLayer is available. (WebCore::MediaPlayerPrivateMediaStreamAVFObjC::enqueueAudioSampleBufferFromTrack): Placeholder. (WebCore::MediaPlayerPrivateMediaStreamAVFObjC::enqueueVideoSampleBufferFromTrack): Responsible for enqueuing sample buffers to the active video track. (WebCore::MediaPlayerPrivateMediaStreamAVFObjC::ensureLayer): Ensures that an AVSampleBufferDisplayLayer exists. (WebCore::MediaPlayerPrivateMediaStreamAVFObjC::destroyLayer): Destroys the AVSampleBufferDisplayLayer. (WebCore::MediaPlayerPrivateMediaStreamAVFObjC::platformLayer): Replace CALayer with AVSampleBufferDisplayLayer. (WebCore::MediaPlayerPrivateMediaStreamAVFObjC::currentDisplayMode): Ditto. (WebCore::MediaPlayerPrivateMediaStreamAVFObjC::play): Call updateReadyState as a deferred task. (WebCore::MediaPlayerPrivateMediaStreamAVFObjC::currentReadyState): readyState is bumped to HAVE_ENOUGH_DATA only when the MediaPlayerPrivateMediaStreamAVFObjC has received a media sample. (WebCore::MediaPlayerPrivateMediaStreamAVFObjC::sampleBufferUpdated): Called from MediaStreamTrackPrivate when a new SampleBuffer is available. (WebCore::updateTracksOfType): Manage adding and removing self as observer from tracks. (WebCore::MediaPlayerPrivateMediaStreamAVFObjC::updateTracks): Replace CALayer with AVSampleBufferDisplayLayer (WebCore::MediaPlayerPrivateMediaStreamAVFObjC::acceleratedRenderingStateChanged): Copied from MediaPlayerPrivateMediaSourceAVFObjC.mm (WebCore::MediaPlayerPrivateMediaStreamAVFObjC::load): Deleted CALayer. (WebCore::MediaPlayerPrivateMediaStreamAVFObjC::updateDisplayMode): Deleted process of updating CALayer. (WebCore::MediaPlayerPrivateMediaStreamAVFObjC::updateIntrinsicSize): Deleted CALayer. (WebCore::MediaPlayerPrivateMediaStreamAVFObjC::createPreviewLayers): Deleted. * platform/mediastream/MediaStreamPrivate.cpp: (WebCore::MediaStreamPrivate::updateActiveVideoTrack): Remove redundant check. * platform/mediastream/MediaStreamTrackPrivate.cpp: (WebCore::MediaStreamTrackPrivate::sourceHasMoreMediaData): Called from RealtimeMediaSource when a new SampleBuffer is available. * platform/mediastream/MediaStreamTrackPrivate.h: (WebCore::MediaStreamTrackPrivate::Observer::sampleBufferUpdated): Relays to MediaPlayerPrivateMediaStreamAVFObjC that a new SampleBuffer is available to enqueue to the AVSampleBufferDisplayLayer. * platform/mediastream/RealtimeMediaSource.cpp: (WebCore::RealtimeMediaSource::settingsDidChange): Fix grammatical mistake in function name settingsDidChanged(). (WebCore::RealtimeMediaSource::mediaDataUpdated): Relays to all observers that a new SampleBuffer is available. (WebCore::RealtimeMediaSource::settingsDidChanged): Deleted. * platform/mediastream/RealtimeMediaSource.h: * platform/mediastream/mac/AVVideoCaptureSource.mm: (WebCore::AVVideoCaptureSource::processNewFrame): Calls mediaDataUpdated when a new SampleBuffer is captured. * platform/mediastream/mac/MockRealtimeVideoSourceMac.h: * platform/mediastream/mac/MockRealtimeVideoSourceMac.mm: (WebCore::MockRealtimeVideoSourceMac::CMSampleBufferFromPixelBuffer): Convert CVPixelBuffer to CMSampleBuffer. (WebCore::MockRealtimeVideoSourceMac::pixelBufferFromCGImage): Convert CGImage to CVPixelBuffer. (WebCore::MockRealtimeVideoSourceMac::updateSampleBuffer): Creates a CMSampleBuffer from current imageBuffer and sends the CMSampleBuffer to MediaPlayerPrivateMediaStreamAVFObjC * platform/mock/MockRealtimeVideoSource.cpp: (WebCore::MockRealtimeVideoSource::setFrameRate): Fix grammar of settingsDidChanged() to settingsDidChange(). (WebCore::MockRealtimeVideoSource::setSize): Ditto. (WebCore::MockRealtimeVideoSource::generateFrame): Call updateSampleBuffer(). * platform/mock/MockRealtimeVideoSource.h: Change elapsedTime() from private to protected. (WebCore::MockRealtimeVideoSource::updateSampleBuffer): Overriden by MockRealtimeVideoSourceMac. LayoutTests: * fast/mediastream/MediaStream-video-element-displays-buffer-expected.txt: Added. * fast/mediastream/MediaStream-video-element-displays-buffer.html: Added. Checks that a video element with a mediastream source displays frames that are neither black or transparent. * fast/mediastream/resources/getUserMedia-helper.js: (setupVideoElementWithStream): Sets up video element with global variable mediastream. Canonical link: https://commits.webkit.org/178368@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@203739 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2016-07-26 21:52:27 +00:00
PASS successfullyParsed is true
TEST COMPLETE