haikuwebkit/LayoutTests/media/in-band-tracks.js

191 lines
5.8 KiB
JavaScript
Raw Permalink Normal View History

[GStreamer] media/track/in-band/ layout tests introduced in r154908 are failing https://bugs.webkit.org/show_bug.cgi?id=120665 Reviewed by Philippe Normand. Source/WebCore: Solved a deadlock when processing samples from a text track while the main thread is starting to play. In those circumstances, the text sink is set by Playbin as non-async, so the main thread calling MediaPlayerPrivateGStreamer::play() is blocked waiting for the preroll to finish. While prerolling, a text sample is detected in a non-main thread and sent to the main thread for processing. The current code waits in the non-main thread for the processing to finish, but that won't ever happen because the main thread is busy in play() waiting for the preroll to complete. Since the non-main thread doesn't need any result from the main thread, a simple change to avoid the wait is enough to solve the deadlock. There was also a bug preventing all the languages of the text tracks being detected on time. Sometimes the tag event specifying the track language emitted by the demuxer arrives too late to the webkittextcombiner sink pad (up to several seconds since sent!) and the language remains unset when the test checks for it. As a result, the test fails because the second track isn't reported to be in French language ("fr"). The solution to this is to find an earlier (more upstream) pad and listen there for the (sticky) tag events. There can be multiple tag events, so all of them are checked until one having the GST_TAG_LANGUAGE_CODE tag is found. Covered by existing tests (now unskipped). * platform/graphics/gstreamer/TextSinkGStreamer.cpp: (webkitTextSinkHandleSample): Don't wait for handleTextSample() to be completed in the main thread. This avoids the deadlock. * platform/graphics/gstreamer/TrackPrivateBaseGStreamer.cpp: (WebCore::findBestUpstreamPad): Find a pad more upstream than the current one. (WebCore::TrackPrivateBaseGStreamer::TrackPrivateBaseGStreamer): Call setPad() instead of setting it by hand here. (WebCore::TrackPrivateBaseGStreamer::setPad): Also manage m_bestUpstreamPad in parallel to m_pad. Add a probe to listen to the tag events instead of checking the "tags" property (the property wasn't present in the best upstream pad). (WebCore::TrackPrivateBaseGStreamer::disconnect): Remove the probe from m_bestUpstreamPad, instead of removing the old notify::tags listener from m_pad. (WebCore::TrackPrivateBaseGStreamer::tagsChanged): Check the sticky tag events until the language one is found (or the last event if no language event is available). * platform/graphics/gstreamer/TrackPrivateBaseGStreamer.h: Added m_bestUpstreamPad and m_eventProbe. Moved setPad() implementation to the cpp file. Tools: Fix media/track/in-band/tests Add libkate as dependency, so gst-plugins-bad detects it and can build the GstKateDec element. This isn't actually needed when using Flatpak (the dependency has been added there), but is still useful when using jhbuild and is also coherent with the need of other multimedia dependencies such as libvpx. * gtk/install-dependencies: * wpe/install-dependencies: LayoutTests: Removed track-in-band-mpegts-metadata.html because it relies on private streams on mpegts and those won't ever be exposed by tsdemux. Made testCuesAddedOnce() test more robust when finding text tracks. * media/content/user-private-streams.ts: Removed. * media/in-band-tracks.js: (pollProgress): Make sure that the function repeated at intervals is properly stopped in all the cases. (canplaythrough): More robust track detection. (testCuesAddedOnce): Ditto. * media/track/in-band/track-in-band-mpegts-metadata.html: Removed. * platform/glib/TestExpectations: Unskip media/track/in-band tests. Canonical link: https://commits.webkit.org/237022@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@276586 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-04-26 11:59:51 +00:00
inbandTrack1 = null;
[GStreamer] Support audio and video tracks https://bugs.webkit.org/show_bug.cgi?id=117039 Patch by Brendan Long <b.long@cablelabs.com> on 2013-11-01 Reviewed by Philippe Normand. Source/WebCore: Tests: media/track/audio/audio-track-mkv-vorbis-addtrack.html media/track/audio/audio-track-mkv-vorbis-enabled.html media/track/audio/audio-track-mkv-vorbis-language.html media/track/in-band/track-in-band-kate-ogg-addtrack.html media/track/in-band/track-in-band-srt-mkv-addtrack.html media/track/video/video-track-mkv-theora-addtrack.html media/track/video/video-track-mkv-theora-language.html media/track/video/video-track-mkv-theora-selected.html * GNUmakefile.list.am: Add audio and video track files. * PlatformEfl.cmake: Same. * html/HTMLMediaElement.cpp: (WebCore::HTMLMediaElement::audioTrackEnabledChanged): Schedule "change" event. (WebCore::HTMLMediaElement::videoTrackSelectedChanged): Same. * html/track/AudioTrack.cpp: (WebCore::AudioTrack::setEnabled): Call m_private->setEnabled (WebCore::AudioTrack::enabledChanged): Added callback. (WebCore::AudioTrack::labelChanged): Same. (WebCore::AudioTrack::languageChanged): Same. (WebCore::AudioTrack::willRemoveAudioTrackPrivate): Use ASSERT_UNUSED for consistence. * html/track/AudioTrack.h: Add new enabled, label and language callbacks. * html/track/VideoTrack.cpp: (WebCore::VideoTrack::setSelected): Call m_private->setEnabled (WebCore::VideoTrack::selectedChanged): Added callback. (WebCore::VideoTrack::labelChanged): Same. (WebCore::VideoTrack::languageChanged): Same. (WebCore::VideoTrack::willRemoveVideoTrackPrivate): Use ASSERT_UNUSED for consistence. * html/track/VideoTrack.h: Add new selected, label and language callbacks. * platform/graphics/AudioTrackPrivate.h: (WebCore::AudioTrackPrivate::setEnabled): Call m_client->enabledChanged * platform/graphics/VideoTrackPrivate.h: (WebCore::VideoTrackPrivate::setSelected): Call m_client->selectedChanged * platform/graphics/gstreamer/InbandTextTrackPrivateGStreamer.cpp: (WebCore::InbandTextTrackPrivateGStreamer::InbandTextTrackPrivateGStreamer): Add ASSERT(m_pad) (WebCore::InbandTextTrackPrivateGStreamer::notifyTrackOfTagsChanged): Look at all tag events instead of just the first one. * platform/graphics/gstreamer/MediaPlayerPrivateGStreamer.cpp: (WebCore::mediaPlayerPrivateVideoSinkCapsChangedCallback): Separated callback when caps change from callback when number of videos change. (WebCore::mediaPlayerPrivateVideoCapsChangeTimeoutCallback): Same. (WebCore::MediaPlayerPrivateGStreamer::MediaPlayerPrivateGStreamer): Initialize m_videoCapsTimerHandler. (WebCore::MediaPlayerPrivateGStreamer::~MediaPlayerPrivateGStreamer): Disconnect audio and video tracks and remove callbacks. (WebCore::MediaPlayerPrivateGStreamer::videoCapsChanged): Separated callback when caps change from callback when number of videos change. (WebCore::MediaPlayerPrivateGStreamer::notifyPlayerOfVideo): Create video tracks for each stream. (WebCore::MediaPlayerPrivateGStreamer::notifyPlayerOfVideoCaps): Separated callback when caps change from callback when number of videos change. (WebCore::MediaPlayerPrivateGStreamer::notifyPlayerOfAudio): Create audio tracks for each stream. * platform/graphics/gstreamer/MediaPlayerPrivateGStreamer.h: Add audio and video tracks, and a pointer to keep track of the default audio stream. * platform/graphics/gstreamer/AudioTrackPrivateGStreamer.cpp: Added. * platform/graphics/gstreamer/AudioTrackPrivateGStreamer.h: Added. * platform/graphics/gstreamer/GRefPtrGStreamer.cpp: Add GRefPtr<GstTagList> * platform/graphics/gstreamer/TrackPrivateBaseGStreamer.cpp: Added, handles tags and "active" property of audio and video tracks. * platform/graphics/gstreamer/TrackPrivateBaseGStreamer.h: Added. * platform/graphics/gstreamer/VideoTrackPrivateGStreamer.cpp: Added. * platform/graphics/gstreamer/VideoTrackPrivateGStreamer.h: Added. LayoutTests: * media/content/two-audio-and-video-tracks.mkv: Added. * media/in-band-tracks.js: Renamed from LayoutTests/media/in-band-cues.js. (testAddTrack): Tests addtrack events. (testAttribute): Added "type" parameter so we can use this for audio and video tracks. (testTextTrackMode): Renamed to make it clear that this only applies to text tracks. (testCueStyle): Same. (testTextTrackOrder): Same. * media/track/audio/audio-track-mkv-vorbis-addtrack-expected.txt: Added. * media/track/audio/audio-track-mkv-vorbis-addtrack.html: Copied from LayoutTests/media/track/in-band/track-in-band-srt-mkv-track-order.html. * media/track/audio/audio-track-mkv-vorbis-enabled-expected.txt: Added. * media/track/audio/audio-track-mkv-vorbis-enabled.html: Added. * media/track/audio/audio-track-mkv-vorbis-language-expected.txt: Added. * media/track/audio/audio-track-mkv-vorbis-language.html: Copied from LayoutTests/media/track/in-band/track-in-band-kate-ogg-track-order.html. * media/track/in-band/track-in-band-kate-ogg-addtrack-expected.txt: Added. * media/track/in-band/track-in-band-kate-ogg-addtrack.html: Copied from LayoutTests/media/track/in-band/track-in-band-srt-mkv-track-order.html. * media/track/in-band/track-in-band-kate-ogg-cues-added-once.html: Added. * media/track/in-band/track-in-band-kate-ogg-kind.html: Added. * media/track/in-band/track-in-band-kate-ogg-language.html: Added. * media/track/in-band/track-in-band-kate-ogg-mode.html: Added. * media/track/in-band/track-in-band-kate-ogg-style.html: Added. * media/track/in-band/track-in-band-kate-ogg-track-order.html: Added. * media/track/in-band/track-in-band-srt-mkv-addtrack-expected.txt: Added. * media/track/in-band/track-in-band-srt-mkv-addtrack.html: Copied from LayoutTests/media/track/in-band/track-in-band-srt-mkv-track-order.html. * media/track/in-band/track-in-band-srt-mkv-cues-added-once.html: Added. * media/track/in-band/track-in-band-srt-mkv-kind.html: Added. * media/track/in-band/track-in-band-srt-mkv-language.html: Added. * media/track/in-band/track-in-band-srt-mkv-mode.html: Added. * media/track/in-band/track-in-band-srt-mkv-style.html: Added. * media/track/in-band/track-in-band-srt-mkv-track-order.html: Added. * media/track/video/video-track-mkv-theora-addtrack-expected.txt: Added. * media/track/video/video-track-mkv-theora-addtrack.html: Copied from LayoutTests/media/track/in-band/track-in-band-srt-mkv-track-order.html. * media/track/video/video-track-mkv-theora-language-expected.txt: Added. * media/track/video/video-track-mkv-theora-language.html: Copied from LayoutTests/media/track/in-band/track-in-band-kate-ogg-track-order.html. * media/track/video/video-track-mkv-theora-selected-expected.txt: Added. * media/track/video/video-track-mkv-theora-selected.html: Added. * platform/mac/TestExpectations: Ignore audio and video track tests on mac. Canonical link: https://commits.webkit.org/141792@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@158436 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2013-11-01 16:30:03 +00:00
function testTrackListContainsTrack(listStr, trackStr)
{
var list = eval(listStr);
var track = eval(trackStr);
var found = false;
for (var i = 0; i < list.length; ++i) {
if (list[i] == track) {
found = true;
break;
}
}
reportExpected(found, listStr, " contains ", trackStr, list);
}
function compareTracks(track1, track2)
{
var equal = (eval(track1) == eval(track2));
reportExpected(equal, track1, "==", track2, track1);
}
function testAddTrack(uri, type)
{
var addtrackEventCount = 0;
function trackAdded(event)
{
consoleWrite("EVENT(" + event.type + ")");
++addtrackEventCount;
/* Don't make assumptions about the order of track events. If you know
* the expected order, it should be tested separately. */
testTrackListContainsTrack("video." + type + "Tracks", "event.track");
consoleWrite("");
}
findMediaElement();
var tracks = eval("video." + type + "Tracks");
tracks.addEventListener("addtrack", trackAdded);
video.src = uri;
waitForEventAndEnd('canplaythrough');
}
function testAttribute(uri, type, attribute, values)
Source/WebCore: [GStreamer] support in-band text tracks https://bugs.webkit.org/show_bug.cgi?id=103771 Patch by Brendan Long <b.long@cablelabs.com> on 2013-08-30 Reviewed by Eric Carlson. Tests: New tests added because existing tests were too specific. media/track/in-band/track-in-band-kate-ogg-cues-added-once.html media/track/in-band/track-in-band-kate-ogg-kind.html media/track/in-band/track-in-band-kate-ogg-language.html media/track/in-band/track-in-band-kate-ogg-mode.html media/track/in-band/track-in-band-kate-ogg-style.html media/track/in-band/track-in-band-kate-ogg-track-order.html media/track/in-band/track-in-band-srt-mkv-cues-added-once.html media/track/in-band/track-in-band-srt-mkv-kind.html media/track/in-band/track-in-band-srt-mkv-language.html media/track/in-band/track-in-band-srt-mkv-mode.html media/track/in-band/track-in-band-srt-mkv-style.html media/track/in-band/track-in-band-srt-mkv-track-order.html * CMakeLists.txt: Add InbandTextTrackPrivateGStreamer, InbandGenericTextTrack, InbandWebVTTTextTrack, and TextCombinerGStreamer files. * GNUmakefile.list.am: Same. * PlatformEfl.cmake: Same. * Target.pri: Same. * WebCore.vcxproj/WebCore.vcxproj: Same. * WebCore.vcxproj/WebCore.vcxproj.filters: Same. * WebCore.xcodeproj/project.pbxproj: Same. * html/track/InbandGenericTextTrack.cpp: Split out code for handling generic cues. (WebCore::GenericTextTrackCueMap::GenericTextTrackCueMap): Move from InbandTextTrack. (WebCore::GenericTextTrackCueMap::~GenericTextTrackCueMap): Same. (WebCore::GenericTextTrackCueMap::add): Same. (WebCore::GenericTextTrackCueMap::find): Same. (WebCore::GenericTextTrackCueMap::remove): Same. (WebCore::InbandGenericTextTrack::create): Same. (WebCore::InbandGenericTextTrack::updateCueFromCueData): Same. (WebCore::InbandGenericTextTrack::addGenericCue): Same. (WebCore::InbandGenericTextTrack::updateGenericCue): Same. (WebCore::InbandGenericTextTrack::removeGenericCue): Same. (WebCore::InbandGenericTextTrack::removeCue): Same. (WebCore::InbandGenericTextTrack::InbandGenericTextTrack): Empty. (WebCore::InbandGenericTextTrack::~InbandGenericTextTrack): Empty. * html/track/InbandGenericTextTrack.h: Copied from Source/WebCore/html/track/InbandTextTrack.h. The only addition is the ASSERT_NOT_REACHED() for WebVTT cues. * html/track/InbandTextTrack.cpp: Add label and language changed callbacks. (WebCore::InbandTextTrack::create): Return a generic or WebVTT text track based on the private CueFormat. (WebCore::InbandTextTrack::labelChanged): Added. (WebCore::InbandTextTrack::languageChanged): Added. * html/track/InbandTextTrack.h: Add label and language changed callbacks. * html/track/InbandWebVTTTextTrack.cpp: Added, based on InbandTextTrack. (WebCore::InbandWebVTTTextTrack::create): Same. (WebCore::InbandWebVTTTextTrack::InbandWebVTTTextTrack): Empty. (WebCore::InbandWebVTTTextTrack::~InbandWebVTTTextTrack): Empty. (WebCore::InbandWebVTTTextTrack::parseWebVTTCueData): Sends data to a WebVTTParser. (WebCore::InbandWebVTTTextTrack::newCuesParsed): Adds cues when WebVTTParser parses them. (WebCore::InbandWebVTTTextTrack::fileFailedToParse): Prints a warning when WebVTTParser has a problem. * html/track/InbandWebVTTTextTrack.h: Added. * platform/graphics/InbandTextTrackPrivate.h: (WebCore::InbandTextTrackPrivate::cueFormat): For determining if the track will have generic or WebVTT cues. (WebCore::InbandTextTrackPrivate::InbandTextTrackPrivate): Pass CueFormat in the constructor. * platform/graphics/InbandTextTrackPrivateClient.h: Same. * platform/graphics/avfoundation/InbandTextTrackPrivateAVF.cpp: (WebCore::InbandTextTrackPrivateAVF::InbandTextTrackPrivateAVF): Pass CueFormat (Generic) to InbandTextTrackPrivate. * platform/graphics/gstreamer/GRefPtrGStreamer.cpp: Add GRefPtr specializations for GstSample and GstEvent. * platform/graphics/gstreamer/GRefPtrGStreamer.h: Same. * platform/graphics/gstreamer/GStreamerUtilities.h: Add WARN_MEDIA_MESSAGE. * platform/graphics/gstreamer/GStreamerVersioning.h: Add a function to check GStreamer version at runtime. * platform/graphics/gstreamer/InbandTextTrackPrivateGStreamer.cpp: Added. (WebCore::textTrackPrivateEventCallback): Watches for tag and stream start events. (WebCore::textTrackPrivateSampleTimeoutCallback): See notifyTrackOfSample(); (WebCore::textTrackPrivateStreamTimeoutCallback): See notifyTrackOfStreamChanged(); (WebCore::textTrackPrivateTagsChangeTimeoutCallback): See notifyTrackOfTagsChanged(); (WebCore::InbandTextTrackPrivateGStreamer::InbandTextTrackPrivateGStreamer): Initializes tags and stream and sets up event callback. (WebCore::InbandTextTrackPrivateGStreamer::~InbandTextTrackPrivateGStreamer): Calls disconnect: (WebCore::InbandTextTrackPrivateGStreamer::disconnect): Removes signal handlers and frees some memory. (WebCore::InbandTextTrackPrivateGStreamer::handleSample): Adds samples to a list and sets up callback. (WebCore::InbandTextTrackPrivateGStreamer::streamChanged): Sets up callback. (WebCore::InbandTextTrackPrivateGStreamer::tagsChanged): Same. (WebCore::InbandTextTrackPrivateGStreamer::notifyTrackOfSample): Parses all queued samples with WebVTTParser. (WebCore::InbandTextTrackPrivateGStreamer::notifyTrackOfStreamChanged): Keeps track of current stream. (WebCore::InbandTextTrackPrivateGStreamer::notifyTrackOfTagsChanged): Sets label and language from tags. * platform/graphics/gstreamer/InbandTextTrackPrivateGStreamer.h: Added. (WebCore::InbandTextTrackPrivateGStreamer::create): Basic RefPtr create function. (WebCore::InbandTextTrackPrivateGStreamer::pad): Returns the pad this track is associated with (used to determine if a playbin text stream has already been associated with a text track). (WebCore::InbandTextTrackPrivateGStreamer::setIndex): Sets the track index (used for sorting). (WebCore::InbandTextTrackPrivateGStreamer::streamId): Returns the stream ID (used to handle new samples). * platform/graphics/gstreamer/MediaPlayerPrivateGStreamer.cpp: (WebCore::mediaPlayerPrivateTextChangedCallback): Called for playbin "text-changed" event. See textChanged(). (WebCore::mediaPlayerPrivateTextChangeTimeoutCallback): See notifyPlayerOfText(). (WebCore::mediaPlayerPrivateNewTextSampleCallback): See newTextSample(). (WebCore::MediaPlayerPrivateGStreamer::MediaPlayerPrivateGStreamer): Initialize m_textTimerHandler. (WebCore::MediaPlayerPrivateGStreamer::~MediaPlayerPrivateGStreamer): Disconnect text tracks (they don't necessarily get freed here, since a script could hold a reference). (WebCore::MediaPlayerPrivateGStreamer::textChanged): Setup callback for notifyPlayerOfText. (WebCore::MediaPlayerPrivateGStreamer::notifyPlayerOfText): Create text tracks. (WebCore::MediaPlayerPrivateGStreamer::newTextSample): Handle new samples by giving them to a text track with a matching stream. This method is syncryonous because we need to get the stream start sticky event immediately. * platform/graphics/gstreamer/MediaPlayerPrivateGStreamer.h: * platform/graphics/gstreamer/TextCombinerGStreamer.cpp: Added. This element forwards buffers from all of its input pads, but also converts plain text to WebVTT as needed. (webkit_text_combiner_init): Setup internal funnel. (webkitTextCombinerPadEvent): If the caps are plain text make sure we have a webvttenv, otherwise connect directly to the funnel. (webkitTextCombinerRequestNewPad): Setup ghostpad and event callback. (webkitTextCombinerReleasePad): Release pad and optional associated webvttenc. (webkit_text_combiner_class_init): Setup pad templates and request/release pad functions. (webkitTextCombinerNew): Returns a new WebKitTextCombiner. * platform/graphics/gstreamer/TextCombinerGStreamer.h: Added. * platform/graphics/gstreamer/TextSinkGStreamer.cpp: Added. (webkit_text_sink_init): Set sync=false. (webkitTextSinkGetProperty): Ignore sync property. (webkitTextSinkSetProperty): Same. (webkitTextSinkQuery): Ignore position and duration queries, forward everything else to appsink. (webkit_text_sink_class_init): Setup property and query functions. (webkitTextSinkNew): Return a new WebKitTextSink. * platform/graphics/gstreamer/TextSinkGStreamer.h: Added. LayoutTests: <https://webkit.org/b/103771> [GStreamer] support in-band text tracks Patch by Brendan Long <b.long@cablelabs.com> on 2013-08-30 Reviewed by Eric Carlson. * media/content/counting-subtitled-kate.ogv: Added. * media/content/counting-subtitled-srt.mkv: Added. * media/in-band-cues.js: Added. * media/track/in-band/track-in-band-kate-ogg-cues-added-once-expected.txt: Added. * media/track/in-band/track-in-band-kate-ogg-cues-added-once.html: Added. * media/track/in-band/track-in-band-kate-ogg-kind-expected.txt: Added. * media/track/in-band/track-in-band-kate-ogg-kind.html: Added. * media/track/in-band/track-in-band-kate-ogg-language-expected.txt: Added. * media/track/in-band/track-in-band-kate-ogg-language.html: Added. * media/track/in-band/track-in-band-kate-ogg-mode-expected.txt: Added. * media/track/in-band/track-in-band-kate-ogg-mode.html: Added. * media/track/in-band/track-in-band-kate-ogg-style-expected.txt: Added. * media/track/in-band/track-in-band-kate-ogg-style.html: Added. * media/track/in-band/track-in-band-kate-ogg-track-order-expected.txt: Added. * media/track/in-band/track-in-band-kate-ogg-track-order.html: Added. * media/track/in-band/track-in-band-srt-mkv-cues-added-once-expected.txt: Added. * media/track/in-band/track-in-band-srt-mkv-cues-added-once.html: Added. * media/track/in-band/track-in-band-srt-mkv-kind-expected.txt: Added. * media/track/in-band/track-in-band-srt-mkv-kind.html: Added. * media/track/in-band/track-in-band-srt-mkv-language-expected.txt: Added. * media/track/in-band/track-in-band-srt-mkv-language.html: Added. * media/track/in-band/track-in-band-srt-mkv-mode-expected.txt: Added. * media/track/in-band/track-in-band-srt-mkv-mode.html: Added. * media/track/in-band/track-in-band-srt-mkv-style-expected.txt: Added. * media/track/in-band/track-in-band-srt-mkv-style.html: Added. * media/track/in-band/track-in-band-srt-mkv-track-order-expected.txt: Added. * media/track/in-band/track-in-band-srt-mkv-track-order.html: Added. * platform/mac/TestExpectations: Skip MKV and OGG tests. Canonical link: https://commits.webkit.org/138536@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@154908 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2013-08-30 20:17:27 +00:00
{
function canplaythrough()
{
consoleWrite("<br><i>** Check in-band kind attributes</i>");
if (values instanceof Array) {
testExpected(`video.${type}Tracks.length`, values.length);
for (let i = 0; i < values.length; ++i)
testExpected(`video.${type}Tracks[${i}].${attribute}`, values[i]);
} else {
testExpected(`video.${type}Tracks.length`, Object.keys(values).length);
for (let id in values)
testExpected(`video.${type}Tracks.getTrackById('${id}').${attribute}`, values[id]);
}
Source/WebCore: [GStreamer] support in-band text tracks https://bugs.webkit.org/show_bug.cgi?id=103771 Patch by Brendan Long <b.long@cablelabs.com> on 2013-08-30 Reviewed by Eric Carlson. Tests: New tests added because existing tests were too specific. media/track/in-band/track-in-band-kate-ogg-cues-added-once.html media/track/in-band/track-in-band-kate-ogg-kind.html media/track/in-band/track-in-band-kate-ogg-language.html media/track/in-band/track-in-band-kate-ogg-mode.html media/track/in-band/track-in-band-kate-ogg-style.html media/track/in-band/track-in-band-kate-ogg-track-order.html media/track/in-band/track-in-band-srt-mkv-cues-added-once.html media/track/in-band/track-in-band-srt-mkv-kind.html media/track/in-band/track-in-band-srt-mkv-language.html media/track/in-band/track-in-band-srt-mkv-mode.html media/track/in-band/track-in-band-srt-mkv-style.html media/track/in-band/track-in-band-srt-mkv-track-order.html * CMakeLists.txt: Add InbandTextTrackPrivateGStreamer, InbandGenericTextTrack, InbandWebVTTTextTrack, and TextCombinerGStreamer files. * GNUmakefile.list.am: Same. * PlatformEfl.cmake: Same. * Target.pri: Same. * WebCore.vcxproj/WebCore.vcxproj: Same. * WebCore.vcxproj/WebCore.vcxproj.filters: Same. * WebCore.xcodeproj/project.pbxproj: Same. * html/track/InbandGenericTextTrack.cpp: Split out code for handling generic cues. (WebCore::GenericTextTrackCueMap::GenericTextTrackCueMap): Move from InbandTextTrack. (WebCore::GenericTextTrackCueMap::~GenericTextTrackCueMap): Same. (WebCore::GenericTextTrackCueMap::add): Same. (WebCore::GenericTextTrackCueMap::find): Same. (WebCore::GenericTextTrackCueMap::remove): Same. (WebCore::InbandGenericTextTrack::create): Same. (WebCore::InbandGenericTextTrack::updateCueFromCueData): Same. (WebCore::InbandGenericTextTrack::addGenericCue): Same. (WebCore::InbandGenericTextTrack::updateGenericCue): Same. (WebCore::InbandGenericTextTrack::removeGenericCue): Same. (WebCore::InbandGenericTextTrack::removeCue): Same. (WebCore::InbandGenericTextTrack::InbandGenericTextTrack): Empty. (WebCore::InbandGenericTextTrack::~InbandGenericTextTrack): Empty. * html/track/InbandGenericTextTrack.h: Copied from Source/WebCore/html/track/InbandTextTrack.h. The only addition is the ASSERT_NOT_REACHED() for WebVTT cues. * html/track/InbandTextTrack.cpp: Add label and language changed callbacks. (WebCore::InbandTextTrack::create): Return a generic or WebVTT text track based on the private CueFormat. (WebCore::InbandTextTrack::labelChanged): Added. (WebCore::InbandTextTrack::languageChanged): Added. * html/track/InbandTextTrack.h: Add label and language changed callbacks. * html/track/InbandWebVTTTextTrack.cpp: Added, based on InbandTextTrack. (WebCore::InbandWebVTTTextTrack::create): Same. (WebCore::InbandWebVTTTextTrack::InbandWebVTTTextTrack): Empty. (WebCore::InbandWebVTTTextTrack::~InbandWebVTTTextTrack): Empty. (WebCore::InbandWebVTTTextTrack::parseWebVTTCueData): Sends data to a WebVTTParser. (WebCore::InbandWebVTTTextTrack::newCuesParsed): Adds cues when WebVTTParser parses them. (WebCore::InbandWebVTTTextTrack::fileFailedToParse): Prints a warning when WebVTTParser has a problem. * html/track/InbandWebVTTTextTrack.h: Added. * platform/graphics/InbandTextTrackPrivate.h: (WebCore::InbandTextTrackPrivate::cueFormat): For determining if the track will have generic or WebVTT cues. (WebCore::InbandTextTrackPrivate::InbandTextTrackPrivate): Pass CueFormat in the constructor. * platform/graphics/InbandTextTrackPrivateClient.h: Same. * platform/graphics/avfoundation/InbandTextTrackPrivateAVF.cpp: (WebCore::InbandTextTrackPrivateAVF::InbandTextTrackPrivateAVF): Pass CueFormat (Generic) to InbandTextTrackPrivate. * platform/graphics/gstreamer/GRefPtrGStreamer.cpp: Add GRefPtr specializations for GstSample and GstEvent. * platform/graphics/gstreamer/GRefPtrGStreamer.h: Same. * platform/graphics/gstreamer/GStreamerUtilities.h: Add WARN_MEDIA_MESSAGE. * platform/graphics/gstreamer/GStreamerVersioning.h: Add a function to check GStreamer version at runtime. * platform/graphics/gstreamer/InbandTextTrackPrivateGStreamer.cpp: Added. (WebCore::textTrackPrivateEventCallback): Watches for tag and stream start events. (WebCore::textTrackPrivateSampleTimeoutCallback): See notifyTrackOfSample(); (WebCore::textTrackPrivateStreamTimeoutCallback): See notifyTrackOfStreamChanged(); (WebCore::textTrackPrivateTagsChangeTimeoutCallback): See notifyTrackOfTagsChanged(); (WebCore::InbandTextTrackPrivateGStreamer::InbandTextTrackPrivateGStreamer): Initializes tags and stream and sets up event callback. (WebCore::InbandTextTrackPrivateGStreamer::~InbandTextTrackPrivateGStreamer): Calls disconnect: (WebCore::InbandTextTrackPrivateGStreamer::disconnect): Removes signal handlers and frees some memory. (WebCore::InbandTextTrackPrivateGStreamer::handleSample): Adds samples to a list and sets up callback. (WebCore::InbandTextTrackPrivateGStreamer::streamChanged): Sets up callback. (WebCore::InbandTextTrackPrivateGStreamer::tagsChanged): Same. (WebCore::InbandTextTrackPrivateGStreamer::notifyTrackOfSample): Parses all queued samples with WebVTTParser. (WebCore::InbandTextTrackPrivateGStreamer::notifyTrackOfStreamChanged): Keeps track of current stream. (WebCore::InbandTextTrackPrivateGStreamer::notifyTrackOfTagsChanged): Sets label and language from tags. * platform/graphics/gstreamer/InbandTextTrackPrivateGStreamer.h: Added. (WebCore::InbandTextTrackPrivateGStreamer::create): Basic RefPtr create function. (WebCore::InbandTextTrackPrivateGStreamer::pad): Returns the pad this track is associated with (used to determine if a playbin text stream has already been associated with a text track). (WebCore::InbandTextTrackPrivateGStreamer::setIndex): Sets the track index (used for sorting). (WebCore::InbandTextTrackPrivateGStreamer::streamId): Returns the stream ID (used to handle new samples). * platform/graphics/gstreamer/MediaPlayerPrivateGStreamer.cpp: (WebCore::mediaPlayerPrivateTextChangedCallback): Called for playbin "text-changed" event. See textChanged(). (WebCore::mediaPlayerPrivateTextChangeTimeoutCallback): See notifyPlayerOfText(). (WebCore::mediaPlayerPrivateNewTextSampleCallback): See newTextSample(). (WebCore::MediaPlayerPrivateGStreamer::MediaPlayerPrivateGStreamer): Initialize m_textTimerHandler. (WebCore::MediaPlayerPrivateGStreamer::~MediaPlayerPrivateGStreamer): Disconnect text tracks (they don't necessarily get freed here, since a script could hold a reference). (WebCore::MediaPlayerPrivateGStreamer::textChanged): Setup callback for notifyPlayerOfText. (WebCore::MediaPlayerPrivateGStreamer::notifyPlayerOfText): Create text tracks. (WebCore::MediaPlayerPrivateGStreamer::newTextSample): Handle new samples by giving them to a text track with a matching stream. This method is syncryonous because we need to get the stream start sticky event immediately. * platform/graphics/gstreamer/MediaPlayerPrivateGStreamer.h: * platform/graphics/gstreamer/TextCombinerGStreamer.cpp: Added. This element forwards buffers from all of its input pads, but also converts plain text to WebVTT as needed. (webkit_text_combiner_init): Setup internal funnel. (webkitTextCombinerPadEvent): If the caps are plain text make sure we have a webvttenv, otherwise connect directly to the funnel. (webkitTextCombinerRequestNewPad): Setup ghostpad and event callback. (webkitTextCombinerReleasePad): Release pad and optional associated webvttenc. (webkit_text_combiner_class_init): Setup pad templates and request/release pad functions. (webkitTextCombinerNew): Returns a new WebKitTextCombiner. * platform/graphics/gstreamer/TextCombinerGStreamer.h: Added. * platform/graphics/gstreamer/TextSinkGStreamer.cpp: Added. (webkit_text_sink_init): Set sync=false. (webkitTextSinkGetProperty): Ignore sync property. (webkitTextSinkSetProperty): Same. (webkitTextSinkQuery): Ignore position and duration queries, forward everything else to appsink. (webkit_text_sink_class_init): Setup property and query functions. (webkitTextSinkNew): Return a new WebKitTextSink. * platform/graphics/gstreamer/TextSinkGStreamer.h: Added. LayoutTests: <https://webkit.org/b/103771> [GStreamer] support in-band text tracks Patch by Brendan Long <b.long@cablelabs.com> on 2013-08-30 Reviewed by Eric Carlson. * media/content/counting-subtitled-kate.ogv: Added. * media/content/counting-subtitled-srt.mkv: Added. * media/in-band-cues.js: Added. * media/track/in-band/track-in-band-kate-ogg-cues-added-once-expected.txt: Added. * media/track/in-band/track-in-band-kate-ogg-cues-added-once.html: Added. * media/track/in-band/track-in-band-kate-ogg-kind-expected.txt: Added. * media/track/in-band/track-in-band-kate-ogg-kind.html: Added. * media/track/in-band/track-in-band-kate-ogg-language-expected.txt: Added. * media/track/in-band/track-in-band-kate-ogg-language.html: Added. * media/track/in-band/track-in-band-kate-ogg-mode-expected.txt: Added. * media/track/in-band/track-in-band-kate-ogg-mode.html: Added. * media/track/in-band/track-in-band-kate-ogg-style-expected.txt: Added. * media/track/in-band/track-in-band-kate-ogg-style.html: Added. * media/track/in-band/track-in-band-kate-ogg-track-order-expected.txt: Added. * media/track/in-band/track-in-band-kate-ogg-track-order.html: Added. * media/track/in-band/track-in-band-srt-mkv-cues-added-once-expected.txt: Added. * media/track/in-band/track-in-band-srt-mkv-cues-added-once.html: Added. * media/track/in-band/track-in-band-srt-mkv-kind-expected.txt: Added. * media/track/in-band/track-in-band-srt-mkv-kind.html: Added. * media/track/in-band/track-in-band-srt-mkv-language-expected.txt: Added. * media/track/in-band/track-in-band-srt-mkv-language.html: Added. * media/track/in-band/track-in-band-srt-mkv-mode-expected.txt: Added. * media/track/in-band/track-in-band-srt-mkv-mode.html: Added. * media/track/in-band/track-in-band-srt-mkv-style-expected.txt: Added. * media/track/in-band/track-in-band-srt-mkv-style.html: Added. * media/track/in-band/track-in-band-srt-mkv-track-order-expected.txt: Added. * media/track/in-band/track-in-band-srt-mkv-track-order.html: Added. * platform/mac/TestExpectations: Skip MKV and OGG tests. Canonical link: https://commits.webkit.org/138536@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@154908 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2013-08-30 20:17:27 +00:00
consoleWrite("");
endTest();
}
findMediaElement();
video.src = uri;
waitForEvent('canplaythrough', canplaythrough);
}
[GStreamer] Support "chapter" text tracks https://bugs.webkit.org/show_bug.cgi?id=122000 Patch by Brendan Long <b.long@cablelabs.com> on 2013-10-01 Reviewed by Eric Carlson. Source/WebCore: Test: media/track/in-band/track-in-band-mkv-chapters.html * platform/graphics/gstreamer/InbandMetadataTextTrackPrivateGStreamer.h: New class, needed for cues that aren't associated with GstPads. * platform/graphics/gstreamer/MediaPlayerPrivateGStreamer.cpp: (WebCore::MediaPlayerPrivateGStreamer::handleMessage): Check for table of contents messages. (WebCore::MediaPlayerPrivateGStreamer::processTableOfContents): Parse table of contents messages into text tracks. (WebCore::MediaPlayerPrivateGStreamer::processTableOfContentsEntry): Recursive method for parsing all of the TOC entries. * platform/graphics/gstreamer/MediaPlayerPrivateGStreamer.h: Add the processTableOfContents functions. LayoutTests: * media/content/counting-subtitled-srt.mkv: Added chapters metadata. * media/track/in-band/track-in-band-mkv-chapters-expected.txt: New test, similar to cues-added-one, but uses "chapters" track. * media/track/in-band/track-in-band-mkv-chapters.html: Same. * media/in-band-cues.js: Updated tests to handle the new "chapters" track. * media/track/in-band/track-in-band-kate-ogg-cues-added-once-expected.txt: Same. * media/track/in-band/track-in-band-kate-ogg-cues-added-once.html: Same. * media/track/in-band/track-in-band-kate-ogg-mode-expected.txt: Same. * media/track/in-band/track-in-band-kate-ogg-mode.html: Same. * media/track/in-band/track-in-band-kate-ogg-style-expected.txt: Same. * media/track/in-band/track-in-band-kate-ogg-track-order-expected.txt: Same. * media/track/in-band/track-in-band-kate-ogg-track-order.html: Same. * media/track/in-band/track-in-band-srt-mkv-cues-added-once-expected.txt: Same. * media/track/in-band/track-in-band-srt-mkv-cues-added-once.html: Same. * media/track/in-band/track-in-band-srt-mkv-kind-expected.txt: Same. * media/track/in-band/track-in-band-srt-mkv-kind.html: Same. * media/track/in-band/track-in-band-srt-mkv-language-expected.txt: Same. * media/track/in-band/track-in-band-srt-mkv-language.html: Same. * media/track/in-band/track-in-band-srt-mkv-mode-expected.txt: Same. * media/track/in-band/track-in-band-srt-mkv-mode.html: Same. * media/track/in-band/track-in-band-srt-mkv-style-expected.txt: Same. * media/track/in-band/track-in-band-srt-mkv-track-order-expected.txt: Same. * media/track/in-band/track-in-band-srt-mkv-track-order.html: Same. Canonical link: https://commits.webkit.org/140209@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@156713 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2013-10-01 16:26:52 +00:00
function testCuesAddedOnce(uri, kind)
Source/WebCore: [GStreamer] support in-band text tracks https://bugs.webkit.org/show_bug.cgi?id=103771 Patch by Brendan Long <b.long@cablelabs.com> on 2013-08-30 Reviewed by Eric Carlson. Tests: New tests added because existing tests were too specific. media/track/in-band/track-in-band-kate-ogg-cues-added-once.html media/track/in-band/track-in-band-kate-ogg-kind.html media/track/in-band/track-in-band-kate-ogg-language.html media/track/in-band/track-in-band-kate-ogg-mode.html media/track/in-band/track-in-band-kate-ogg-style.html media/track/in-band/track-in-band-kate-ogg-track-order.html media/track/in-band/track-in-band-srt-mkv-cues-added-once.html media/track/in-band/track-in-band-srt-mkv-kind.html media/track/in-band/track-in-band-srt-mkv-language.html media/track/in-band/track-in-band-srt-mkv-mode.html media/track/in-band/track-in-band-srt-mkv-style.html media/track/in-band/track-in-band-srt-mkv-track-order.html * CMakeLists.txt: Add InbandTextTrackPrivateGStreamer, InbandGenericTextTrack, InbandWebVTTTextTrack, and TextCombinerGStreamer files. * GNUmakefile.list.am: Same. * PlatformEfl.cmake: Same. * Target.pri: Same. * WebCore.vcxproj/WebCore.vcxproj: Same. * WebCore.vcxproj/WebCore.vcxproj.filters: Same. * WebCore.xcodeproj/project.pbxproj: Same. * html/track/InbandGenericTextTrack.cpp: Split out code for handling generic cues. (WebCore::GenericTextTrackCueMap::GenericTextTrackCueMap): Move from InbandTextTrack. (WebCore::GenericTextTrackCueMap::~GenericTextTrackCueMap): Same. (WebCore::GenericTextTrackCueMap::add): Same. (WebCore::GenericTextTrackCueMap::find): Same. (WebCore::GenericTextTrackCueMap::remove): Same. (WebCore::InbandGenericTextTrack::create): Same. (WebCore::InbandGenericTextTrack::updateCueFromCueData): Same. (WebCore::InbandGenericTextTrack::addGenericCue): Same. (WebCore::InbandGenericTextTrack::updateGenericCue): Same. (WebCore::InbandGenericTextTrack::removeGenericCue): Same. (WebCore::InbandGenericTextTrack::removeCue): Same. (WebCore::InbandGenericTextTrack::InbandGenericTextTrack): Empty. (WebCore::InbandGenericTextTrack::~InbandGenericTextTrack): Empty. * html/track/InbandGenericTextTrack.h: Copied from Source/WebCore/html/track/InbandTextTrack.h. The only addition is the ASSERT_NOT_REACHED() for WebVTT cues. * html/track/InbandTextTrack.cpp: Add label and language changed callbacks. (WebCore::InbandTextTrack::create): Return a generic or WebVTT text track based on the private CueFormat. (WebCore::InbandTextTrack::labelChanged): Added. (WebCore::InbandTextTrack::languageChanged): Added. * html/track/InbandTextTrack.h: Add label and language changed callbacks. * html/track/InbandWebVTTTextTrack.cpp: Added, based on InbandTextTrack. (WebCore::InbandWebVTTTextTrack::create): Same. (WebCore::InbandWebVTTTextTrack::InbandWebVTTTextTrack): Empty. (WebCore::InbandWebVTTTextTrack::~InbandWebVTTTextTrack): Empty. (WebCore::InbandWebVTTTextTrack::parseWebVTTCueData): Sends data to a WebVTTParser. (WebCore::InbandWebVTTTextTrack::newCuesParsed): Adds cues when WebVTTParser parses them. (WebCore::InbandWebVTTTextTrack::fileFailedToParse): Prints a warning when WebVTTParser has a problem. * html/track/InbandWebVTTTextTrack.h: Added. * platform/graphics/InbandTextTrackPrivate.h: (WebCore::InbandTextTrackPrivate::cueFormat): For determining if the track will have generic or WebVTT cues. (WebCore::InbandTextTrackPrivate::InbandTextTrackPrivate): Pass CueFormat in the constructor. * platform/graphics/InbandTextTrackPrivateClient.h: Same. * platform/graphics/avfoundation/InbandTextTrackPrivateAVF.cpp: (WebCore::InbandTextTrackPrivateAVF::InbandTextTrackPrivateAVF): Pass CueFormat (Generic) to InbandTextTrackPrivate. * platform/graphics/gstreamer/GRefPtrGStreamer.cpp: Add GRefPtr specializations for GstSample and GstEvent. * platform/graphics/gstreamer/GRefPtrGStreamer.h: Same. * platform/graphics/gstreamer/GStreamerUtilities.h: Add WARN_MEDIA_MESSAGE. * platform/graphics/gstreamer/GStreamerVersioning.h: Add a function to check GStreamer version at runtime. * platform/graphics/gstreamer/InbandTextTrackPrivateGStreamer.cpp: Added. (WebCore::textTrackPrivateEventCallback): Watches for tag and stream start events. (WebCore::textTrackPrivateSampleTimeoutCallback): See notifyTrackOfSample(); (WebCore::textTrackPrivateStreamTimeoutCallback): See notifyTrackOfStreamChanged(); (WebCore::textTrackPrivateTagsChangeTimeoutCallback): See notifyTrackOfTagsChanged(); (WebCore::InbandTextTrackPrivateGStreamer::InbandTextTrackPrivateGStreamer): Initializes tags and stream and sets up event callback. (WebCore::InbandTextTrackPrivateGStreamer::~InbandTextTrackPrivateGStreamer): Calls disconnect: (WebCore::InbandTextTrackPrivateGStreamer::disconnect): Removes signal handlers and frees some memory. (WebCore::InbandTextTrackPrivateGStreamer::handleSample): Adds samples to a list and sets up callback. (WebCore::InbandTextTrackPrivateGStreamer::streamChanged): Sets up callback. (WebCore::InbandTextTrackPrivateGStreamer::tagsChanged): Same. (WebCore::InbandTextTrackPrivateGStreamer::notifyTrackOfSample): Parses all queued samples with WebVTTParser. (WebCore::InbandTextTrackPrivateGStreamer::notifyTrackOfStreamChanged): Keeps track of current stream. (WebCore::InbandTextTrackPrivateGStreamer::notifyTrackOfTagsChanged): Sets label and language from tags. * platform/graphics/gstreamer/InbandTextTrackPrivateGStreamer.h: Added. (WebCore::InbandTextTrackPrivateGStreamer::create): Basic RefPtr create function. (WebCore::InbandTextTrackPrivateGStreamer::pad): Returns the pad this track is associated with (used to determine if a playbin text stream has already been associated with a text track). (WebCore::InbandTextTrackPrivateGStreamer::setIndex): Sets the track index (used for sorting). (WebCore::InbandTextTrackPrivateGStreamer::streamId): Returns the stream ID (used to handle new samples). * platform/graphics/gstreamer/MediaPlayerPrivateGStreamer.cpp: (WebCore::mediaPlayerPrivateTextChangedCallback): Called for playbin "text-changed" event. See textChanged(). (WebCore::mediaPlayerPrivateTextChangeTimeoutCallback): See notifyPlayerOfText(). (WebCore::mediaPlayerPrivateNewTextSampleCallback): See newTextSample(). (WebCore::MediaPlayerPrivateGStreamer::MediaPlayerPrivateGStreamer): Initialize m_textTimerHandler. (WebCore::MediaPlayerPrivateGStreamer::~MediaPlayerPrivateGStreamer): Disconnect text tracks (they don't necessarily get freed here, since a script could hold a reference). (WebCore::MediaPlayerPrivateGStreamer::textChanged): Setup callback for notifyPlayerOfText. (WebCore::MediaPlayerPrivateGStreamer::notifyPlayerOfText): Create text tracks. (WebCore::MediaPlayerPrivateGStreamer::newTextSample): Handle new samples by giving them to a text track with a matching stream. This method is syncryonous because we need to get the stream start sticky event immediately. * platform/graphics/gstreamer/MediaPlayerPrivateGStreamer.h: * platform/graphics/gstreamer/TextCombinerGStreamer.cpp: Added. This element forwards buffers from all of its input pads, but also converts plain text to WebVTT as needed. (webkit_text_combiner_init): Setup internal funnel. (webkitTextCombinerPadEvent): If the caps are plain text make sure we have a webvttenv, otherwise connect directly to the funnel. (webkitTextCombinerRequestNewPad): Setup ghostpad and event callback. (webkitTextCombinerReleasePad): Release pad and optional associated webvttenc. (webkit_text_combiner_class_init): Setup pad templates and request/release pad functions. (webkitTextCombinerNew): Returns a new WebKitTextCombiner. * platform/graphics/gstreamer/TextCombinerGStreamer.h: Added. * platform/graphics/gstreamer/TextSinkGStreamer.cpp: Added. (webkit_text_sink_init): Set sync=false. (webkitTextSinkGetProperty): Ignore sync property. (webkitTextSinkSetProperty): Same. (webkitTextSinkQuery): Ignore position and duration queries, forward everything else to appsink. (webkit_text_sink_class_init): Setup property and query functions. (webkitTextSinkNew): Return a new WebKitTextSink. * platform/graphics/gstreamer/TextSinkGStreamer.h: Added. LayoutTests: <https://webkit.org/b/103771> [GStreamer] support in-band text tracks Patch by Brendan Long <b.long@cablelabs.com> on 2013-08-30 Reviewed by Eric Carlson. * media/content/counting-subtitled-kate.ogv: Added. * media/content/counting-subtitled-srt.mkv: Added. * media/in-band-cues.js: Added. * media/track/in-band/track-in-band-kate-ogg-cues-added-once-expected.txt: Added. * media/track/in-band/track-in-band-kate-ogg-cues-added-once.html: Added. * media/track/in-band/track-in-band-kate-ogg-kind-expected.txt: Added. * media/track/in-band/track-in-band-kate-ogg-kind.html: Added. * media/track/in-band/track-in-band-kate-ogg-language-expected.txt: Added. * media/track/in-band/track-in-band-kate-ogg-language.html: Added. * media/track/in-band/track-in-band-kate-ogg-mode-expected.txt: Added. * media/track/in-band/track-in-band-kate-ogg-mode.html: Added. * media/track/in-band/track-in-band-kate-ogg-style-expected.txt: Added. * media/track/in-band/track-in-band-kate-ogg-style.html: Added. * media/track/in-band/track-in-band-kate-ogg-track-order-expected.txt: Added. * media/track/in-band/track-in-band-kate-ogg-track-order.html: Added. * media/track/in-band/track-in-band-srt-mkv-cues-added-once-expected.txt: Added. * media/track/in-band/track-in-band-srt-mkv-cues-added-once.html: Added. * media/track/in-band/track-in-band-srt-mkv-kind-expected.txt: Added. * media/track/in-band/track-in-band-srt-mkv-kind.html: Added. * media/track/in-band/track-in-band-srt-mkv-language-expected.txt: Added. * media/track/in-band/track-in-band-srt-mkv-language.html: Added. * media/track/in-band/track-in-band-srt-mkv-mode-expected.txt: Added. * media/track/in-band/track-in-band-srt-mkv-mode.html: Added. * media/track/in-band/track-in-band-srt-mkv-style-expected.txt: Added. * media/track/in-band/track-in-band-srt-mkv-style.html: Added. * media/track/in-band/track-in-band-srt-mkv-track-order-expected.txt: Added. * media/track/in-band/track-in-band-srt-mkv-track-order.html: Added. * platform/mac/TestExpectations: Skip MKV and OGG tests. Canonical link: https://commits.webkit.org/138536@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@154908 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2013-08-30 20:17:27 +00:00
{
[GStreamer] media/track/in-band/ layout tests introduced in r154908 are failing https://bugs.webkit.org/show_bug.cgi?id=120665 Reviewed by Philippe Normand. Source/WebCore: Solved a deadlock when processing samples from a text track while the main thread is starting to play. In those circumstances, the text sink is set by Playbin as non-async, so the main thread calling MediaPlayerPrivateGStreamer::play() is blocked waiting for the preroll to finish. While prerolling, a text sample is detected in a non-main thread and sent to the main thread for processing. The current code waits in the non-main thread for the processing to finish, but that won't ever happen because the main thread is busy in play() waiting for the preroll to complete. Since the non-main thread doesn't need any result from the main thread, a simple change to avoid the wait is enough to solve the deadlock. There was also a bug preventing all the languages of the text tracks being detected on time. Sometimes the tag event specifying the track language emitted by the demuxer arrives too late to the webkittextcombiner sink pad (up to several seconds since sent!) and the language remains unset when the test checks for it. As a result, the test fails because the second track isn't reported to be in French language ("fr"). The solution to this is to find an earlier (more upstream) pad and listen there for the (sticky) tag events. There can be multiple tag events, so all of them are checked until one having the GST_TAG_LANGUAGE_CODE tag is found. Covered by existing tests (now unskipped). * platform/graphics/gstreamer/TextSinkGStreamer.cpp: (webkitTextSinkHandleSample): Don't wait for handleTextSample() to be completed in the main thread. This avoids the deadlock. * platform/graphics/gstreamer/TrackPrivateBaseGStreamer.cpp: (WebCore::findBestUpstreamPad): Find a pad more upstream than the current one. (WebCore::TrackPrivateBaseGStreamer::TrackPrivateBaseGStreamer): Call setPad() instead of setting it by hand here. (WebCore::TrackPrivateBaseGStreamer::setPad): Also manage m_bestUpstreamPad in parallel to m_pad. Add a probe to listen to the tag events instead of checking the "tags" property (the property wasn't present in the best upstream pad). (WebCore::TrackPrivateBaseGStreamer::disconnect): Remove the probe from m_bestUpstreamPad, instead of removing the old notify::tags listener from m_pad. (WebCore::TrackPrivateBaseGStreamer::tagsChanged): Check the sticky tag events until the language one is found (or the last event if no language event is available). * platform/graphics/gstreamer/TrackPrivateBaseGStreamer.h: Added m_bestUpstreamPad and m_eventProbe. Moved setPad() implementation to the cpp file. Tools: Fix media/track/in-band/tests Add libkate as dependency, so gst-plugins-bad detects it and can build the GstKateDec element. This isn't actually needed when using Flatpak (the dependency has been added there), but is still useful when using jhbuild and is also coherent with the need of other multimedia dependencies such as libvpx. * gtk/install-dependencies: * wpe/install-dependencies: LayoutTests: Removed track-in-band-mpegts-metadata.html because it relies on private streams on mpegts and those won't ever be exposed by tsdemux. Made testCuesAddedOnce() test more robust when finding text tracks. * media/content/user-private-streams.ts: Removed. * media/in-band-tracks.js: (pollProgress): Make sure that the function repeated at intervals is properly stopped in all the cases. (canplaythrough): More robust track detection. (testCuesAddedOnce): Ditto. * media/track/in-band/track-in-band-mpegts-metadata.html: Removed. * platform/glib/TestExpectations: Unskip media/track/in-band tests. Canonical link: https://commits.webkit.org/237022@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@276586 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-04-26 11:59:51 +00:00
Source/WebCore: [GStreamer] support in-band text tracks https://bugs.webkit.org/show_bug.cgi?id=103771 Patch by Brendan Long <b.long@cablelabs.com> on 2013-08-30 Reviewed by Eric Carlson. Tests: New tests added because existing tests were too specific. media/track/in-band/track-in-band-kate-ogg-cues-added-once.html media/track/in-band/track-in-band-kate-ogg-kind.html media/track/in-band/track-in-band-kate-ogg-language.html media/track/in-band/track-in-band-kate-ogg-mode.html media/track/in-band/track-in-band-kate-ogg-style.html media/track/in-band/track-in-band-kate-ogg-track-order.html media/track/in-band/track-in-band-srt-mkv-cues-added-once.html media/track/in-band/track-in-band-srt-mkv-kind.html media/track/in-band/track-in-band-srt-mkv-language.html media/track/in-band/track-in-band-srt-mkv-mode.html media/track/in-band/track-in-band-srt-mkv-style.html media/track/in-band/track-in-band-srt-mkv-track-order.html * CMakeLists.txt: Add InbandTextTrackPrivateGStreamer, InbandGenericTextTrack, InbandWebVTTTextTrack, and TextCombinerGStreamer files. * GNUmakefile.list.am: Same. * PlatformEfl.cmake: Same. * Target.pri: Same. * WebCore.vcxproj/WebCore.vcxproj: Same. * WebCore.vcxproj/WebCore.vcxproj.filters: Same. * WebCore.xcodeproj/project.pbxproj: Same. * html/track/InbandGenericTextTrack.cpp: Split out code for handling generic cues. (WebCore::GenericTextTrackCueMap::GenericTextTrackCueMap): Move from InbandTextTrack. (WebCore::GenericTextTrackCueMap::~GenericTextTrackCueMap): Same. (WebCore::GenericTextTrackCueMap::add): Same. (WebCore::GenericTextTrackCueMap::find): Same. (WebCore::GenericTextTrackCueMap::remove): Same. (WebCore::InbandGenericTextTrack::create): Same. (WebCore::InbandGenericTextTrack::updateCueFromCueData): Same. (WebCore::InbandGenericTextTrack::addGenericCue): Same. (WebCore::InbandGenericTextTrack::updateGenericCue): Same. (WebCore::InbandGenericTextTrack::removeGenericCue): Same. (WebCore::InbandGenericTextTrack::removeCue): Same. (WebCore::InbandGenericTextTrack::InbandGenericTextTrack): Empty. (WebCore::InbandGenericTextTrack::~InbandGenericTextTrack): Empty. * html/track/InbandGenericTextTrack.h: Copied from Source/WebCore/html/track/InbandTextTrack.h. The only addition is the ASSERT_NOT_REACHED() for WebVTT cues. * html/track/InbandTextTrack.cpp: Add label and language changed callbacks. (WebCore::InbandTextTrack::create): Return a generic or WebVTT text track based on the private CueFormat. (WebCore::InbandTextTrack::labelChanged): Added. (WebCore::InbandTextTrack::languageChanged): Added. * html/track/InbandTextTrack.h: Add label and language changed callbacks. * html/track/InbandWebVTTTextTrack.cpp: Added, based on InbandTextTrack. (WebCore::InbandWebVTTTextTrack::create): Same. (WebCore::InbandWebVTTTextTrack::InbandWebVTTTextTrack): Empty. (WebCore::InbandWebVTTTextTrack::~InbandWebVTTTextTrack): Empty. (WebCore::InbandWebVTTTextTrack::parseWebVTTCueData): Sends data to a WebVTTParser. (WebCore::InbandWebVTTTextTrack::newCuesParsed): Adds cues when WebVTTParser parses them. (WebCore::InbandWebVTTTextTrack::fileFailedToParse): Prints a warning when WebVTTParser has a problem. * html/track/InbandWebVTTTextTrack.h: Added. * platform/graphics/InbandTextTrackPrivate.h: (WebCore::InbandTextTrackPrivate::cueFormat): For determining if the track will have generic or WebVTT cues. (WebCore::InbandTextTrackPrivate::InbandTextTrackPrivate): Pass CueFormat in the constructor. * platform/graphics/InbandTextTrackPrivateClient.h: Same. * platform/graphics/avfoundation/InbandTextTrackPrivateAVF.cpp: (WebCore::InbandTextTrackPrivateAVF::InbandTextTrackPrivateAVF): Pass CueFormat (Generic) to InbandTextTrackPrivate. * platform/graphics/gstreamer/GRefPtrGStreamer.cpp: Add GRefPtr specializations for GstSample and GstEvent. * platform/graphics/gstreamer/GRefPtrGStreamer.h: Same. * platform/graphics/gstreamer/GStreamerUtilities.h: Add WARN_MEDIA_MESSAGE. * platform/graphics/gstreamer/GStreamerVersioning.h: Add a function to check GStreamer version at runtime. * platform/graphics/gstreamer/InbandTextTrackPrivateGStreamer.cpp: Added. (WebCore::textTrackPrivateEventCallback): Watches for tag and stream start events. (WebCore::textTrackPrivateSampleTimeoutCallback): See notifyTrackOfSample(); (WebCore::textTrackPrivateStreamTimeoutCallback): See notifyTrackOfStreamChanged(); (WebCore::textTrackPrivateTagsChangeTimeoutCallback): See notifyTrackOfTagsChanged(); (WebCore::InbandTextTrackPrivateGStreamer::InbandTextTrackPrivateGStreamer): Initializes tags and stream and sets up event callback. (WebCore::InbandTextTrackPrivateGStreamer::~InbandTextTrackPrivateGStreamer): Calls disconnect: (WebCore::InbandTextTrackPrivateGStreamer::disconnect): Removes signal handlers and frees some memory. (WebCore::InbandTextTrackPrivateGStreamer::handleSample): Adds samples to a list and sets up callback. (WebCore::InbandTextTrackPrivateGStreamer::streamChanged): Sets up callback. (WebCore::InbandTextTrackPrivateGStreamer::tagsChanged): Same. (WebCore::InbandTextTrackPrivateGStreamer::notifyTrackOfSample): Parses all queued samples with WebVTTParser. (WebCore::InbandTextTrackPrivateGStreamer::notifyTrackOfStreamChanged): Keeps track of current stream. (WebCore::InbandTextTrackPrivateGStreamer::notifyTrackOfTagsChanged): Sets label and language from tags. * platform/graphics/gstreamer/InbandTextTrackPrivateGStreamer.h: Added. (WebCore::InbandTextTrackPrivateGStreamer::create): Basic RefPtr create function. (WebCore::InbandTextTrackPrivateGStreamer::pad): Returns the pad this track is associated with (used to determine if a playbin text stream has already been associated with a text track). (WebCore::InbandTextTrackPrivateGStreamer::setIndex): Sets the track index (used for sorting). (WebCore::InbandTextTrackPrivateGStreamer::streamId): Returns the stream ID (used to handle new samples). * platform/graphics/gstreamer/MediaPlayerPrivateGStreamer.cpp: (WebCore::mediaPlayerPrivateTextChangedCallback): Called for playbin "text-changed" event. See textChanged(). (WebCore::mediaPlayerPrivateTextChangeTimeoutCallback): See notifyPlayerOfText(). (WebCore::mediaPlayerPrivateNewTextSampleCallback): See newTextSample(). (WebCore::MediaPlayerPrivateGStreamer::MediaPlayerPrivateGStreamer): Initialize m_textTimerHandler. (WebCore::MediaPlayerPrivateGStreamer::~MediaPlayerPrivateGStreamer): Disconnect text tracks (they don't necessarily get freed here, since a script could hold a reference). (WebCore::MediaPlayerPrivateGStreamer::textChanged): Setup callback for notifyPlayerOfText. (WebCore::MediaPlayerPrivateGStreamer::notifyPlayerOfText): Create text tracks. (WebCore::MediaPlayerPrivateGStreamer::newTextSample): Handle new samples by giving them to a text track with a matching stream. This method is syncryonous because we need to get the stream start sticky event immediately. * platform/graphics/gstreamer/MediaPlayerPrivateGStreamer.h: * platform/graphics/gstreamer/TextCombinerGStreamer.cpp: Added. This element forwards buffers from all of its input pads, but also converts plain text to WebVTT as needed. (webkit_text_combiner_init): Setup internal funnel. (webkitTextCombinerPadEvent): If the caps are plain text make sure we have a webvttenv, otherwise connect directly to the funnel. (webkitTextCombinerRequestNewPad): Setup ghostpad and event callback. (webkitTextCombinerReleasePad): Release pad and optional associated webvttenc. (webkit_text_combiner_class_init): Setup pad templates and request/release pad functions. (webkitTextCombinerNew): Returns a new WebKitTextCombiner. * platform/graphics/gstreamer/TextCombinerGStreamer.h: Added. * platform/graphics/gstreamer/TextSinkGStreamer.cpp: Added. (webkit_text_sink_init): Set sync=false. (webkitTextSinkGetProperty): Ignore sync property. (webkitTextSinkSetProperty): Same. (webkitTextSinkQuery): Ignore position and duration queries, forward everything else to appsink. (webkit_text_sink_class_init): Setup property and query functions. (webkitTextSinkNew): Return a new WebKitTextSink. * platform/graphics/gstreamer/TextSinkGStreamer.h: Added. LayoutTests: <https://webkit.org/b/103771> [GStreamer] support in-band text tracks Patch by Brendan Long <b.long@cablelabs.com> on 2013-08-30 Reviewed by Eric Carlson. * media/content/counting-subtitled-kate.ogv: Added. * media/content/counting-subtitled-srt.mkv: Added. * media/in-band-cues.js: Added. * media/track/in-band/track-in-band-kate-ogg-cues-added-once-expected.txt: Added. * media/track/in-band/track-in-band-kate-ogg-cues-added-once.html: Added. * media/track/in-band/track-in-band-kate-ogg-kind-expected.txt: Added. * media/track/in-band/track-in-band-kate-ogg-kind.html: Added. * media/track/in-band/track-in-band-kate-ogg-language-expected.txt: Added. * media/track/in-band/track-in-band-kate-ogg-language.html: Added. * media/track/in-band/track-in-band-kate-ogg-mode-expected.txt: Added. * media/track/in-band/track-in-band-kate-ogg-mode.html: Added. * media/track/in-band/track-in-band-kate-ogg-style-expected.txt: Added. * media/track/in-band/track-in-band-kate-ogg-style.html: Added. * media/track/in-band/track-in-band-kate-ogg-track-order-expected.txt: Added. * media/track/in-band/track-in-band-kate-ogg-track-order.html: Added. * media/track/in-band/track-in-band-srt-mkv-cues-added-once-expected.txt: Added. * media/track/in-band/track-in-band-srt-mkv-cues-added-once.html: Added. * media/track/in-band/track-in-band-srt-mkv-kind-expected.txt: Added. * media/track/in-band/track-in-band-srt-mkv-kind.html: Added. * media/track/in-band/track-in-band-srt-mkv-language-expected.txt: Added. * media/track/in-band/track-in-band-srt-mkv-language.html: Added. * media/track/in-band/track-in-band-srt-mkv-mode-expected.txt: Added. * media/track/in-band/track-in-band-srt-mkv-mode.html: Added. * media/track/in-band/track-in-band-srt-mkv-style-expected.txt: Added. * media/track/in-band/track-in-band-srt-mkv-style.html: Added. * media/track/in-band/track-in-band-srt-mkv-track-order-expected.txt: Added. * media/track/in-band/track-in-band-srt-mkv-track-order.html: Added. * platform/mac/TestExpectations: Skip MKV and OGG tests. Canonical link: https://commits.webkit.org/138536@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@154908 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2013-08-30 20:17:27 +00:00
var seekedCount = 0;
var cuesStarts = [];
[GStreamer] media/track/in-band/ layout tests introduced in r154908 are failing https://bugs.webkit.org/show_bug.cgi?id=120665 Reviewed by Philippe Normand. Source/WebCore: Solved a deadlock when processing samples from a text track while the main thread is starting to play. In those circumstances, the text sink is set by Playbin as non-async, so the main thread calling MediaPlayerPrivateGStreamer::play() is blocked waiting for the preroll to finish. While prerolling, a text sample is detected in a non-main thread and sent to the main thread for processing. The current code waits in the non-main thread for the processing to finish, but that won't ever happen because the main thread is busy in play() waiting for the preroll to complete. Since the non-main thread doesn't need any result from the main thread, a simple change to avoid the wait is enough to solve the deadlock. There was also a bug preventing all the languages of the text tracks being detected on time. Sometimes the tag event specifying the track language emitted by the demuxer arrives too late to the webkittextcombiner sink pad (up to several seconds since sent!) and the language remains unset when the test checks for it. As a result, the test fails because the second track isn't reported to be in French language ("fr"). The solution to this is to find an earlier (more upstream) pad and listen there for the (sticky) tag events. There can be multiple tag events, so all of them are checked until one having the GST_TAG_LANGUAGE_CODE tag is found. Covered by existing tests (now unskipped). * platform/graphics/gstreamer/TextSinkGStreamer.cpp: (webkitTextSinkHandleSample): Don't wait for handleTextSample() to be completed in the main thread. This avoids the deadlock. * platform/graphics/gstreamer/TrackPrivateBaseGStreamer.cpp: (WebCore::findBestUpstreamPad): Find a pad more upstream than the current one. (WebCore::TrackPrivateBaseGStreamer::TrackPrivateBaseGStreamer): Call setPad() instead of setting it by hand here. (WebCore::TrackPrivateBaseGStreamer::setPad): Also manage m_bestUpstreamPad in parallel to m_pad. Add a probe to listen to the tag events instead of checking the "tags" property (the property wasn't present in the best upstream pad). (WebCore::TrackPrivateBaseGStreamer::disconnect): Remove the probe from m_bestUpstreamPad, instead of removing the old notify::tags listener from m_pad. (WebCore::TrackPrivateBaseGStreamer::tagsChanged): Check the sticky tag events until the language one is found (or the last event if no language event is available). * platform/graphics/gstreamer/TrackPrivateBaseGStreamer.h: Added m_bestUpstreamPad and m_eventProbe. Moved setPad() implementation to the cpp file. Tools: Fix media/track/in-band/tests Add libkate as dependency, so gst-plugins-bad detects it and can build the GstKateDec element. This isn't actually needed when using Flatpak (the dependency has been added there), but is still useful when using jhbuild and is also coherent with the need of other multimedia dependencies such as libvpx. * gtk/install-dependencies: * wpe/install-dependencies: LayoutTests: Removed track-in-band-mpegts-metadata.html because it relies on private streams on mpegts and those won't ever be exposed by tsdemux. Made testCuesAddedOnce() test more robust when finding text tracks. * media/content/user-private-streams.ts: Removed. * media/in-band-tracks.js: (pollProgress): Make sure that the function repeated at intervals is properly stopped in all the cases. (canplaythrough): More robust track detection. (testCuesAddedOnce): Ditto. * media/track/in-band/track-in-band-mpegts-metadata.html: Removed. * platform/glib/TestExpectations: Unskip media/track/in-band tests. Canonical link: https://commits.webkit.org/237022@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@276586 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-04-26 11:59:51 +00:00
var intervalId = null;
inbandTrack1 = null;
Source/WebCore: [GStreamer] support in-band text tracks https://bugs.webkit.org/show_bug.cgi?id=103771 Patch by Brendan Long <b.long@cablelabs.com> on 2013-08-30 Reviewed by Eric Carlson. Tests: New tests added because existing tests were too specific. media/track/in-band/track-in-band-kate-ogg-cues-added-once.html media/track/in-band/track-in-band-kate-ogg-kind.html media/track/in-band/track-in-band-kate-ogg-language.html media/track/in-band/track-in-band-kate-ogg-mode.html media/track/in-band/track-in-band-kate-ogg-style.html media/track/in-band/track-in-band-kate-ogg-track-order.html media/track/in-band/track-in-band-srt-mkv-cues-added-once.html media/track/in-band/track-in-band-srt-mkv-kind.html media/track/in-band/track-in-band-srt-mkv-language.html media/track/in-band/track-in-band-srt-mkv-mode.html media/track/in-band/track-in-band-srt-mkv-style.html media/track/in-band/track-in-band-srt-mkv-track-order.html * CMakeLists.txt: Add InbandTextTrackPrivateGStreamer, InbandGenericTextTrack, InbandWebVTTTextTrack, and TextCombinerGStreamer files. * GNUmakefile.list.am: Same. * PlatformEfl.cmake: Same. * Target.pri: Same. * WebCore.vcxproj/WebCore.vcxproj: Same. * WebCore.vcxproj/WebCore.vcxproj.filters: Same. * WebCore.xcodeproj/project.pbxproj: Same. * html/track/InbandGenericTextTrack.cpp: Split out code for handling generic cues. (WebCore::GenericTextTrackCueMap::GenericTextTrackCueMap): Move from InbandTextTrack. (WebCore::GenericTextTrackCueMap::~GenericTextTrackCueMap): Same. (WebCore::GenericTextTrackCueMap::add): Same. (WebCore::GenericTextTrackCueMap::find): Same. (WebCore::GenericTextTrackCueMap::remove): Same. (WebCore::InbandGenericTextTrack::create): Same. (WebCore::InbandGenericTextTrack::updateCueFromCueData): Same. (WebCore::InbandGenericTextTrack::addGenericCue): Same. (WebCore::InbandGenericTextTrack::updateGenericCue): Same. (WebCore::InbandGenericTextTrack::removeGenericCue): Same. (WebCore::InbandGenericTextTrack::removeCue): Same. (WebCore::InbandGenericTextTrack::InbandGenericTextTrack): Empty. (WebCore::InbandGenericTextTrack::~InbandGenericTextTrack): Empty. * html/track/InbandGenericTextTrack.h: Copied from Source/WebCore/html/track/InbandTextTrack.h. The only addition is the ASSERT_NOT_REACHED() for WebVTT cues. * html/track/InbandTextTrack.cpp: Add label and language changed callbacks. (WebCore::InbandTextTrack::create): Return a generic or WebVTT text track based on the private CueFormat. (WebCore::InbandTextTrack::labelChanged): Added. (WebCore::InbandTextTrack::languageChanged): Added. * html/track/InbandTextTrack.h: Add label and language changed callbacks. * html/track/InbandWebVTTTextTrack.cpp: Added, based on InbandTextTrack. (WebCore::InbandWebVTTTextTrack::create): Same. (WebCore::InbandWebVTTTextTrack::InbandWebVTTTextTrack): Empty. (WebCore::InbandWebVTTTextTrack::~InbandWebVTTTextTrack): Empty. (WebCore::InbandWebVTTTextTrack::parseWebVTTCueData): Sends data to a WebVTTParser. (WebCore::InbandWebVTTTextTrack::newCuesParsed): Adds cues when WebVTTParser parses them. (WebCore::InbandWebVTTTextTrack::fileFailedToParse): Prints a warning when WebVTTParser has a problem. * html/track/InbandWebVTTTextTrack.h: Added. * platform/graphics/InbandTextTrackPrivate.h: (WebCore::InbandTextTrackPrivate::cueFormat): For determining if the track will have generic or WebVTT cues. (WebCore::InbandTextTrackPrivate::InbandTextTrackPrivate): Pass CueFormat in the constructor. * platform/graphics/InbandTextTrackPrivateClient.h: Same. * platform/graphics/avfoundation/InbandTextTrackPrivateAVF.cpp: (WebCore::InbandTextTrackPrivateAVF::InbandTextTrackPrivateAVF): Pass CueFormat (Generic) to InbandTextTrackPrivate. * platform/graphics/gstreamer/GRefPtrGStreamer.cpp: Add GRefPtr specializations for GstSample and GstEvent. * platform/graphics/gstreamer/GRefPtrGStreamer.h: Same. * platform/graphics/gstreamer/GStreamerUtilities.h: Add WARN_MEDIA_MESSAGE. * platform/graphics/gstreamer/GStreamerVersioning.h: Add a function to check GStreamer version at runtime. * platform/graphics/gstreamer/InbandTextTrackPrivateGStreamer.cpp: Added. (WebCore::textTrackPrivateEventCallback): Watches for tag and stream start events. (WebCore::textTrackPrivateSampleTimeoutCallback): See notifyTrackOfSample(); (WebCore::textTrackPrivateStreamTimeoutCallback): See notifyTrackOfStreamChanged(); (WebCore::textTrackPrivateTagsChangeTimeoutCallback): See notifyTrackOfTagsChanged(); (WebCore::InbandTextTrackPrivateGStreamer::InbandTextTrackPrivateGStreamer): Initializes tags and stream and sets up event callback. (WebCore::InbandTextTrackPrivateGStreamer::~InbandTextTrackPrivateGStreamer): Calls disconnect: (WebCore::InbandTextTrackPrivateGStreamer::disconnect): Removes signal handlers and frees some memory. (WebCore::InbandTextTrackPrivateGStreamer::handleSample): Adds samples to a list and sets up callback. (WebCore::InbandTextTrackPrivateGStreamer::streamChanged): Sets up callback. (WebCore::InbandTextTrackPrivateGStreamer::tagsChanged): Same. (WebCore::InbandTextTrackPrivateGStreamer::notifyTrackOfSample): Parses all queued samples with WebVTTParser. (WebCore::InbandTextTrackPrivateGStreamer::notifyTrackOfStreamChanged): Keeps track of current stream. (WebCore::InbandTextTrackPrivateGStreamer::notifyTrackOfTagsChanged): Sets label and language from tags. * platform/graphics/gstreamer/InbandTextTrackPrivateGStreamer.h: Added. (WebCore::InbandTextTrackPrivateGStreamer::create): Basic RefPtr create function. (WebCore::InbandTextTrackPrivateGStreamer::pad): Returns the pad this track is associated with (used to determine if a playbin text stream has already been associated with a text track). (WebCore::InbandTextTrackPrivateGStreamer::setIndex): Sets the track index (used for sorting). (WebCore::InbandTextTrackPrivateGStreamer::streamId): Returns the stream ID (used to handle new samples). * platform/graphics/gstreamer/MediaPlayerPrivateGStreamer.cpp: (WebCore::mediaPlayerPrivateTextChangedCallback): Called for playbin "text-changed" event. See textChanged(). (WebCore::mediaPlayerPrivateTextChangeTimeoutCallback): See notifyPlayerOfText(). (WebCore::mediaPlayerPrivateNewTextSampleCallback): See newTextSample(). (WebCore::MediaPlayerPrivateGStreamer::MediaPlayerPrivateGStreamer): Initialize m_textTimerHandler. (WebCore::MediaPlayerPrivateGStreamer::~MediaPlayerPrivateGStreamer): Disconnect text tracks (they don't necessarily get freed here, since a script could hold a reference). (WebCore::MediaPlayerPrivateGStreamer::textChanged): Setup callback for notifyPlayerOfText. (WebCore::MediaPlayerPrivateGStreamer::notifyPlayerOfText): Create text tracks. (WebCore::MediaPlayerPrivateGStreamer::newTextSample): Handle new samples by giving them to a text track with a matching stream. This method is syncryonous because we need to get the stream start sticky event immediately. * platform/graphics/gstreamer/MediaPlayerPrivateGStreamer.h: * platform/graphics/gstreamer/TextCombinerGStreamer.cpp: Added. This element forwards buffers from all of its input pads, but also converts plain text to WebVTT as needed. (webkit_text_combiner_init): Setup internal funnel. (webkitTextCombinerPadEvent): If the caps are plain text make sure we have a webvttenv, otherwise connect directly to the funnel. (webkitTextCombinerRequestNewPad): Setup ghostpad and event callback. (webkitTextCombinerReleasePad): Release pad and optional associated webvttenc. (webkit_text_combiner_class_init): Setup pad templates and request/release pad functions. (webkitTextCombinerNew): Returns a new WebKitTextCombiner. * platform/graphics/gstreamer/TextCombinerGStreamer.h: Added. * platform/graphics/gstreamer/TextSinkGStreamer.cpp: Added. (webkit_text_sink_init): Set sync=false. (webkitTextSinkGetProperty): Ignore sync property. (webkitTextSinkSetProperty): Same. (webkitTextSinkQuery): Ignore position and duration queries, forward everything else to appsink. (webkit_text_sink_class_init): Setup property and query functions. (webkitTextSinkNew): Return a new WebKitTextSink. * platform/graphics/gstreamer/TextSinkGStreamer.h: Added. LayoutTests: <https://webkit.org/b/103771> [GStreamer] support in-band text tracks Patch by Brendan Long <b.long@cablelabs.com> on 2013-08-30 Reviewed by Eric Carlson. * media/content/counting-subtitled-kate.ogv: Added. * media/content/counting-subtitled-srt.mkv: Added. * media/in-band-cues.js: Added. * media/track/in-band/track-in-band-kate-ogg-cues-added-once-expected.txt: Added. * media/track/in-band/track-in-band-kate-ogg-cues-added-once.html: Added. * media/track/in-band/track-in-band-kate-ogg-kind-expected.txt: Added. * media/track/in-band/track-in-band-kate-ogg-kind.html: Added. * media/track/in-band/track-in-band-kate-ogg-language-expected.txt: Added. * media/track/in-band/track-in-band-kate-ogg-language.html: Added. * media/track/in-band/track-in-band-kate-ogg-mode-expected.txt: Added. * media/track/in-band/track-in-band-kate-ogg-mode.html: Added. * media/track/in-band/track-in-band-kate-ogg-style-expected.txt: Added. * media/track/in-band/track-in-band-kate-ogg-style.html: Added. * media/track/in-band/track-in-band-kate-ogg-track-order-expected.txt: Added. * media/track/in-band/track-in-band-kate-ogg-track-order.html: Added. * media/track/in-band/track-in-band-srt-mkv-cues-added-once-expected.txt: Added. * media/track/in-band/track-in-band-srt-mkv-cues-added-once.html: Added. * media/track/in-band/track-in-band-srt-mkv-kind-expected.txt: Added. * media/track/in-band/track-in-band-srt-mkv-kind.html: Added. * media/track/in-band/track-in-band-srt-mkv-language-expected.txt: Added. * media/track/in-band/track-in-band-srt-mkv-language.html: Added. * media/track/in-band/track-in-band-srt-mkv-mode-expected.txt: Added. * media/track/in-band/track-in-band-srt-mkv-mode.html: Added. * media/track/in-band/track-in-band-srt-mkv-style-expected.txt: Added. * media/track/in-band/track-in-band-srt-mkv-style.html: Added. * media/track/in-band/track-in-band-srt-mkv-track-order-expected.txt: Added. * media/track/in-band/track-in-band-srt-mkv-track-order.html: Added. * platform/mac/TestExpectations: Skip MKV and OGG tests. Canonical link: https://commits.webkit.org/138536@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@154908 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2013-08-30 20:17:27 +00:00
function pollProgress()
{
if (video.currentTime < 2)
return;
[GStreamer] media/track/in-band/ layout tests introduced in r154908 are failing https://bugs.webkit.org/show_bug.cgi?id=120665 Reviewed by Philippe Normand. Source/WebCore: Solved a deadlock when processing samples from a text track while the main thread is starting to play. In those circumstances, the text sink is set by Playbin as non-async, so the main thread calling MediaPlayerPrivateGStreamer::play() is blocked waiting for the preroll to finish. While prerolling, a text sample is detected in a non-main thread and sent to the main thread for processing. The current code waits in the non-main thread for the processing to finish, but that won't ever happen because the main thread is busy in play() waiting for the preroll to complete. Since the non-main thread doesn't need any result from the main thread, a simple change to avoid the wait is enough to solve the deadlock. There was also a bug preventing all the languages of the text tracks being detected on time. Sometimes the tag event specifying the track language emitted by the demuxer arrives too late to the webkittextcombiner sink pad (up to several seconds since sent!) and the language remains unset when the test checks for it. As a result, the test fails because the second track isn't reported to be in French language ("fr"). The solution to this is to find an earlier (more upstream) pad and listen there for the (sticky) tag events. There can be multiple tag events, so all of them are checked until one having the GST_TAG_LANGUAGE_CODE tag is found. Covered by existing tests (now unskipped). * platform/graphics/gstreamer/TextSinkGStreamer.cpp: (webkitTextSinkHandleSample): Don't wait for handleTextSample() to be completed in the main thread. This avoids the deadlock. * platform/graphics/gstreamer/TrackPrivateBaseGStreamer.cpp: (WebCore::findBestUpstreamPad): Find a pad more upstream than the current one. (WebCore::TrackPrivateBaseGStreamer::TrackPrivateBaseGStreamer): Call setPad() instead of setting it by hand here. (WebCore::TrackPrivateBaseGStreamer::setPad): Also manage m_bestUpstreamPad in parallel to m_pad. Add a probe to listen to the tag events instead of checking the "tags" property (the property wasn't present in the best upstream pad). (WebCore::TrackPrivateBaseGStreamer::disconnect): Remove the probe from m_bestUpstreamPad, instead of removing the old notify::tags listener from m_pad. (WebCore::TrackPrivateBaseGStreamer::tagsChanged): Check the sticky tag events until the language one is found (or the last event if no language event is available). * platform/graphics/gstreamer/TrackPrivateBaseGStreamer.h: Added m_bestUpstreamPad and m_eventProbe. Moved setPad() implementation to the cpp file. Tools: Fix media/track/in-band/tests Add libkate as dependency, so gst-plugins-bad detects it and can build the GstKateDec element. This isn't actually needed when using Flatpak (the dependency has been added there), but is still useful when using jhbuild and is also coherent with the need of other multimedia dependencies such as libvpx. * gtk/install-dependencies: * wpe/install-dependencies: LayoutTests: Removed track-in-band-mpegts-metadata.html because it relies on private streams on mpegts and those won't ever be exposed by tsdemux. Made testCuesAddedOnce() test more robust when finding text tracks. * media/content/user-private-streams.ts: Removed. * media/in-band-tracks.js: (pollProgress): Make sure that the function repeated at intervals is properly stopped in all the cases. (canplaythrough): More robust track detection. (testCuesAddedOnce): Ditto. * media/track/in-band/track-in-band-mpegts-metadata.html: Removed. * platform/glib/TestExpectations: Unskip media/track/in-band tests. Canonical link: https://commits.webkit.org/237022@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@276586 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-04-26 11:59:51 +00:00
if (!inbandTrack1) {
failTest("No text track of kind '" + kind + "'");
clearInterval(intervalId);
return;
}
if (!inbandTrack1.cues) {
failTest("No text track of kind '" + kind + "'");
clearInterval(intervalId);
return;
}
Source/WebCore: [GStreamer] support in-band text tracks https://bugs.webkit.org/show_bug.cgi?id=103771 Patch by Brendan Long <b.long@cablelabs.com> on 2013-08-30 Reviewed by Eric Carlson. Tests: New tests added because existing tests were too specific. media/track/in-band/track-in-band-kate-ogg-cues-added-once.html media/track/in-band/track-in-band-kate-ogg-kind.html media/track/in-band/track-in-band-kate-ogg-language.html media/track/in-band/track-in-band-kate-ogg-mode.html media/track/in-band/track-in-band-kate-ogg-style.html media/track/in-band/track-in-band-kate-ogg-track-order.html media/track/in-band/track-in-band-srt-mkv-cues-added-once.html media/track/in-band/track-in-band-srt-mkv-kind.html media/track/in-band/track-in-band-srt-mkv-language.html media/track/in-band/track-in-band-srt-mkv-mode.html media/track/in-band/track-in-band-srt-mkv-style.html media/track/in-band/track-in-band-srt-mkv-track-order.html * CMakeLists.txt: Add InbandTextTrackPrivateGStreamer, InbandGenericTextTrack, InbandWebVTTTextTrack, and TextCombinerGStreamer files. * GNUmakefile.list.am: Same. * PlatformEfl.cmake: Same. * Target.pri: Same. * WebCore.vcxproj/WebCore.vcxproj: Same. * WebCore.vcxproj/WebCore.vcxproj.filters: Same. * WebCore.xcodeproj/project.pbxproj: Same. * html/track/InbandGenericTextTrack.cpp: Split out code for handling generic cues. (WebCore::GenericTextTrackCueMap::GenericTextTrackCueMap): Move from InbandTextTrack. (WebCore::GenericTextTrackCueMap::~GenericTextTrackCueMap): Same. (WebCore::GenericTextTrackCueMap::add): Same. (WebCore::GenericTextTrackCueMap::find): Same. (WebCore::GenericTextTrackCueMap::remove): Same. (WebCore::InbandGenericTextTrack::create): Same. (WebCore::InbandGenericTextTrack::updateCueFromCueData): Same. (WebCore::InbandGenericTextTrack::addGenericCue): Same. (WebCore::InbandGenericTextTrack::updateGenericCue): Same. (WebCore::InbandGenericTextTrack::removeGenericCue): Same. (WebCore::InbandGenericTextTrack::removeCue): Same. (WebCore::InbandGenericTextTrack::InbandGenericTextTrack): Empty. (WebCore::InbandGenericTextTrack::~InbandGenericTextTrack): Empty. * html/track/InbandGenericTextTrack.h: Copied from Source/WebCore/html/track/InbandTextTrack.h. The only addition is the ASSERT_NOT_REACHED() for WebVTT cues. * html/track/InbandTextTrack.cpp: Add label and language changed callbacks. (WebCore::InbandTextTrack::create): Return a generic or WebVTT text track based on the private CueFormat. (WebCore::InbandTextTrack::labelChanged): Added. (WebCore::InbandTextTrack::languageChanged): Added. * html/track/InbandTextTrack.h: Add label and language changed callbacks. * html/track/InbandWebVTTTextTrack.cpp: Added, based on InbandTextTrack. (WebCore::InbandWebVTTTextTrack::create): Same. (WebCore::InbandWebVTTTextTrack::InbandWebVTTTextTrack): Empty. (WebCore::InbandWebVTTTextTrack::~InbandWebVTTTextTrack): Empty. (WebCore::InbandWebVTTTextTrack::parseWebVTTCueData): Sends data to a WebVTTParser. (WebCore::InbandWebVTTTextTrack::newCuesParsed): Adds cues when WebVTTParser parses them. (WebCore::InbandWebVTTTextTrack::fileFailedToParse): Prints a warning when WebVTTParser has a problem. * html/track/InbandWebVTTTextTrack.h: Added. * platform/graphics/InbandTextTrackPrivate.h: (WebCore::InbandTextTrackPrivate::cueFormat): For determining if the track will have generic or WebVTT cues. (WebCore::InbandTextTrackPrivate::InbandTextTrackPrivate): Pass CueFormat in the constructor. * platform/graphics/InbandTextTrackPrivateClient.h: Same. * platform/graphics/avfoundation/InbandTextTrackPrivateAVF.cpp: (WebCore::InbandTextTrackPrivateAVF::InbandTextTrackPrivateAVF): Pass CueFormat (Generic) to InbandTextTrackPrivate. * platform/graphics/gstreamer/GRefPtrGStreamer.cpp: Add GRefPtr specializations for GstSample and GstEvent. * platform/graphics/gstreamer/GRefPtrGStreamer.h: Same. * platform/graphics/gstreamer/GStreamerUtilities.h: Add WARN_MEDIA_MESSAGE. * platform/graphics/gstreamer/GStreamerVersioning.h: Add a function to check GStreamer version at runtime. * platform/graphics/gstreamer/InbandTextTrackPrivateGStreamer.cpp: Added. (WebCore::textTrackPrivateEventCallback): Watches for tag and stream start events. (WebCore::textTrackPrivateSampleTimeoutCallback): See notifyTrackOfSample(); (WebCore::textTrackPrivateStreamTimeoutCallback): See notifyTrackOfStreamChanged(); (WebCore::textTrackPrivateTagsChangeTimeoutCallback): See notifyTrackOfTagsChanged(); (WebCore::InbandTextTrackPrivateGStreamer::InbandTextTrackPrivateGStreamer): Initializes tags and stream and sets up event callback. (WebCore::InbandTextTrackPrivateGStreamer::~InbandTextTrackPrivateGStreamer): Calls disconnect: (WebCore::InbandTextTrackPrivateGStreamer::disconnect): Removes signal handlers and frees some memory. (WebCore::InbandTextTrackPrivateGStreamer::handleSample): Adds samples to a list and sets up callback. (WebCore::InbandTextTrackPrivateGStreamer::streamChanged): Sets up callback. (WebCore::InbandTextTrackPrivateGStreamer::tagsChanged): Same. (WebCore::InbandTextTrackPrivateGStreamer::notifyTrackOfSample): Parses all queued samples with WebVTTParser. (WebCore::InbandTextTrackPrivateGStreamer::notifyTrackOfStreamChanged): Keeps track of current stream. (WebCore::InbandTextTrackPrivateGStreamer::notifyTrackOfTagsChanged): Sets label and language from tags. * platform/graphics/gstreamer/InbandTextTrackPrivateGStreamer.h: Added. (WebCore::InbandTextTrackPrivateGStreamer::create): Basic RefPtr create function. (WebCore::InbandTextTrackPrivateGStreamer::pad): Returns the pad this track is associated with (used to determine if a playbin text stream has already been associated with a text track). (WebCore::InbandTextTrackPrivateGStreamer::setIndex): Sets the track index (used for sorting). (WebCore::InbandTextTrackPrivateGStreamer::streamId): Returns the stream ID (used to handle new samples). * platform/graphics/gstreamer/MediaPlayerPrivateGStreamer.cpp: (WebCore::mediaPlayerPrivateTextChangedCallback): Called for playbin "text-changed" event. See textChanged(). (WebCore::mediaPlayerPrivateTextChangeTimeoutCallback): See notifyPlayerOfText(). (WebCore::mediaPlayerPrivateNewTextSampleCallback): See newTextSample(). (WebCore::MediaPlayerPrivateGStreamer::MediaPlayerPrivateGStreamer): Initialize m_textTimerHandler. (WebCore::MediaPlayerPrivateGStreamer::~MediaPlayerPrivateGStreamer): Disconnect text tracks (they don't necessarily get freed here, since a script could hold a reference). (WebCore::MediaPlayerPrivateGStreamer::textChanged): Setup callback for notifyPlayerOfText. (WebCore::MediaPlayerPrivateGStreamer::notifyPlayerOfText): Create text tracks. (WebCore::MediaPlayerPrivateGStreamer::newTextSample): Handle new samples by giving them to a text track with a matching stream. This method is syncryonous because we need to get the stream start sticky event immediately. * platform/graphics/gstreamer/MediaPlayerPrivateGStreamer.h: * platform/graphics/gstreamer/TextCombinerGStreamer.cpp: Added. This element forwards buffers from all of its input pads, but also converts plain text to WebVTT as needed. (webkit_text_combiner_init): Setup internal funnel. (webkitTextCombinerPadEvent): If the caps are plain text make sure we have a webvttenv, otherwise connect directly to the funnel. (webkitTextCombinerRequestNewPad): Setup ghostpad and event callback. (webkitTextCombinerReleasePad): Release pad and optional associated webvttenc. (webkit_text_combiner_class_init): Setup pad templates and request/release pad functions. (webkitTextCombinerNew): Returns a new WebKitTextCombiner. * platform/graphics/gstreamer/TextCombinerGStreamer.h: Added. * platform/graphics/gstreamer/TextSinkGStreamer.cpp: Added. (webkit_text_sink_init): Set sync=false. (webkitTextSinkGetProperty): Ignore sync property. (webkitTextSinkSetProperty): Same. (webkitTextSinkQuery): Ignore position and duration queries, forward everything else to appsink. (webkit_text_sink_class_init): Setup property and query functions. (webkitTextSinkNew): Return a new WebKitTextSink. * platform/graphics/gstreamer/TextSinkGStreamer.h: Added. LayoutTests: <https://webkit.org/b/103771> [GStreamer] support in-band text tracks Patch by Brendan Long <b.long@cablelabs.com> on 2013-08-30 Reviewed by Eric Carlson. * media/content/counting-subtitled-kate.ogv: Added. * media/content/counting-subtitled-srt.mkv: Added. * media/in-band-cues.js: Added. * media/track/in-band/track-in-band-kate-ogg-cues-added-once-expected.txt: Added. * media/track/in-band/track-in-band-kate-ogg-cues-added-once.html: Added. * media/track/in-band/track-in-band-kate-ogg-kind-expected.txt: Added. * media/track/in-band/track-in-band-kate-ogg-kind.html: Added. * media/track/in-band/track-in-band-kate-ogg-language-expected.txt: Added. * media/track/in-band/track-in-band-kate-ogg-language.html: Added. * media/track/in-band/track-in-band-kate-ogg-mode-expected.txt: Added. * media/track/in-band/track-in-band-kate-ogg-mode.html: Added. * media/track/in-band/track-in-band-kate-ogg-style-expected.txt: Added. * media/track/in-band/track-in-band-kate-ogg-style.html: Added. * media/track/in-band/track-in-band-kate-ogg-track-order-expected.txt: Added. * media/track/in-band/track-in-band-kate-ogg-track-order.html: Added. * media/track/in-band/track-in-band-srt-mkv-cues-added-once-expected.txt: Added. * media/track/in-band/track-in-band-srt-mkv-cues-added-once.html: Added. * media/track/in-band/track-in-band-srt-mkv-kind-expected.txt: Added. * media/track/in-band/track-in-band-srt-mkv-kind.html: Added. * media/track/in-band/track-in-band-srt-mkv-language-expected.txt: Added. * media/track/in-band/track-in-band-srt-mkv-language.html: Added. * media/track/in-band/track-in-band-srt-mkv-mode-expected.txt: Added. * media/track/in-band/track-in-band-srt-mkv-mode.html: Added. * media/track/in-band/track-in-band-srt-mkv-style-expected.txt: Added. * media/track/in-band/track-in-band-srt-mkv-style.html: Added. * media/track/in-band/track-in-band-srt-mkv-track-order-expected.txt: Added. * media/track/in-band/track-in-band-srt-mkv-track-order.html: Added. * platform/mac/TestExpectations: Skip MKV and OGG tests. Canonical link: https://commits.webkit.org/138536@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@154908 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2013-08-30 20:17:27 +00:00
testExpected("inbandTrack1.cues.length", 0, ">");
if (!seekedCount) {
// Collect the start times of all cues, seek back to the\beginning and play
// the same segment of the video file.
run("video.pause()");
for (var i = 0; i < inbandTrack1.cues.length; ++i)
cuesStarts.push(inbandTrack1.cues[i].startTime);
run("video.currentTime = 0");
run("video.play()");
consoleWrite("");
return;
}
run("video.pause()");
try {
var success = true;
for (var i = 0; i < cuesStarts.length; ++i) {
if (inbandTrack1.cues[i].startTime != cuesStarts[i]) {
// Since we don't know the exact number of cues, only print
// output if the test fails.
testExpected("inbandTrack1.cues[" + i + "].startTime", cuesStarts[i]);
success = false;
}
}
[GStreamer] media/track/in-band/ layout tests introduced in r154908 are failing https://bugs.webkit.org/show_bug.cgi?id=120665 Reviewed by Philippe Normand. Source/WebCore: Solved a deadlock when processing samples from a text track while the main thread is starting to play. In those circumstances, the text sink is set by Playbin as non-async, so the main thread calling MediaPlayerPrivateGStreamer::play() is blocked waiting for the preroll to finish. While prerolling, a text sample is detected in a non-main thread and sent to the main thread for processing. The current code waits in the non-main thread for the processing to finish, but that won't ever happen because the main thread is busy in play() waiting for the preroll to complete. Since the non-main thread doesn't need any result from the main thread, a simple change to avoid the wait is enough to solve the deadlock. There was also a bug preventing all the languages of the text tracks being detected on time. Sometimes the tag event specifying the track language emitted by the demuxer arrives too late to the webkittextcombiner sink pad (up to several seconds since sent!) and the language remains unset when the test checks for it. As a result, the test fails because the second track isn't reported to be in French language ("fr"). The solution to this is to find an earlier (more upstream) pad and listen there for the (sticky) tag events. There can be multiple tag events, so all of them are checked until one having the GST_TAG_LANGUAGE_CODE tag is found. Covered by existing tests (now unskipped). * platform/graphics/gstreamer/TextSinkGStreamer.cpp: (webkitTextSinkHandleSample): Don't wait for handleTextSample() to be completed in the main thread. This avoids the deadlock. * platform/graphics/gstreamer/TrackPrivateBaseGStreamer.cpp: (WebCore::findBestUpstreamPad): Find a pad more upstream than the current one. (WebCore::TrackPrivateBaseGStreamer::TrackPrivateBaseGStreamer): Call setPad() instead of setting it by hand here. (WebCore::TrackPrivateBaseGStreamer::setPad): Also manage m_bestUpstreamPad in parallel to m_pad. Add a probe to listen to the tag events instead of checking the "tags" property (the property wasn't present in the best upstream pad). (WebCore::TrackPrivateBaseGStreamer::disconnect): Remove the probe from m_bestUpstreamPad, instead of removing the old notify::tags listener from m_pad. (WebCore::TrackPrivateBaseGStreamer::tagsChanged): Check the sticky tag events until the language one is found (or the last event if no language event is available). * platform/graphics/gstreamer/TrackPrivateBaseGStreamer.h: Added m_bestUpstreamPad and m_eventProbe. Moved setPad() implementation to the cpp file. Tools: Fix media/track/in-band/tests Add libkate as dependency, so gst-plugins-bad detects it and can build the GstKateDec element. This isn't actually needed when using Flatpak (the dependency has been added there), but is still useful when using jhbuild and is also coherent with the need of other multimedia dependencies such as libvpx. * gtk/install-dependencies: * wpe/install-dependencies: LayoutTests: Removed track-in-band-mpegts-metadata.html because it relies on private streams on mpegts and those won't ever be exposed by tsdemux. Made testCuesAddedOnce() test more robust when finding text tracks. * media/content/user-private-streams.ts: Removed. * media/in-band-tracks.js: (pollProgress): Make sure that the function repeated at intervals is properly stopped in all the cases. (canplaythrough): More robust track detection. (testCuesAddedOnce): Ditto. * media/track/in-band/track-in-band-mpegts-metadata.html: Removed. * platform/glib/TestExpectations: Unskip media/track/in-band tests. Canonical link: https://commits.webkit.org/237022@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@276586 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-04-26 11:59:51 +00:00
clearInterval(intervalId);
Source/WebCore: [GStreamer] support in-band text tracks https://bugs.webkit.org/show_bug.cgi?id=103771 Patch by Brendan Long <b.long@cablelabs.com> on 2013-08-30 Reviewed by Eric Carlson. Tests: New tests added because existing tests were too specific. media/track/in-band/track-in-band-kate-ogg-cues-added-once.html media/track/in-band/track-in-band-kate-ogg-kind.html media/track/in-band/track-in-band-kate-ogg-language.html media/track/in-band/track-in-band-kate-ogg-mode.html media/track/in-band/track-in-band-kate-ogg-style.html media/track/in-band/track-in-band-kate-ogg-track-order.html media/track/in-band/track-in-band-srt-mkv-cues-added-once.html media/track/in-band/track-in-band-srt-mkv-kind.html media/track/in-band/track-in-band-srt-mkv-language.html media/track/in-band/track-in-band-srt-mkv-mode.html media/track/in-band/track-in-band-srt-mkv-style.html media/track/in-band/track-in-band-srt-mkv-track-order.html * CMakeLists.txt: Add InbandTextTrackPrivateGStreamer, InbandGenericTextTrack, InbandWebVTTTextTrack, and TextCombinerGStreamer files. * GNUmakefile.list.am: Same. * PlatformEfl.cmake: Same. * Target.pri: Same. * WebCore.vcxproj/WebCore.vcxproj: Same. * WebCore.vcxproj/WebCore.vcxproj.filters: Same. * WebCore.xcodeproj/project.pbxproj: Same. * html/track/InbandGenericTextTrack.cpp: Split out code for handling generic cues. (WebCore::GenericTextTrackCueMap::GenericTextTrackCueMap): Move from InbandTextTrack. (WebCore::GenericTextTrackCueMap::~GenericTextTrackCueMap): Same. (WebCore::GenericTextTrackCueMap::add): Same. (WebCore::GenericTextTrackCueMap::find): Same. (WebCore::GenericTextTrackCueMap::remove): Same. (WebCore::InbandGenericTextTrack::create): Same. (WebCore::InbandGenericTextTrack::updateCueFromCueData): Same. (WebCore::InbandGenericTextTrack::addGenericCue): Same. (WebCore::InbandGenericTextTrack::updateGenericCue): Same. (WebCore::InbandGenericTextTrack::removeGenericCue): Same. (WebCore::InbandGenericTextTrack::removeCue): Same. (WebCore::InbandGenericTextTrack::InbandGenericTextTrack): Empty. (WebCore::InbandGenericTextTrack::~InbandGenericTextTrack): Empty. * html/track/InbandGenericTextTrack.h: Copied from Source/WebCore/html/track/InbandTextTrack.h. The only addition is the ASSERT_NOT_REACHED() for WebVTT cues. * html/track/InbandTextTrack.cpp: Add label and language changed callbacks. (WebCore::InbandTextTrack::create): Return a generic or WebVTT text track based on the private CueFormat. (WebCore::InbandTextTrack::labelChanged): Added. (WebCore::InbandTextTrack::languageChanged): Added. * html/track/InbandTextTrack.h: Add label and language changed callbacks. * html/track/InbandWebVTTTextTrack.cpp: Added, based on InbandTextTrack. (WebCore::InbandWebVTTTextTrack::create): Same. (WebCore::InbandWebVTTTextTrack::InbandWebVTTTextTrack): Empty. (WebCore::InbandWebVTTTextTrack::~InbandWebVTTTextTrack): Empty. (WebCore::InbandWebVTTTextTrack::parseWebVTTCueData): Sends data to a WebVTTParser. (WebCore::InbandWebVTTTextTrack::newCuesParsed): Adds cues when WebVTTParser parses them. (WebCore::InbandWebVTTTextTrack::fileFailedToParse): Prints a warning when WebVTTParser has a problem. * html/track/InbandWebVTTTextTrack.h: Added. * platform/graphics/InbandTextTrackPrivate.h: (WebCore::InbandTextTrackPrivate::cueFormat): For determining if the track will have generic or WebVTT cues. (WebCore::InbandTextTrackPrivate::InbandTextTrackPrivate): Pass CueFormat in the constructor. * platform/graphics/InbandTextTrackPrivateClient.h: Same. * platform/graphics/avfoundation/InbandTextTrackPrivateAVF.cpp: (WebCore::InbandTextTrackPrivateAVF::InbandTextTrackPrivateAVF): Pass CueFormat (Generic) to InbandTextTrackPrivate. * platform/graphics/gstreamer/GRefPtrGStreamer.cpp: Add GRefPtr specializations for GstSample and GstEvent. * platform/graphics/gstreamer/GRefPtrGStreamer.h: Same. * platform/graphics/gstreamer/GStreamerUtilities.h: Add WARN_MEDIA_MESSAGE. * platform/graphics/gstreamer/GStreamerVersioning.h: Add a function to check GStreamer version at runtime. * platform/graphics/gstreamer/InbandTextTrackPrivateGStreamer.cpp: Added. (WebCore::textTrackPrivateEventCallback): Watches for tag and stream start events. (WebCore::textTrackPrivateSampleTimeoutCallback): See notifyTrackOfSample(); (WebCore::textTrackPrivateStreamTimeoutCallback): See notifyTrackOfStreamChanged(); (WebCore::textTrackPrivateTagsChangeTimeoutCallback): See notifyTrackOfTagsChanged(); (WebCore::InbandTextTrackPrivateGStreamer::InbandTextTrackPrivateGStreamer): Initializes tags and stream and sets up event callback. (WebCore::InbandTextTrackPrivateGStreamer::~InbandTextTrackPrivateGStreamer): Calls disconnect: (WebCore::InbandTextTrackPrivateGStreamer::disconnect): Removes signal handlers and frees some memory. (WebCore::InbandTextTrackPrivateGStreamer::handleSample): Adds samples to a list and sets up callback. (WebCore::InbandTextTrackPrivateGStreamer::streamChanged): Sets up callback. (WebCore::InbandTextTrackPrivateGStreamer::tagsChanged): Same. (WebCore::InbandTextTrackPrivateGStreamer::notifyTrackOfSample): Parses all queued samples with WebVTTParser. (WebCore::InbandTextTrackPrivateGStreamer::notifyTrackOfStreamChanged): Keeps track of current stream. (WebCore::InbandTextTrackPrivateGStreamer::notifyTrackOfTagsChanged): Sets label and language from tags. * platform/graphics/gstreamer/InbandTextTrackPrivateGStreamer.h: Added. (WebCore::InbandTextTrackPrivateGStreamer::create): Basic RefPtr create function. (WebCore::InbandTextTrackPrivateGStreamer::pad): Returns the pad this track is associated with (used to determine if a playbin text stream has already been associated with a text track). (WebCore::InbandTextTrackPrivateGStreamer::setIndex): Sets the track index (used for sorting). (WebCore::InbandTextTrackPrivateGStreamer::streamId): Returns the stream ID (used to handle new samples). * platform/graphics/gstreamer/MediaPlayerPrivateGStreamer.cpp: (WebCore::mediaPlayerPrivateTextChangedCallback): Called for playbin "text-changed" event. See textChanged(). (WebCore::mediaPlayerPrivateTextChangeTimeoutCallback): See notifyPlayerOfText(). (WebCore::mediaPlayerPrivateNewTextSampleCallback): See newTextSample(). (WebCore::MediaPlayerPrivateGStreamer::MediaPlayerPrivateGStreamer): Initialize m_textTimerHandler. (WebCore::MediaPlayerPrivateGStreamer::~MediaPlayerPrivateGStreamer): Disconnect text tracks (they don't necessarily get freed here, since a script could hold a reference). (WebCore::MediaPlayerPrivateGStreamer::textChanged): Setup callback for notifyPlayerOfText. (WebCore::MediaPlayerPrivateGStreamer::notifyPlayerOfText): Create text tracks. (WebCore::MediaPlayerPrivateGStreamer::newTextSample): Handle new samples by giving them to a text track with a matching stream. This method is syncryonous because we need to get the stream start sticky event immediately. * platform/graphics/gstreamer/MediaPlayerPrivateGStreamer.h: * platform/graphics/gstreamer/TextCombinerGStreamer.cpp: Added. This element forwards buffers from all of its input pads, but also converts plain text to WebVTT as needed. (webkit_text_combiner_init): Setup internal funnel. (webkitTextCombinerPadEvent): If the caps are plain text make sure we have a webvttenv, otherwise connect directly to the funnel. (webkitTextCombinerRequestNewPad): Setup ghostpad and event callback. (webkitTextCombinerReleasePad): Release pad and optional associated webvttenc. (webkit_text_combiner_class_init): Setup pad templates and request/release pad functions. (webkitTextCombinerNew): Returns a new WebKitTextCombiner. * platform/graphics/gstreamer/TextCombinerGStreamer.h: Added. * platform/graphics/gstreamer/TextSinkGStreamer.cpp: Added. (webkit_text_sink_init): Set sync=false. (webkitTextSinkGetProperty): Ignore sync property. (webkitTextSinkSetProperty): Same. (webkitTextSinkQuery): Ignore position and duration queries, forward everything else to appsink. (webkit_text_sink_class_init): Setup property and query functions. (webkitTextSinkNew): Return a new WebKitTextSink. * platform/graphics/gstreamer/TextSinkGStreamer.h: Added. LayoutTests: <https://webkit.org/b/103771> [GStreamer] support in-band text tracks Patch by Brendan Long <b.long@cablelabs.com> on 2013-08-30 Reviewed by Eric Carlson. * media/content/counting-subtitled-kate.ogv: Added. * media/content/counting-subtitled-srt.mkv: Added. * media/in-band-cues.js: Added. * media/track/in-band/track-in-band-kate-ogg-cues-added-once-expected.txt: Added. * media/track/in-band/track-in-band-kate-ogg-cues-added-once.html: Added. * media/track/in-band/track-in-band-kate-ogg-kind-expected.txt: Added. * media/track/in-band/track-in-band-kate-ogg-kind.html: Added. * media/track/in-band/track-in-band-kate-ogg-language-expected.txt: Added. * media/track/in-band/track-in-band-kate-ogg-language.html: Added. * media/track/in-band/track-in-band-kate-ogg-mode-expected.txt: Added. * media/track/in-band/track-in-band-kate-ogg-mode.html: Added. * media/track/in-band/track-in-band-kate-ogg-style-expected.txt: Added. * media/track/in-band/track-in-band-kate-ogg-style.html: Added. * media/track/in-band/track-in-band-kate-ogg-track-order-expected.txt: Added. * media/track/in-band/track-in-band-kate-ogg-track-order.html: Added. * media/track/in-band/track-in-band-srt-mkv-cues-added-once-expected.txt: Added. * media/track/in-band/track-in-band-srt-mkv-cues-added-once.html: Added. * media/track/in-band/track-in-band-srt-mkv-kind-expected.txt: Added. * media/track/in-band/track-in-band-srt-mkv-kind.html: Added. * media/track/in-band/track-in-band-srt-mkv-language-expected.txt: Added. * media/track/in-band/track-in-band-srt-mkv-language.html: Added. * media/track/in-band/track-in-band-srt-mkv-mode-expected.txt: Added. * media/track/in-band/track-in-band-srt-mkv-mode.html: Added. * media/track/in-band/track-in-band-srt-mkv-style-expected.txt: Added. * media/track/in-band/track-in-band-srt-mkv-style.html: Added. * media/track/in-band/track-in-band-srt-mkv-track-order-expected.txt: Added. * media/track/in-band/track-in-band-srt-mkv-track-order.html: Added. * platform/mac/TestExpectations: Skip MKV and OGG tests. Canonical link: https://commits.webkit.org/138536@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@154908 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2013-08-30 20:17:27 +00:00
logResult(success, "Test all cues are equal");
endTest();
} catch (e) {
[GStreamer] media/track/in-band/ layout tests introduced in r154908 are failing https://bugs.webkit.org/show_bug.cgi?id=120665 Reviewed by Philippe Normand. Source/WebCore: Solved a deadlock when processing samples from a text track while the main thread is starting to play. In those circumstances, the text sink is set by Playbin as non-async, so the main thread calling MediaPlayerPrivateGStreamer::play() is blocked waiting for the preroll to finish. While prerolling, a text sample is detected in a non-main thread and sent to the main thread for processing. The current code waits in the non-main thread for the processing to finish, but that won't ever happen because the main thread is busy in play() waiting for the preroll to complete. Since the non-main thread doesn't need any result from the main thread, a simple change to avoid the wait is enough to solve the deadlock. There was also a bug preventing all the languages of the text tracks being detected on time. Sometimes the tag event specifying the track language emitted by the demuxer arrives too late to the webkittextcombiner sink pad (up to several seconds since sent!) and the language remains unset when the test checks for it. As a result, the test fails because the second track isn't reported to be in French language ("fr"). The solution to this is to find an earlier (more upstream) pad and listen there for the (sticky) tag events. There can be multiple tag events, so all of them are checked until one having the GST_TAG_LANGUAGE_CODE tag is found. Covered by existing tests (now unskipped). * platform/graphics/gstreamer/TextSinkGStreamer.cpp: (webkitTextSinkHandleSample): Don't wait for handleTextSample() to be completed in the main thread. This avoids the deadlock. * platform/graphics/gstreamer/TrackPrivateBaseGStreamer.cpp: (WebCore::findBestUpstreamPad): Find a pad more upstream than the current one. (WebCore::TrackPrivateBaseGStreamer::TrackPrivateBaseGStreamer): Call setPad() instead of setting it by hand here. (WebCore::TrackPrivateBaseGStreamer::setPad): Also manage m_bestUpstreamPad in parallel to m_pad. Add a probe to listen to the tag events instead of checking the "tags" property (the property wasn't present in the best upstream pad). (WebCore::TrackPrivateBaseGStreamer::disconnect): Remove the probe from m_bestUpstreamPad, instead of removing the old notify::tags listener from m_pad. (WebCore::TrackPrivateBaseGStreamer::tagsChanged): Check the sticky tag events until the language one is found (or the last event if no language event is available). * platform/graphics/gstreamer/TrackPrivateBaseGStreamer.h: Added m_bestUpstreamPad and m_eventProbe. Moved setPad() implementation to the cpp file. Tools: Fix media/track/in-band/tests Add libkate as dependency, so gst-plugins-bad detects it and can build the GstKateDec element. This isn't actually needed when using Flatpak (the dependency has been added there), but is still useful when using jhbuild and is also coherent with the need of other multimedia dependencies such as libvpx. * gtk/install-dependencies: * wpe/install-dependencies: LayoutTests: Removed track-in-band-mpegts-metadata.html because it relies on private streams on mpegts and those won't ever be exposed by tsdemux. Made testCuesAddedOnce() test more robust when finding text tracks. * media/content/user-private-streams.ts: Removed. * media/in-band-tracks.js: (pollProgress): Make sure that the function repeated at intervals is properly stopped in all the cases. (canplaythrough): More robust track detection. (testCuesAddedOnce): Ditto. * media/track/in-band/track-in-band-mpegts-metadata.html: Removed. * platform/glib/TestExpectations: Unskip media/track/in-band tests. Canonical link: https://commits.webkit.org/237022@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@276586 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-04-26 11:59:51 +00:00
clearInterval(intervalId);
Source/WebCore: [GStreamer] support in-band text tracks https://bugs.webkit.org/show_bug.cgi?id=103771 Patch by Brendan Long <b.long@cablelabs.com> on 2013-08-30 Reviewed by Eric Carlson. Tests: New tests added because existing tests were too specific. media/track/in-band/track-in-band-kate-ogg-cues-added-once.html media/track/in-band/track-in-band-kate-ogg-kind.html media/track/in-band/track-in-band-kate-ogg-language.html media/track/in-band/track-in-band-kate-ogg-mode.html media/track/in-band/track-in-band-kate-ogg-style.html media/track/in-band/track-in-band-kate-ogg-track-order.html media/track/in-band/track-in-band-srt-mkv-cues-added-once.html media/track/in-band/track-in-band-srt-mkv-kind.html media/track/in-band/track-in-band-srt-mkv-language.html media/track/in-band/track-in-band-srt-mkv-mode.html media/track/in-band/track-in-band-srt-mkv-style.html media/track/in-band/track-in-band-srt-mkv-track-order.html * CMakeLists.txt: Add InbandTextTrackPrivateGStreamer, InbandGenericTextTrack, InbandWebVTTTextTrack, and TextCombinerGStreamer files. * GNUmakefile.list.am: Same. * PlatformEfl.cmake: Same. * Target.pri: Same. * WebCore.vcxproj/WebCore.vcxproj: Same. * WebCore.vcxproj/WebCore.vcxproj.filters: Same. * WebCore.xcodeproj/project.pbxproj: Same. * html/track/InbandGenericTextTrack.cpp: Split out code for handling generic cues. (WebCore::GenericTextTrackCueMap::GenericTextTrackCueMap): Move from InbandTextTrack. (WebCore::GenericTextTrackCueMap::~GenericTextTrackCueMap): Same. (WebCore::GenericTextTrackCueMap::add): Same. (WebCore::GenericTextTrackCueMap::find): Same. (WebCore::GenericTextTrackCueMap::remove): Same. (WebCore::InbandGenericTextTrack::create): Same. (WebCore::InbandGenericTextTrack::updateCueFromCueData): Same. (WebCore::InbandGenericTextTrack::addGenericCue): Same. (WebCore::InbandGenericTextTrack::updateGenericCue): Same. (WebCore::InbandGenericTextTrack::removeGenericCue): Same. (WebCore::InbandGenericTextTrack::removeCue): Same. (WebCore::InbandGenericTextTrack::InbandGenericTextTrack): Empty. (WebCore::InbandGenericTextTrack::~InbandGenericTextTrack): Empty. * html/track/InbandGenericTextTrack.h: Copied from Source/WebCore/html/track/InbandTextTrack.h. The only addition is the ASSERT_NOT_REACHED() for WebVTT cues. * html/track/InbandTextTrack.cpp: Add label and language changed callbacks. (WebCore::InbandTextTrack::create): Return a generic or WebVTT text track based on the private CueFormat. (WebCore::InbandTextTrack::labelChanged): Added. (WebCore::InbandTextTrack::languageChanged): Added. * html/track/InbandTextTrack.h: Add label and language changed callbacks. * html/track/InbandWebVTTTextTrack.cpp: Added, based on InbandTextTrack. (WebCore::InbandWebVTTTextTrack::create): Same. (WebCore::InbandWebVTTTextTrack::InbandWebVTTTextTrack): Empty. (WebCore::InbandWebVTTTextTrack::~InbandWebVTTTextTrack): Empty. (WebCore::InbandWebVTTTextTrack::parseWebVTTCueData): Sends data to a WebVTTParser. (WebCore::InbandWebVTTTextTrack::newCuesParsed): Adds cues when WebVTTParser parses them. (WebCore::InbandWebVTTTextTrack::fileFailedToParse): Prints a warning when WebVTTParser has a problem. * html/track/InbandWebVTTTextTrack.h: Added. * platform/graphics/InbandTextTrackPrivate.h: (WebCore::InbandTextTrackPrivate::cueFormat): For determining if the track will have generic or WebVTT cues. (WebCore::InbandTextTrackPrivate::InbandTextTrackPrivate): Pass CueFormat in the constructor. * platform/graphics/InbandTextTrackPrivateClient.h: Same. * platform/graphics/avfoundation/InbandTextTrackPrivateAVF.cpp: (WebCore::InbandTextTrackPrivateAVF::InbandTextTrackPrivateAVF): Pass CueFormat (Generic) to InbandTextTrackPrivate. * platform/graphics/gstreamer/GRefPtrGStreamer.cpp: Add GRefPtr specializations for GstSample and GstEvent. * platform/graphics/gstreamer/GRefPtrGStreamer.h: Same. * platform/graphics/gstreamer/GStreamerUtilities.h: Add WARN_MEDIA_MESSAGE. * platform/graphics/gstreamer/GStreamerVersioning.h: Add a function to check GStreamer version at runtime. * platform/graphics/gstreamer/InbandTextTrackPrivateGStreamer.cpp: Added. (WebCore::textTrackPrivateEventCallback): Watches for tag and stream start events. (WebCore::textTrackPrivateSampleTimeoutCallback): See notifyTrackOfSample(); (WebCore::textTrackPrivateStreamTimeoutCallback): See notifyTrackOfStreamChanged(); (WebCore::textTrackPrivateTagsChangeTimeoutCallback): See notifyTrackOfTagsChanged(); (WebCore::InbandTextTrackPrivateGStreamer::InbandTextTrackPrivateGStreamer): Initializes tags and stream and sets up event callback. (WebCore::InbandTextTrackPrivateGStreamer::~InbandTextTrackPrivateGStreamer): Calls disconnect: (WebCore::InbandTextTrackPrivateGStreamer::disconnect): Removes signal handlers and frees some memory. (WebCore::InbandTextTrackPrivateGStreamer::handleSample): Adds samples to a list and sets up callback. (WebCore::InbandTextTrackPrivateGStreamer::streamChanged): Sets up callback. (WebCore::InbandTextTrackPrivateGStreamer::tagsChanged): Same. (WebCore::InbandTextTrackPrivateGStreamer::notifyTrackOfSample): Parses all queued samples with WebVTTParser. (WebCore::InbandTextTrackPrivateGStreamer::notifyTrackOfStreamChanged): Keeps track of current stream. (WebCore::InbandTextTrackPrivateGStreamer::notifyTrackOfTagsChanged): Sets label and language from tags. * platform/graphics/gstreamer/InbandTextTrackPrivateGStreamer.h: Added. (WebCore::InbandTextTrackPrivateGStreamer::create): Basic RefPtr create function. (WebCore::InbandTextTrackPrivateGStreamer::pad): Returns the pad this track is associated with (used to determine if a playbin text stream has already been associated with a text track). (WebCore::InbandTextTrackPrivateGStreamer::setIndex): Sets the track index (used for sorting). (WebCore::InbandTextTrackPrivateGStreamer::streamId): Returns the stream ID (used to handle new samples). * platform/graphics/gstreamer/MediaPlayerPrivateGStreamer.cpp: (WebCore::mediaPlayerPrivateTextChangedCallback): Called for playbin "text-changed" event. See textChanged(). (WebCore::mediaPlayerPrivateTextChangeTimeoutCallback): See notifyPlayerOfText(). (WebCore::mediaPlayerPrivateNewTextSampleCallback): See newTextSample(). (WebCore::MediaPlayerPrivateGStreamer::MediaPlayerPrivateGStreamer): Initialize m_textTimerHandler. (WebCore::MediaPlayerPrivateGStreamer::~MediaPlayerPrivateGStreamer): Disconnect text tracks (they don't necessarily get freed here, since a script could hold a reference). (WebCore::MediaPlayerPrivateGStreamer::textChanged): Setup callback for notifyPlayerOfText. (WebCore::MediaPlayerPrivateGStreamer::notifyPlayerOfText): Create text tracks. (WebCore::MediaPlayerPrivateGStreamer::newTextSample): Handle new samples by giving them to a text track with a matching stream. This method is syncryonous because we need to get the stream start sticky event immediately. * platform/graphics/gstreamer/MediaPlayerPrivateGStreamer.h: * platform/graphics/gstreamer/TextCombinerGStreamer.cpp: Added. This element forwards buffers from all of its input pads, but also converts plain text to WebVTT as needed. (webkit_text_combiner_init): Setup internal funnel. (webkitTextCombinerPadEvent): If the caps are plain text make sure we have a webvttenv, otherwise connect directly to the funnel. (webkitTextCombinerRequestNewPad): Setup ghostpad and event callback. (webkitTextCombinerReleasePad): Release pad and optional associated webvttenc. (webkit_text_combiner_class_init): Setup pad templates and request/release pad functions. (webkitTextCombinerNew): Returns a new WebKitTextCombiner. * platform/graphics/gstreamer/TextCombinerGStreamer.h: Added. * platform/graphics/gstreamer/TextSinkGStreamer.cpp: Added. (webkit_text_sink_init): Set sync=false. (webkitTextSinkGetProperty): Ignore sync property. (webkitTextSinkSetProperty): Same. (webkitTextSinkQuery): Ignore position and duration queries, forward everything else to appsink. (webkit_text_sink_class_init): Setup property and query functions. (webkitTextSinkNew): Return a new WebKitTextSink. * platform/graphics/gstreamer/TextSinkGStreamer.h: Added. LayoutTests: <https://webkit.org/b/103771> [GStreamer] support in-band text tracks Patch by Brendan Long <b.long@cablelabs.com> on 2013-08-30 Reviewed by Eric Carlson. * media/content/counting-subtitled-kate.ogv: Added. * media/content/counting-subtitled-srt.mkv: Added. * media/in-band-cues.js: Added. * media/track/in-band/track-in-band-kate-ogg-cues-added-once-expected.txt: Added. * media/track/in-band/track-in-band-kate-ogg-cues-added-once.html: Added. * media/track/in-band/track-in-band-kate-ogg-kind-expected.txt: Added. * media/track/in-band/track-in-band-kate-ogg-kind.html: Added. * media/track/in-band/track-in-band-kate-ogg-language-expected.txt: Added. * media/track/in-band/track-in-band-kate-ogg-language.html: Added. * media/track/in-band/track-in-band-kate-ogg-mode-expected.txt: Added. * media/track/in-band/track-in-band-kate-ogg-mode.html: Added. * media/track/in-band/track-in-band-kate-ogg-style-expected.txt: Added. * media/track/in-band/track-in-band-kate-ogg-style.html: Added. * media/track/in-band/track-in-band-kate-ogg-track-order-expected.txt: Added. * media/track/in-band/track-in-band-kate-ogg-track-order.html: Added. * media/track/in-band/track-in-band-srt-mkv-cues-added-once-expected.txt: Added. * media/track/in-band/track-in-band-srt-mkv-cues-added-once.html: Added. * media/track/in-band/track-in-band-srt-mkv-kind-expected.txt: Added. * media/track/in-band/track-in-band-srt-mkv-kind.html: Added. * media/track/in-band/track-in-band-srt-mkv-language-expected.txt: Added. * media/track/in-band/track-in-band-srt-mkv-language.html: Added. * media/track/in-band/track-in-band-srt-mkv-mode-expected.txt: Added. * media/track/in-band/track-in-band-srt-mkv-mode.html: Added. * media/track/in-band/track-in-band-srt-mkv-style-expected.txt: Added. * media/track/in-band/track-in-band-srt-mkv-style.html: Added. * media/track/in-band/track-in-band-srt-mkv-track-order-expected.txt: Added. * media/track/in-band/track-in-band-srt-mkv-track-order.html: Added. * platform/mac/TestExpectations: Skip MKV and OGG tests. Canonical link: https://commits.webkit.org/138536@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@154908 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2013-08-30 20:17:27 +00:00
failTest(e);
}
}
function canplaythrough()
{
waitForEvent('seeked', function() { ++seekedCount; });
[GStreamer] media/track/in-band/ layout tests introduced in r154908 are failing https://bugs.webkit.org/show_bug.cgi?id=120665 Reviewed by Philippe Normand. Source/WebCore: Solved a deadlock when processing samples from a text track while the main thread is starting to play. In those circumstances, the text sink is set by Playbin as non-async, so the main thread calling MediaPlayerPrivateGStreamer::play() is blocked waiting for the preroll to finish. While prerolling, a text sample is detected in a non-main thread and sent to the main thread for processing. The current code waits in the non-main thread for the processing to finish, but that won't ever happen because the main thread is busy in play() waiting for the preroll to complete. Since the non-main thread doesn't need any result from the main thread, a simple change to avoid the wait is enough to solve the deadlock. There was also a bug preventing all the languages of the text tracks being detected on time. Sometimes the tag event specifying the track language emitted by the demuxer arrives too late to the webkittextcombiner sink pad (up to several seconds since sent!) and the language remains unset when the test checks for it. As a result, the test fails because the second track isn't reported to be in French language ("fr"). The solution to this is to find an earlier (more upstream) pad and listen there for the (sticky) tag events. There can be multiple tag events, so all of them are checked until one having the GST_TAG_LANGUAGE_CODE tag is found. Covered by existing tests (now unskipped). * platform/graphics/gstreamer/TextSinkGStreamer.cpp: (webkitTextSinkHandleSample): Don't wait for handleTextSample() to be completed in the main thread. This avoids the deadlock. * platform/graphics/gstreamer/TrackPrivateBaseGStreamer.cpp: (WebCore::findBestUpstreamPad): Find a pad more upstream than the current one. (WebCore::TrackPrivateBaseGStreamer::TrackPrivateBaseGStreamer): Call setPad() instead of setting it by hand here. (WebCore::TrackPrivateBaseGStreamer::setPad): Also manage m_bestUpstreamPad in parallel to m_pad. Add a probe to listen to the tag events instead of checking the "tags" property (the property wasn't present in the best upstream pad). (WebCore::TrackPrivateBaseGStreamer::disconnect): Remove the probe from m_bestUpstreamPad, instead of removing the old notify::tags listener from m_pad. (WebCore::TrackPrivateBaseGStreamer::tagsChanged): Check the sticky tag events until the language one is found (or the last event if no language event is available). * platform/graphics/gstreamer/TrackPrivateBaseGStreamer.h: Added m_bestUpstreamPad and m_eventProbe. Moved setPad() implementation to the cpp file. Tools: Fix media/track/in-band/tests Add libkate as dependency, so gst-plugins-bad detects it and can build the GstKateDec element. This isn't actually needed when using Flatpak (the dependency has been added there), but is still useful when using jhbuild and is also coherent with the need of other multimedia dependencies such as libvpx. * gtk/install-dependencies: * wpe/install-dependencies: LayoutTests: Removed track-in-band-mpegts-metadata.html because it relies on private streams on mpegts and those won't ever be exposed by tsdemux. Made testCuesAddedOnce() test more robust when finding text tracks. * media/content/user-private-streams.ts: Removed. * media/in-band-tracks.js: (pollProgress): Make sure that the function repeated at intervals is properly stopped in all the cases. (canplaythrough): More robust track detection. (testCuesAddedOnce): Ditto. * media/track/in-band/track-in-band-mpegts-metadata.html: Removed. * platform/glib/TestExpectations: Unskip media/track/in-band tests. Canonical link: https://commits.webkit.org/237022@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@276586 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-04-26 11:59:51 +00:00
intervalId = setInterval(pollProgress, 100);
Source/WebCore: [GStreamer] support in-band text tracks https://bugs.webkit.org/show_bug.cgi?id=103771 Patch by Brendan Long <b.long@cablelabs.com> on 2013-08-30 Reviewed by Eric Carlson. Tests: New tests added because existing tests were too specific. media/track/in-band/track-in-band-kate-ogg-cues-added-once.html media/track/in-band/track-in-band-kate-ogg-kind.html media/track/in-band/track-in-band-kate-ogg-language.html media/track/in-band/track-in-band-kate-ogg-mode.html media/track/in-band/track-in-band-kate-ogg-style.html media/track/in-band/track-in-band-kate-ogg-track-order.html media/track/in-band/track-in-band-srt-mkv-cues-added-once.html media/track/in-band/track-in-band-srt-mkv-kind.html media/track/in-band/track-in-band-srt-mkv-language.html media/track/in-band/track-in-band-srt-mkv-mode.html media/track/in-band/track-in-band-srt-mkv-style.html media/track/in-band/track-in-band-srt-mkv-track-order.html * CMakeLists.txt: Add InbandTextTrackPrivateGStreamer, InbandGenericTextTrack, InbandWebVTTTextTrack, and TextCombinerGStreamer files. * GNUmakefile.list.am: Same. * PlatformEfl.cmake: Same. * Target.pri: Same. * WebCore.vcxproj/WebCore.vcxproj: Same. * WebCore.vcxproj/WebCore.vcxproj.filters: Same. * WebCore.xcodeproj/project.pbxproj: Same. * html/track/InbandGenericTextTrack.cpp: Split out code for handling generic cues. (WebCore::GenericTextTrackCueMap::GenericTextTrackCueMap): Move from InbandTextTrack. (WebCore::GenericTextTrackCueMap::~GenericTextTrackCueMap): Same. (WebCore::GenericTextTrackCueMap::add): Same. (WebCore::GenericTextTrackCueMap::find): Same. (WebCore::GenericTextTrackCueMap::remove): Same. (WebCore::InbandGenericTextTrack::create): Same. (WebCore::InbandGenericTextTrack::updateCueFromCueData): Same. (WebCore::InbandGenericTextTrack::addGenericCue): Same. (WebCore::InbandGenericTextTrack::updateGenericCue): Same. (WebCore::InbandGenericTextTrack::removeGenericCue): Same. (WebCore::InbandGenericTextTrack::removeCue): Same. (WebCore::InbandGenericTextTrack::InbandGenericTextTrack): Empty. (WebCore::InbandGenericTextTrack::~InbandGenericTextTrack): Empty. * html/track/InbandGenericTextTrack.h: Copied from Source/WebCore/html/track/InbandTextTrack.h. The only addition is the ASSERT_NOT_REACHED() for WebVTT cues. * html/track/InbandTextTrack.cpp: Add label and language changed callbacks. (WebCore::InbandTextTrack::create): Return a generic or WebVTT text track based on the private CueFormat. (WebCore::InbandTextTrack::labelChanged): Added. (WebCore::InbandTextTrack::languageChanged): Added. * html/track/InbandTextTrack.h: Add label and language changed callbacks. * html/track/InbandWebVTTTextTrack.cpp: Added, based on InbandTextTrack. (WebCore::InbandWebVTTTextTrack::create): Same. (WebCore::InbandWebVTTTextTrack::InbandWebVTTTextTrack): Empty. (WebCore::InbandWebVTTTextTrack::~InbandWebVTTTextTrack): Empty. (WebCore::InbandWebVTTTextTrack::parseWebVTTCueData): Sends data to a WebVTTParser. (WebCore::InbandWebVTTTextTrack::newCuesParsed): Adds cues when WebVTTParser parses them. (WebCore::InbandWebVTTTextTrack::fileFailedToParse): Prints a warning when WebVTTParser has a problem. * html/track/InbandWebVTTTextTrack.h: Added. * platform/graphics/InbandTextTrackPrivate.h: (WebCore::InbandTextTrackPrivate::cueFormat): For determining if the track will have generic or WebVTT cues. (WebCore::InbandTextTrackPrivate::InbandTextTrackPrivate): Pass CueFormat in the constructor. * platform/graphics/InbandTextTrackPrivateClient.h: Same. * platform/graphics/avfoundation/InbandTextTrackPrivateAVF.cpp: (WebCore::InbandTextTrackPrivateAVF::InbandTextTrackPrivateAVF): Pass CueFormat (Generic) to InbandTextTrackPrivate. * platform/graphics/gstreamer/GRefPtrGStreamer.cpp: Add GRefPtr specializations for GstSample and GstEvent. * platform/graphics/gstreamer/GRefPtrGStreamer.h: Same. * platform/graphics/gstreamer/GStreamerUtilities.h: Add WARN_MEDIA_MESSAGE. * platform/graphics/gstreamer/GStreamerVersioning.h: Add a function to check GStreamer version at runtime. * platform/graphics/gstreamer/InbandTextTrackPrivateGStreamer.cpp: Added. (WebCore::textTrackPrivateEventCallback): Watches for tag and stream start events. (WebCore::textTrackPrivateSampleTimeoutCallback): See notifyTrackOfSample(); (WebCore::textTrackPrivateStreamTimeoutCallback): See notifyTrackOfStreamChanged(); (WebCore::textTrackPrivateTagsChangeTimeoutCallback): See notifyTrackOfTagsChanged(); (WebCore::InbandTextTrackPrivateGStreamer::InbandTextTrackPrivateGStreamer): Initializes tags and stream and sets up event callback. (WebCore::InbandTextTrackPrivateGStreamer::~InbandTextTrackPrivateGStreamer): Calls disconnect: (WebCore::InbandTextTrackPrivateGStreamer::disconnect): Removes signal handlers and frees some memory. (WebCore::InbandTextTrackPrivateGStreamer::handleSample): Adds samples to a list and sets up callback. (WebCore::InbandTextTrackPrivateGStreamer::streamChanged): Sets up callback. (WebCore::InbandTextTrackPrivateGStreamer::tagsChanged): Same. (WebCore::InbandTextTrackPrivateGStreamer::notifyTrackOfSample): Parses all queued samples with WebVTTParser. (WebCore::InbandTextTrackPrivateGStreamer::notifyTrackOfStreamChanged): Keeps track of current stream. (WebCore::InbandTextTrackPrivateGStreamer::notifyTrackOfTagsChanged): Sets label and language from tags. * platform/graphics/gstreamer/InbandTextTrackPrivateGStreamer.h: Added. (WebCore::InbandTextTrackPrivateGStreamer::create): Basic RefPtr create function. (WebCore::InbandTextTrackPrivateGStreamer::pad): Returns the pad this track is associated with (used to determine if a playbin text stream has already been associated with a text track). (WebCore::InbandTextTrackPrivateGStreamer::setIndex): Sets the track index (used for sorting). (WebCore::InbandTextTrackPrivateGStreamer::streamId): Returns the stream ID (used to handle new samples). * platform/graphics/gstreamer/MediaPlayerPrivateGStreamer.cpp: (WebCore::mediaPlayerPrivateTextChangedCallback): Called for playbin "text-changed" event. See textChanged(). (WebCore::mediaPlayerPrivateTextChangeTimeoutCallback): See notifyPlayerOfText(). (WebCore::mediaPlayerPrivateNewTextSampleCallback): See newTextSample(). (WebCore::MediaPlayerPrivateGStreamer::MediaPlayerPrivateGStreamer): Initialize m_textTimerHandler. (WebCore::MediaPlayerPrivateGStreamer::~MediaPlayerPrivateGStreamer): Disconnect text tracks (they don't necessarily get freed here, since a script could hold a reference). (WebCore::MediaPlayerPrivateGStreamer::textChanged): Setup callback for notifyPlayerOfText. (WebCore::MediaPlayerPrivateGStreamer::notifyPlayerOfText): Create text tracks. (WebCore::MediaPlayerPrivateGStreamer::newTextSample): Handle new samples by giving them to a text track with a matching stream. This method is syncryonous because we need to get the stream start sticky event immediately. * platform/graphics/gstreamer/MediaPlayerPrivateGStreamer.h: * platform/graphics/gstreamer/TextCombinerGStreamer.cpp: Added. This element forwards buffers from all of its input pads, but also converts plain text to WebVTT as needed. (webkit_text_combiner_init): Setup internal funnel. (webkitTextCombinerPadEvent): If the caps are plain text make sure we have a webvttenv, otherwise connect directly to the funnel. (webkitTextCombinerRequestNewPad): Setup ghostpad and event callback. (webkitTextCombinerReleasePad): Release pad and optional associated webvttenc. (webkit_text_combiner_class_init): Setup pad templates and request/release pad functions. (webkitTextCombinerNew): Returns a new WebKitTextCombiner. * platform/graphics/gstreamer/TextCombinerGStreamer.h: Added. * platform/graphics/gstreamer/TextSinkGStreamer.cpp: Added. (webkit_text_sink_init): Set sync=false. (webkitTextSinkGetProperty): Ignore sync property. (webkitTextSinkSetProperty): Same. (webkitTextSinkQuery): Ignore position and duration queries, forward everything else to appsink. (webkit_text_sink_class_init): Setup property and query functions. (webkitTextSinkNew): Return a new WebKitTextSink. * platform/graphics/gstreamer/TextSinkGStreamer.h: Added. LayoutTests: <https://webkit.org/b/103771> [GStreamer] support in-band text tracks Patch by Brendan Long <b.long@cablelabs.com> on 2013-08-30 Reviewed by Eric Carlson. * media/content/counting-subtitled-kate.ogv: Added. * media/content/counting-subtitled-srt.mkv: Added. * media/in-band-cues.js: Added. * media/track/in-band/track-in-band-kate-ogg-cues-added-once-expected.txt: Added. * media/track/in-band/track-in-band-kate-ogg-cues-added-once.html: Added. * media/track/in-band/track-in-band-kate-ogg-kind-expected.txt: Added. * media/track/in-band/track-in-band-kate-ogg-kind.html: Added. * media/track/in-band/track-in-band-kate-ogg-language-expected.txt: Added. * media/track/in-band/track-in-band-kate-ogg-language.html: Added. * media/track/in-band/track-in-band-kate-ogg-mode-expected.txt: Added. * media/track/in-band/track-in-band-kate-ogg-mode.html: Added. * media/track/in-band/track-in-band-kate-ogg-style-expected.txt: Added. * media/track/in-band/track-in-band-kate-ogg-style.html: Added. * media/track/in-band/track-in-band-kate-ogg-track-order-expected.txt: Added. * media/track/in-band/track-in-band-kate-ogg-track-order.html: Added. * media/track/in-band/track-in-band-srt-mkv-cues-added-once-expected.txt: Added. * media/track/in-band/track-in-band-srt-mkv-cues-added-once.html: Added. * media/track/in-band/track-in-band-srt-mkv-kind-expected.txt: Added. * media/track/in-band/track-in-band-srt-mkv-kind.html: Added. * media/track/in-band/track-in-band-srt-mkv-language-expected.txt: Added. * media/track/in-band/track-in-band-srt-mkv-language.html: Added. * media/track/in-band/track-in-band-srt-mkv-mode-expected.txt: Added. * media/track/in-band/track-in-band-srt-mkv-mode.html: Added. * media/track/in-band/track-in-band-srt-mkv-style-expected.txt: Added. * media/track/in-band/track-in-band-srt-mkv-style.html: Added. * media/track/in-band/track-in-band-srt-mkv-track-order-expected.txt: Added. * media/track/in-band/track-in-band-srt-mkv-track-order.html: Added. * platform/mac/TestExpectations: Skip MKV and OGG tests. Canonical link: https://commits.webkit.org/138536@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@154908 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2013-08-30 20:17:27 +00:00
consoleWrite("<br><i>** Setting track 1 to showing</i>");
[GStreamer] media/track/in-band/ layout tests introduced in r154908 are failing https://bugs.webkit.org/show_bug.cgi?id=120665 Reviewed by Philippe Normand. Source/WebCore: Solved a deadlock when processing samples from a text track while the main thread is starting to play. In those circumstances, the text sink is set by Playbin as non-async, so the main thread calling MediaPlayerPrivateGStreamer::play() is blocked waiting for the preroll to finish. While prerolling, a text sample is detected in a non-main thread and sent to the main thread for processing. The current code waits in the non-main thread for the processing to finish, but that won't ever happen because the main thread is busy in play() waiting for the preroll to complete. Since the non-main thread doesn't need any result from the main thread, a simple change to avoid the wait is enough to solve the deadlock. There was also a bug preventing all the languages of the text tracks being detected on time. Sometimes the tag event specifying the track language emitted by the demuxer arrives too late to the webkittextcombiner sink pad (up to several seconds since sent!) and the language remains unset when the test checks for it. As a result, the test fails because the second track isn't reported to be in French language ("fr"). The solution to this is to find an earlier (more upstream) pad and listen there for the (sticky) tag events. There can be multiple tag events, so all of them are checked until one having the GST_TAG_LANGUAGE_CODE tag is found. Covered by existing tests (now unskipped). * platform/graphics/gstreamer/TextSinkGStreamer.cpp: (webkitTextSinkHandleSample): Don't wait for handleTextSample() to be completed in the main thread. This avoids the deadlock. * platform/graphics/gstreamer/TrackPrivateBaseGStreamer.cpp: (WebCore::findBestUpstreamPad): Find a pad more upstream than the current one. (WebCore::TrackPrivateBaseGStreamer::TrackPrivateBaseGStreamer): Call setPad() instead of setting it by hand here. (WebCore::TrackPrivateBaseGStreamer::setPad): Also manage m_bestUpstreamPad in parallel to m_pad. Add a probe to listen to the tag events instead of checking the "tags" property (the property wasn't present in the best upstream pad). (WebCore::TrackPrivateBaseGStreamer::disconnect): Remove the probe from m_bestUpstreamPad, instead of removing the old notify::tags listener from m_pad. (WebCore::TrackPrivateBaseGStreamer::tagsChanged): Check the sticky tag events until the language one is found (or the last event if no language event is available). * platform/graphics/gstreamer/TrackPrivateBaseGStreamer.h: Added m_bestUpstreamPad and m_eventProbe. Moved setPad() implementation to the cpp file. Tools: Fix media/track/in-band/tests Add libkate as dependency, so gst-plugins-bad detects it and can build the GstKateDec element. This isn't actually needed when using Flatpak (the dependency has been added there), but is still useful when using jhbuild and is also coherent with the need of other multimedia dependencies such as libvpx. * gtk/install-dependencies: * wpe/install-dependencies: LayoutTests: Removed track-in-band-mpegts-metadata.html because it relies on private streams on mpegts and those won't ever be exposed by tsdemux. Made testCuesAddedOnce() test more robust when finding text tracks. * media/content/user-private-streams.ts: Removed. * media/in-band-tracks.js: (pollProgress): Make sure that the function repeated at intervals is properly stopped in all the cases. (canplaythrough): More robust track detection. (testCuesAddedOnce): Ditto. * media/track/in-band/track-in-band-mpegts-metadata.html: Removed. * platform/glib/TestExpectations: Unskip media/track/in-band tests. Canonical link: https://commits.webkit.org/237022@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@276586 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-04-26 11:59:51 +00:00
inbandTrack1 = null;
[GStreamer] Support "chapter" text tracks https://bugs.webkit.org/show_bug.cgi?id=122000 Patch by Brendan Long <b.long@cablelabs.com> on 2013-10-01 Reviewed by Eric Carlson. Source/WebCore: Test: media/track/in-band/track-in-band-mkv-chapters.html * platform/graphics/gstreamer/InbandMetadataTextTrackPrivateGStreamer.h: New class, needed for cues that aren't associated with GstPads. * platform/graphics/gstreamer/MediaPlayerPrivateGStreamer.cpp: (WebCore::MediaPlayerPrivateGStreamer::handleMessage): Check for table of contents messages. (WebCore::MediaPlayerPrivateGStreamer::processTableOfContents): Parse table of contents messages into text tracks. (WebCore::MediaPlayerPrivateGStreamer::processTableOfContentsEntry): Recursive method for parsing all of the TOC entries. * platform/graphics/gstreamer/MediaPlayerPrivateGStreamer.h: Add the processTableOfContents functions. LayoutTests: * media/content/counting-subtitled-srt.mkv: Added chapters metadata. * media/track/in-band/track-in-band-mkv-chapters-expected.txt: New test, similar to cues-added-one, but uses "chapters" track. * media/track/in-band/track-in-band-mkv-chapters.html: Same. * media/in-band-cues.js: Updated tests to handle the new "chapters" track. * media/track/in-band/track-in-band-kate-ogg-cues-added-once-expected.txt: Same. * media/track/in-band/track-in-band-kate-ogg-cues-added-once.html: Same. * media/track/in-band/track-in-band-kate-ogg-mode-expected.txt: Same. * media/track/in-band/track-in-band-kate-ogg-mode.html: Same. * media/track/in-band/track-in-band-kate-ogg-style-expected.txt: Same. * media/track/in-band/track-in-band-kate-ogg-track-order-expected.txt: Same. * media/track/in-band/track-in-band-kate-ogg-track-order.html: Same. * media/track/in-band/track-in-band-srt-mkv-cues-added-once-expected.txt: Same. * media/track/in-band/track-in-band-srt-mkv-cues-added-once.html: Same. * media/track/in-band/track-in-band-srt-mkv-kind-expected.txt: Same. * media/track/in-band/track-in-band-srt-mkv-kind.html: Same. * media/track/in-band/track-in-band-srt-mkv-language-expected.txt: Same. * media/track/in-band/track-in-band-srt-mkv-language.html: Same. * media/track/in-band/track-in-band-srt-mkv-mode-expected.txt: Same. * media/track/in-band/track-in-band-srt-mkv-mode.html: Same. * media/track/in-band/track-in-band-srt-mkv-style-expected.txt: Same. * media/track/in-band/track-in-band-srt-mkv-track-order-expected.txt: Same. * media/track/in-band/track-in-band-srt-mkv-track-order.html: Same. Canonical link: https://commits.webkit.org/140209@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@156713 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2013-10-01 16:26:52 +00:00
for (var i = 0; i < video.textTracks.length; ++i) {
if (video.textTracks[i].kind == kind) {
inbandTrack1 = video.textTracks[i];
break;
}
}
[GStreamer] media/track/in-band/ layout tests introduced in r154908 are failing https://bugs.webkit.org/show_bug.cgi?id=120665 Reviewed by Philippe Normand. Source/WebCore: Solved a deadlock when processing samples from a text track while the main thread is starting to play. In those circumstances, the text sink is set by Playbin as non-async, so the main thread calling MediaPlayerPrivateGStreamer::play() is blocked waiting for the preroll to finish. While prerolling, a text sample is detected in a non-main thread and sent to the main thread for processing. The current code waits in the non-main thread for the processing to finish, but that won't ever happen because the main thread is busy in play() waiting for the preroll to complete. Since the non-main thread doesn't need any result from the main thread, a simple change to avoid the wait is enough to solve the deadlock. There was also a bug preventing all the languages of the text tracks being detected on time. Sometimes the tag event specifying the track language emitted by the demuxer arrives too late to the webkittextcombiner sink pad (up to several seconds since sent!) and the language remains unset when the test checks for it. As a result, the test fails because the second track isn't reported to be in French language ("fr"). The solution to this is to find an earlier (more upstream) pad and listen there for the (sticky) tag events. There can be multiple tag events, so all of them are checked until one having the GST_TAG_LANGUAGE_CODE tag is found. Covered by existing tests (now unskipped). * platform/graphics/gstreamer/TextSinkGStreamer.cpp: (webkitTextSinkHandleSample): Don't wait for handleTextSample() to be completed in the main thread. This avoids the deadlock. * platform/graphics/gstreamer/TrackPrivateBaseGStreamer.cpp: (WebCore::findBestUpstreamPad): Find a pad more upstream than the current one. (WebCore::TrackPrivateBaseGStreamer::TrackPrivateBaseGStreamer): Call setPad() instead of setting it by hand here. (WebCore::TrackPrivateBaseGStreamer::setPad): Also manage m_bestUpstreamPad in parallel to m_pad. Add a probe to listen to the tag events instead of checking the "tags" property (the property wasn't present in the best upstream pad). (WebCore::TrackPrivateBaseGStreamer::disconnect): Remove the probe from m_bestUpstreamPad, instead of removing the old notify::tags listener from m_pad. (WebCore::TrackPrivateBaseGStreamer::tagsChanged): Check the sticky tag events until the language one is found (or the last event if no language event is available). * platform/graphics/gstreamer/TrackPrivateBaseGStreamer.h: Added m_bestUpstreamPad and m_eventProbe. Moved setPad() implementation to the cpp file. Tools: Fix media/track/in-band/tests Add libkate as dependency, so gst-plugins-bad detects it and can build the GstKateDec element. This isn't actually needed when using Flatpak (the dependency has been added there), but is still useful when using jhbuild and is also coherent with the need of other multimedia dependencies such as libvpx. * gtk/install-dependencies: * wpe/install-dependencies: LayoutTests: Removed track-in-band-mpegts-metadata.html because it relies on private streams on mpegts and those won't ever be exposed by tsdemux. Made testCuesAddedOnce() test more robust when finding text tracks. * media/content/user-private-streams.ts: Removed. * media/in-band-tracks.js: (pollProgress): Make sure that the function repeated at intervals is properly stopped in all the cases. (canplaythrough): More robust track detection. (testCuesAddedOnce): Ditto. * media/track/in-band/track-in-band-mpegts-metadata.html: Removed. * platform/glib/TestExpectations: Unskip media/track/in-band tests. Canonical link: https://commits.webkit.org/237022@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@276586 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-04-26 11:59:51 +00:00
if (!inbandTrack1)
failTest("No text track of kind '" + kind + "'");
Source/WebCore: [GStreamer] support in-band text tracks https://bugs.webkit.org/show_bug.cgi?id=103771 Patch by Brendan Long <b.long@cablelabs.com> on 2013-08-30 Reviewed by Eric Carlson. Tests: New tests added because existing tests were too specific. media/track/in-band/track-in-band-kate-ogg-cues-added-once.html media/track/in-band/track-in-band-kate-ogg-kind.html media/track/in-band/track-in-band-kate-ogg-language.html media/track/in-band/track-in-band-kate-ogg-mode.html media/track/in-band/track-in-band-kate-ogg-style.html media/track/in-band/track-in-band-kate-ogg-track-order.html media/track/in-band/track-in-band-srt-mkv-cues-added-once.html media/track/in-band/track-in-band-srt-mkv-kind.html media/track/in-band/track-in-band-srt-mkv-language.html media/track/in-band/track-in-band-srt-mkv-mode.html media/track/in-band/track-in-band-srt-mkv-style.html media/track/in-band/track-in-band-srt-mkv-track-order.html * CMakeLists.txt: Add InbandTextTrackPrivateGStreamer, InbandGenericTextTrack, InbandWebVTTTextTrack, and TextCombinerGStreamer files. * GNUmakefile.list.am: Same. * PlatformEfl.cmake: Same. * Target.pri: Same. * WebCore.vcxproj/WebCore.vcxproj: Same. * WebCore.vcxproj/WebCore.vcxproj.filters: Same. * WebCore.xcodeproj/project.pbxproj: Same. * html/track/InbandGenericTextTrack.cpp: Split out code for handling generic cues. (WebCore::GenericTextTrackCueMap::GenericTextTrackCueMap): Move from InbandTextTrack. (WebCore::GenericTextTrackCueMap::~GenericTextTrackCueMap): Same. (WebCore::GenericTextTrackCueMap::add): Same. (WebCore::GenericTextTrackCueMap::find): Same. (WebCore::GenericTextTrackCueMap::remove): Same. (WebCore::InbandGenericTextTrack::create): Same. (WebCore::InbandGenericTextTrack::updateCueFromCueData): Same. (WebCore::InbandGenericTextTrack::addGenericCue): Same. (WebCore::InbandGenericTextTrack::updateGenericCue): Same. (WebCore::InbandGenericTextTrack::removeGenericCue): Same. (WebCore::InbandGenericTextTrack::removeCue): Same. (WebCore::InbandGenericTextTrack::InbandGenericTextTrack): Empty. (WebCore::InbandGenericTextTrack::~InbandGenericTextTrack): Empty. * html/track/InbandGenericTextTrack.h: Copied from Source/WebCore/html/track/InbandTextTrack.h. The only addition is the ASSERT_NOT_REACHED() for WebVTT cues. * html/track/InbandTextTrack.cpp: Add label and language changed callbacks. (WebCore::InbandTextTrack::create): Return a generic or WebVTT text track based on the private CueFormat. (WebCore::InbandTextTrack::labelChanged): Added. (WebCore::InbandTextTrack::languageChanged): Added. * html/track/InbandTextTrack.h: Add label and language changed callbacks. * html/track/InbandWebVTTTextTrack.cpp: Added, based on InbandTextTrack. (WebCore::InbandWebVTTTextTrack::create): Same. (WebCore::InbandWebVTTTextTrack::InbandWebVTTTextTrack): Empty. (WebCore::InbandWebVTTTextTrack::~InbandWebVTTTextTrack): Empty. (WebCore::InbandWebVTTTextTrack::parseWebVTTCueData): Sends data to a WebVTTParser. (WebCore::InbandWebVTTTextTrack::newCuesParsed): Adds cues when WebVTTParser parses them. (WebCore::InbandWebVTTTextTrack::fileFailedToParse): Prints a warning when WebVTTParser has a problem. * html/track/InbandWebVTTTextTrack.h: Added. * platform/graphics/InbandTextTrackPrivate.h: (WebCore::InbandTextTrackPrivate::cueFormat): For determining if the track will have generic or WebVTT cues. (WebCore::InbandTextTrackPrivate::InbandTextTrackPrivate): Pass CueFormat in the constructor. * platform/graphics/InbandTextTrackPrivateClient.h: Same. * platform/graphics/avfoundation/InbandTextTrackPrivateAVF.cpp: (WebCore::InbandTextTrackPrivateAVF::InbandTextTrackPrivateAVF): Pass CueFormat (Generic) to InbandTextTrackPrivate. * platform/graphics/gstreamer/GRefPtrGStreamer.cpp: Add GRefPtr specializations for GstSample and GstEvent. * platform/graphics/gstreamer/GRefPtrGStreamer.h: Same. * platform/graphics/gstreamer/GStreamerUtilities.h: Add WARN_MEDIA_MESSAGE. * platform/graphics/gstreamer/GStreamerVersioning.h: Add a function to check GStreamer version at runtime. * platform/graphics/gstreamer/InbandTextTrackPrivateGStreamer.cpp: Added. (WebCore::textTrackPrivateEventCallback): Watches for tag and stream start events. (WebCore::textTrackPrivateSampleTimeoutCallback): See notifyTrackOfSample(); (WebCore::textTrackPrivateStreamTimeoutCallback): See notifyTrackOfStreamChanged(); (WebCore::textTrackPrivateTagsChangeTimeoutCallback): See notifyTrackOfTagsChanged(); (WebCore::InbandTextTrackPrivateGStreamer::InbandTextTrackPrivateGStreamer): Initializes tags and stream and sets up event callback. (WebCore::InbandTextTrackPrivateGStreamer::~InbandTextTrackPrivateGStreamer): Calls disconnect: (WebCore::InbandTextTrackPrivateGStreamer::disconnect): Removes signal handlers and frees some memory. (WebCore::InbandTextTrackPrivateGStreamer::handleSample): Adds samples to a list and sets up callback. (WebCore::InbandTextTrackPrivateGStreamer::streamChanged): Sets up callback. (WebCore::InbandTextTrackPrivateGStreamer::tagsChanged): Same. (WebCore::InbandTextTrackPrivateGStreamer::notifyTrackOfSample): Parses all queued samples with WebVTTParser. (WebCore::InbandTextTrackPrivateGStreamer::notifyTrackOfStreamChanged): Keeps track of current stream. (WebCore::InbandTextTrackPrivateGStreamer::notifyTrackOfTagsChanged): Sets label and language from tags. * platform/graphics/gstreamer/InbandTextTrackPrivateGStreamer.h: Added. (WebCore::InbandTextTrackPrivateGStreamer::create): Basic RefPtr create function. (WebCore::InbandTextTrackPrivateGStreamer::pad): Returns the pad this track is associated with (used to determine if a playbin text stream has already been associated with a text track). (WebCore::InbandTextTrackPrivateGStreamer::setIndex): Sets the track index (used for sorting). (WebCore::InbandTextTrackPrivateGStreamer::streamId): Returns the stream ID (used to handle new samples). * platform/graphics/gstreamer/MediaPlayerPrivateGStreamer.cpp: (WebCore::mediaPlayerPrivateTextChangedCallback): Called for playbin "text-changed" event. See textChanged(). (WebCore::mediaPlayerPrivateTextChangeTimeoutCallback): See notifyPlayerOfText(). (WebCore::mediaPlayerPrivateNewTextSampleCallback): See newTextSample(). (WebCore::MediaPlayerPrivateGStreamer::MediaPlayerPrivateGStreamer): Initialize m_textTimerHandler. (WebCore::MediaPlayerPrivateGStreamer::~MediaPlayerPrivateGStreamer): Disconnect text tracks (they don't necessarily get freed here, since a script could hold a reference). (WebCore::MediaPlayerPrivateGStreamer::textChanged): Setup callback for notifyPlayerOfText. (WebCore::MediaPlayerPrivateGStreamer::notifyPlayerOfText): Create text tracks. (WebCore::MediaPlayerPrivateGStreamer::newTextSample): Handle new samples by giving them to a text track with a matching stream. This method is syncryonous because we need to get the stream start sticky event immediately. * platform/graphics/gstreamer/MediaPlayerPrivateGStreamer.h: * platform/graphics/gstreamer/TextCombinerGStreamer.cpp: Added. This element forwards buffers from all of its input pads, but also converts plain text to WebVTT as needed. (webkit_text_combiner_init): Setup internal funnel. (webkitTextCombinerPadEvent): If the caps are plain text make sure we have a webvttenv, otherwise connect directly to the funnel. (webkitTextCombinerRequestNewPad): Setup ghostpad and event callback. (webkitTextCombinerReleasePad): Release pad and optional associated webvttenc. (webkit_text_combiner_class_init): Setup pad templates and request/release pad functions. (webkitTextCombinerNew): Returns a new WebKitTextCombiner. * platform/graphics/gstreamer/TextCombinerGStreamer.h: Added. * platform/graphics/gstreamer/TextSinkGStreamer.cpp: Added. (webkit_text_sink_init): Set sync=false. (webkitTextSinkGetProperty): Ignore sync property. (webkitTextSinkSetProperty): Same. (webkitTextSinkQuery): Ignore position and duration queries, forward everything else to appsink. (webkit_text_sink_class_init): Setup property and query functions. (webkitTextSinkNew): Return a new WebKitTextSink. * platform/graphics/gstreamer/TextSinkGStreamer.h: Added. LayoutTests: <https://webkit.org/b/103771> [GStreamer] support in-band text tracks Patch by Brendan Long <b.long@cablelabs.com> on 2013-08-30 Reviewed by Eric Carlson. * media/content/counting-subtitled-kate.ogv: Added. * media/content/counting-subtitled-srt.mkv: Added. * media/in-band-cues.js: Added. * media/track/in-band/track-in-band-kate-ogg-cues-added-once-expected.txt: Added. * media/track/in-band/track-in-band-kate-ogg-cues-added-once.html: Added. * media/track/in-band/track-in-band-kate-ogg-kind-expected.txt: Added. * media/track/in-band/track-in-band-kate-ogg-kind.html: Added. * media/track/in-band/track-in-band-kate-ogg-language-expected.txt: Added. * media/track/in-band/track-in-band-kate-ogg-language.html: Added. * media/track/in-band/track-in-band-kate-ogg-mode-expected.txt: Added. * media/track/in-band/track-in-band-kate-ogg-mode.html: Added. * media/track/in-band/track-in-band-kate-ogg-style-expected.txt: Added. * media/track/in-band/track-in-band-kate-ogg-style.html: Added. * media/track/in-band/track-in-band-kate-ogg-track-order-expected.txt: Added. * media/track/in-band/track-in-band-kate-ogg-track-order.html: Added. * media/track/in-band/track-in-band-srt-mkv-cues-added-once-expected.txt: Added. * media/track/in-band/track-in-band-srt-mkv-cues-added-once.html: Added. * media/track/in-band/track-in-band-srt-mkv-kind-expected.txt: Added. * media/track/in-band/track-in-band-srt-mkv-kind.html: Added. * media/track/in-band/track-in-band-srt-mkv-language-expected.txt: Added. * media/track/in-band/track-in-band-srt-mkv-language.html: Added. * media/track/in-band/track-in-band-srt-mkv-mode-expected.txt: Added. * media/track/in-band/track-in-band-srt-mkv-mode.html: Added. * media/track/in-band/track-in-band-srt-mkv-style-expected.txt: Added. * media/track/in-band/track-in-band-srt-mkv-style.html: Added. * media/track/in-band/track-in-band-srt-mkv-track-order-expected.txt: Added. * media/track/in-band/track-in-band-srt-mkv-track-order.html: Added. * platform/mac/TestExpectations: Skip MKV and OGG tests. Canonical link: https://commits.webkit.org/138536@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@154908 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2013-08-30 20:17:27 +00:00
run("inbandTrack1.mode = 'showing'");
run("video.play()");
}
findMediaElement();
video.src = uri;
waitForEvent('canplaythrough', canplaythrough);
}
[GStreamer] Support audio and video tracks https://bugs.webkit.org/show_bug.cgi?id=117039 Patch by Brendan Long <b.long@cablelabs.com> on 2013-11-01 Reviewed by Philippe Normand. Source/WebCore: Tests: media/track/audio/audio-track-mkv-vorbis-addtrack.html media/track/audio/audio-track-mkv-vorbis-enabled.html media/track/audio/audio-track-mkv-vorbis-language.html media/track/in-band/track-in-band-kate-ogg-addtrack.html media/track/in-band/track-in-band-srt-mkv-addtrack.html media/track/video/video-track-mkv-theora-addtrack.html media/track/video/video-track-mkv-theora-language.html media/track/video/video-track-mkv-theora-selected.html * GNUmakefile.list.am: Add audio and video track files. * PlatformEfl.cmake: Same. * html/HTMLMediaElement.cpp: (WebCore::HTMLMediaElement::audioTrackEnabledChanged): Schedule "change" event. (WebCore::HTMLMediaElement::videoTrackSelectedChanged): Same. * html/track/AudioTrack.cpp: (WebCore::AudioTrack::setEnabled): Call m_private->setEnabled (WebCore::AudioTrack::enabledChanged): Added callback. (WebCore::AudioTrack::labelChanged): Same. (WebCore::AudioTrack::languageChanged): Same. (WebCore::AudioTrack::willRemoveAudioTrackPrivate): Use ASSERT_UNUSED for consistence. * html/track/AudioTrack.h: Add new enabled, label and language callbacks. * html/track/VideoTrack.cpp: (WebCore::VideoTrack::setSelected): Call m_private->setEnabled (WebCore::VideoTrack::selectedChanged): Added callback. (WebCore::VideoTrack::labelChanged): Same. (WebCore::VideoTrack::languageChanged): Same. (WebCore::VideoTrack::willRemoveVideoTrackPrivate): Use ASSERT_UNUSED for consistence. * html/track/VideoTrack.h: Add new selected, label and language callbacks. * platform/graphics/AudioTrackPrivate.h: (WebCore::AudioTrackPrivate::setEnabled): Call m_client->enabledChanged * platform/graphics/VideoTrackPrivate.h: (WebCore::VideoTrackPrivate::setSelected): Call m_client->selectedChanged * platform/graphics/gstreamer/InbandTextTrackPrivateGStreamer.cpp: (WebCore::InbandTextTrackPrivateGStreamer::InbandTextTrackPrivateGStreamer): Add ASSERT(m_pad) (WebCore::InbandTextTrackPrivateGStreamer::notifyTrackOfTagsChanged): Look at all tag events instead of just the first one. * platform/graphics/gstreamer/MediaPlayerPrivateGStreamer.cpp: (WebCore::mediaPlayerPrivateVideoSinkCapsChangedCallback): Separated callback when caps change from callback when number of videos change. (WebCore::mediaPlayerPrivateVideoCapsChangeTimeoutCallback): Same. (WebCore::MediaPlayerPrivateGStreamer::MediaPlayerPrivateGStreamer): Initialize m_videoCapsTimerHandler. (WebCore::MediaPlayerPrivateGStreamer::~MediaPlayerPrivateGStreamer): Disconnect audio and video tracks and remove callbacks. (WebCore::MediaPlayerPrivateGStreamer::videoCapsChanged): Separated callback when caps change from callback when number of videos change. (WebCore::MediaPlayerPrivateGStreamer::notifyPlayerOfVideo): Create video tracks for each stream. (WebCore::MediaPlayerPrivateGStreamer::notifyPlayerOfVideoCaps): Separated callback when caps change from callback when number of videos change. (WebCore::MediaPlayerPrivateGStreamer::notifyPlayerOfAudio): Create audio tracks for each stream. * platform/graphics/gstreamer/MediaPlayerPrivateGStreamer.h: Add audio and video tracks, and a pointer to keep track of the default audio stream. * platform/graphics/gstreamer/AudioTrackPrivateGStreamer.cpp: Added. * platform/graphics/gstreamer/AudioTrackPrivateGStreamer.h: Added. * platform/graphics/gstreamer/GRefPtrGStreamer.cpp: Add GRefPtr<GstTagList> * platform/graphics/gstreamer/TrackPrivateBaseGStreamer.cpp: Added, handles tags and "active" property of audio and video tracks. * platform/graphics/gstreamer/TrackPrivateBaseGStreamer.h: Added. * platform/graphics/gstreamer/VideoTrackPrivateGStreamer.cpp: Added. * platform/graphics/gstreamer/VideoTrackPrivateGStreamer.h: Added. LayoutTests: * media/content/two-audio-and-video-tracks.mkv: Added. * media/in-band-tracks.js: Renamed from LayoutTests/media/in-band-cues.js. (testAddTrack): Tests addtrack events. (testAttribute): Added "type" parameter so we can use this for audio and video tracks. (testTextTrackMode): Renamed to make it clear that this only applies to text tracks. (testCueStyle): Same. (testTextTrackOrder): Same. * media/track/audio/audio-track-mkv-vorbis-addtrack-expected.txt: Added. * media/track/audio/audio-track-mkv-vorbis-addtrack.html: Copied from LayoutTests/media/track/in-band/track-in-band-srt-mkv-track-order.html. * media/track/audio/audio-track-mkv-vorbis-enabled-expected.txt: Added. * media/track/audio/audio-track-mkv-vorbis-enabled.html: Added. * media/track/audio/audio-track-mkv-vorbis-language-expected.txt: Added. * media/track/audio/audio-track-mkv-vorbis-language.html: Copied from LayoutTests/media/track/in-band/track-in-band-kate-ogg-track-order.html. * media/track/in-band/track-in-band-kate-ogg-addtrack-expected.txt: Added. * media/track/in-band/track-in-band-kate-ogg-addtrack.html: Copied from LayoutTests/media/track/in-band/track-in-band-srt-mkv-track-order.html. * media/track/in-band/track-in-band-kate-ogg-cues-added-once.html: Added. * media/track/in-band/track-in-band-kate-ogg-kind.html: Added. * media/track/in-band/track-in-band-kate-ogg-language.html: Added. * media/track/in-band/track-in-band-kate-ogg-mode.html: Added. * media/track/in-band/track-in-band-kate-ogg-style.html: Added. * media/track/in-band/track-in-band-kate-ogg-track-order.html: Added. * media/track/in-band/track-in-band-srt-mkv-addtrack-expected.txt: Added. * media/track/in-band/track-in-band-srt-mkv-addtrack.html: Copied from LayoutTests/media/track/in-band/track-in-band-srt-mkv-track-order.html. * media/track/in-band/track-in-band-srt-mkv-cues-added-once.html: Added. * media/track/in-band/track-in-band-srt-mkv-kind.html: Added. * media/track/in-band/track-in-band-srt-mkv-language.html: Added. * media/track/in-band/track-in-band-srt-mkv-mode.html: Added. * media/track/in-band/track-in-band-srt-mkv-style.html: Added. * media/track/in-band/track-in-band-srt-mkv-track-order.html: Added. * media/track/video/video-track-mkv-theora-addtrack-expected.txt: Added. * media/track/video/video-track-mkv-theora-addtrack.html: Copied from LayoutTests/media/track/in-band/track-in-band-srt-mkv-track-order.html. * media/track/video/video-track-mkv-theora-language-expected.txt: Added. * media/track/video/video-track-mkv-theora-language.html: Copied from LayoutTests/media/track/in-band/track-in-band-kate-ogg-track-order.html. * media/track/video/video-track-mkv-theora-selected-expected.txt: Added. * media/track/video/video-track-mkv-theora-selected.html: Added. * platform/mac/TestExpectations: Ignore audio and video track tests on mac. Canonical link: https://commits.webkit.org/141792@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@158436 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2013-11-01 16:30:03 +00:00
function testTextTrackOrder(uri, numInBandTracks)
{
Source/WebCore: [GStreamer] support in-band text tracks https://bugs.webkit.org/show_bug.cgi?id=103771 Patch by Brendan Long <b.long@cablelabs.com> on 2013-08-30 Reviewed by Eric Carlson. Tests: New tests added because existing tests were too specific. media/track/in-band/track-in-band-kate-ogg-cues-added-once.html media/track/in-band/track-in-band-kate-ogg-kind.html media/track/in-band/track-in-band-kate-ogg-language.html media/track/in-band/track-in-band-kate-ogg-mode.html media/track/in-band/track-in-band-kate-ogg-style.html media/track/in-band/track-in-band-kate-ogg-track-order.html media/track/in-band/track-in-band-srt-mkv-cues-added-once.html media/track/in-band/track-in-band-srt-mkv-kind.html media/track/in-band/track-in-band-srt-mkv-language.html media/track/in-band/track-in-band-srt-mkv-mode.html media/track/in-band/track-in-band-srt-mkv-style.html media/track/in-band/track-in-band-srt-mkv-track-order.html * CMakeLists.txt: Add InbandTextTrackPrivateGStreamer, InbandGenericTextTrack, InbandWebVTTTextTrack, and TextCombinerGStreamer files. * GNUmakefile.list.am: Same. * PlatformEfl.cmake: Same. * Target.pri: Same. * WebCore.vcxproj/WebCore.vcxproj: Same. * WebCore.vcxproj/WebCore.vcxproj.filters: Same. * WebCore.xcodeproj/project.pbxproj: Same. * html/track/InbandGenericTextTrack.cpp: Split out code for handling generic cues. (WebCore::GenericTextTrackCueMap::GenericTextTrackCueMap): Move from InbandTextTrack. (WebCore::GenericTextTrackCueMap::~GenericTextTrackCueMap): Same. (WebCore::GenericTextTrackCueMap::add): Same. (WebCore::GenericTextTrackCueMap::find): Same. (WebCore::GenericTextTrackCueMap::remove): Same. (WebCore::InbandGenericTextTrack::create): Same. (WebCore::InbandGenericTextTrack::updateCueFromCueData): Same. (WebCore::InbandGenericTextTrack::addGenericCue): Same. (WebCore::InbandGenericTextTrack::updateGenericCue): Same. (WebCore::InbandGenericTextTrack::removeGenericCue): Same. (WebCore::InbandGenericTextTrack::removeCue): Same. (WebCore::InbandGenericTextTrack::InbandGenericTextTrack): Empty. (WebCore::InbandGenericTextTrack::~InbandGenericTextTrack): Empty. * html/track/InbandGenericTextTrack.h: Copied from Source/WebCore/html/track/InbandTextTrack.h. The only addition is the ASSERT_NOT_REACHED() for WebVTT cues. * html/track/InbandTextTrack.cpp: Add label and language changed callbacks. (WebCore::InbandTextTrack::create): Return a generic or WebVTT text track based on the private CueFormat. (WebCore::InbandTextTrack::labelChanged): Added. (WebCore::InbandTextTrack::languageChanged): Added. * html/track/InbandTextTrack.h: Add label and language changed callbacks. * html/track/InbandWebVTTTextTrack.cpp: Added, based on InbandTextTrack. (WebCore::InbandWebVTTTextTrack::create): Same. (WebCore::InbandWebVTTTextTrack::InbandWebVTTTextTrack): Empty. (WebCore::InbandWebVTTTextTrack::~InbandWebVTTTextTrack): Empty. (WebCore::InbandWebVTTTextTrack::parseWebVTTCueData): Sends data to a WebVTTParser. (WebCore::InbandWebVTTTextTrack::newCuesParsed): Adds cues when WebVTTParser parses them. (WebCore::InbandWebVTTTextTrack::fileFailedToParse): Prints a warning when WebVTTParser has a problem. * html/track/InbandWebVTTTextTrack.h: Added. * platform/graphics/InbandTextTrackPrivate.h: (WebCore::InbandTextTrackPrivate::cueFormat): For determining if the track will have generic or WebVTT cues. (WebCore::InbandTextTrackPrivate::InbandTextTrackPrivate): Pass CueFormat in the constructor. * platform/graphics/InbandTextTrackPrivateClient.h: Same. * platform/graphics/avfoundation/InbandTextTrackPrivateAVF.cpp: (WebCore::InbandTextTrackPrivateAVF::InbandTextTrackPrivateAVF): Pass CueFormat (Generic) to InbandTextTrackPrivate. * platform/graphics/gstreamer/GRefPtrGStreamer.cpp: Add GRefPtr specializations for GstSample and GstEvent. * platform/graphics/gstreamer/GRefPtrGStreamer.h: Same. * platform/graphics/gstreamer/GStreamerUtilities.h: Add WARN_MEDIA_MESSAGE. * platform/graphics/gstreamer/GStreamerVersioning.h: Add a function to check GStreamer version at runtime. * platform/graphics/gstreamer/InbandTextTrackPrivateGStreamer.cpp: Added. (WebCore::textTrackPrivateEventCallback): Watches for tag and stream start events. (WebCore::textTrackPrivateSampleTimeoutCallback): See notifyTrackOfSample(); (WebCore::textTrackPrivateStreamTimeoutCallback): See notifyTrackOfStreamChanged(); (WebCore::textTrackPrivateTagsChangeTimeoutCallback): See notifyTrackOfTagsChanged(); (WebCore::InbandTextTrackPrivateGStreamer::InbandTextTrackPrivateGStreamer): Initializes tags and stream and sets up event callback. (WebCore::InbandTextTrackPrivateGStreamer::~InbandTextTrackPrivateGStreamer): Calls disconnect: (WebCore::InbandTextTrackPrivateGStreamer::disconnect): Removes signal handlers and frees some memory. (WebCore::InbandTextTrackPrivateGStreamer::handleSample): Adds samples to a list and sets up callback. (WebCore::InbandTextTrackPrivateGStreamer::streamChanged): Sets up callback. (WebCore::InbandTextTrackPrivateGStreamer::tagsChanged): Same. (WebCore::InbandTextTrackPrivateGStreamer::notifyTrackOfSample): Parses all queued samples with WebVTTParser. (WebCore::InbandTextTrackPrivateGStreamer::notifyTrackOfStreamChanged): Keeps track of current stream. (WebCore::InbandTextTrackPrivateGStreamer::notifyTrackOfTagsChanged): Sets label and language from tags. * platform/graphics/gstreamer/InbandTextTrackPrivateGStreamer.h: Added. (WebCore::InbandTextTrackPrivateGStreamer::create): Basic RefPtr create function. (WebCore::InbandTextTrackPrivateGStreamer::pad): Returns the pad this track is associated with (used to determine if a playbin text stream has already been associated with a text track). (WebCore::InbandTextTrackPrivateGStreamer::setIndex): Sets the track index (used for sorting). (WebCore::InbandTextTrackPrivateGStreamer::streamId): Returns the stream ID (used to handle new samples). * platform/graphics/gstreamer/MediaPlayerPrivateGStreamer.cpp: (WebCore::mediaPlayerPrivateTextChangedCallback): Called for playbin "text-changed" event. See textChanged(). (WebCore::mediaPlayerPrivateTextChangeTimeoutCallback): See notifyPlayerOfText(). (WebCore::mediaPlayerPrivateNewTextSampleCallback): See newTextSample(). (WebCore::MediaPlayerPrivateGStreamer::MediaPlayerPrivateGStreamer): Initialize m_textTimerHandler. (WebCore::MediaPlayerPrivateGStreamer::~MediaPlayerPrivateGStreamer): Disconnect text tracks (they don't necessarily get freed here, since a script could hold a reference). (WebCore::MediaPlayerPrivateGStreamer::textChanged): Setup callback for notifyPlayerOfText. (WebCore::MediaPlayerPrivateGStreamer::notifyPlayerOfText): Create text tracks. (WebCore::MediaPlayerPrivateGStreamer::newTextSample): Handle new samples by giving them to a text track with a matching stream. This method is syncryonous because we need to get the stream start sticky event immediately. * platform/graphics/gstreamer/MediaPlayerPrivateGStreamer.h: * platform/graphics/gstreamer/TextCombinerGStreamer.cpp: Added. This element forwards buffers from all of its input pads, but also converts plain text to WebVTT as needed. (webkit_text_combiner_init): Setup internal funnel. (webkitTextCombinerPadEvent): If the caps are plain text make sure we have a webvttenv, otherwise connect directly to the funnel. (webkitTextCombinerRequestNewPad): Setup ghostpad and event callback. (webkitTextCombinerReleasePad): Release pad and optional associated webvttenc. (webkit_text_combiner_class_init): Setup pad templates and request/release pad functions. (webkitTextCombinerNew): Returns a new WebKitTextCombiner. * platform/graphics/gstreamer/TextCombinerGStreamer.h: Added. * platform/graphics/gstreamer/TextSinkGStreamer.cpp: Added. (webkit_text_sink_init): Set sync=false. (webkitTextSinkGetProperty): Ignore sync property. (webkitTextSinkSetProperty): Same. (webkitTextSinkQuery): Ignore position and duration queries, forward everything else to appsink. (webkit_text_sink_class_init): Setup property and query functions. (webkitTextSinkNew): Return a new WebKitTextSink. * platform/graphics/gstreamer/TextSinkGStreamer.h: Added. LayoutTests: <https://webkit.org/b/103771> [GStreamer] support in-band text tracks Patch by Brendan Long <b.long@cablelabs.com> on 2013-08-30 Reviewed by Eric Carlson. * media/content/counting-subtitled-kate.ogv: Added. * media/content/counting-subtitled-srt.mkv: Added. * media/in-band-cues.js: Added. * media/track/in-band/track-in-band-kate-ogg-cues-added-once-expected.txt: Added. * media/track/in-band/track-in-band-kate-ogg-cues-added-once.html: Added. * media/track/in-band/track-in-band-kate-ogg-kind-expected.txt: Added. * media/track/in-band/track-in-band-kate-ogg-kind.html: Added. * media/track/in-band/track-in-band-kate-ogg-language-expected.txt: Added. * media/track/in-band/track-in-band-kate-ogg-language.html: Added. * media/track/in-band/track-in-band-kate-ogg-mode-expected.txt: Added. * media/track/in-band/track-in-band-kate-ogg-mode.html: Added. * media/track/in-band/track-in-band-kate-ogg-style-expected.txt: Added. * media/track/in-band/track-in-band-kate-ogg-style.html: Added. * media/track/in-band/track-in-band-kate-ogg-track-order-expected.txt: Added. * media/track/in-band/track-in-band-kate-ogg-track-order.html: Added. * media/track/in-band/track-in-band-srt-mkv-cues-added-once-expected.txt: Added. * media/track/in-band/track-in-band-srt-mkv-cues-added-once.html: Added. * media/track/in-band/track-in-band-srt-mkv-kind-expected.txt: Added. * media/track/in-band/track-in-band-srt-mkv-kind.html: Added. * media/track/in-band/track-in-band-srt-mkv-language-expected.txt: Added. * media/track/in-band/track-in-band-srt-mkv-language.html: Added. * media/track/in-band/track-in-band-srt-mkv-mode-expected.txt: Added. * media/track/in-band/track-in-band-srt-mkv-mode.html: Added. * media/track/in-band/track-in-band-srt-mkv-style-expected.txt: Added. * media/track/in-band/track-in-band-srt-mkv-style.html: Added. * media/track/in-band/track-in-band-srt-mkv-track-order-expected.txt: Added. * media/track/in-band/track-in-band-srt-mkv-track-order.html: Added. * platform/mac/TestExpectations: Skip MKV and OGG tests. Canonical link: https://commits.webkit.org/138536@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@154908 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2013-08-30 20:17:27 +00:00
function compareTracks(track1, track2)
{
var equal = (eval(track1) == eval(track2));
reportExpected(equal, track1, "==", track2, track1);
}
function canplaythrough()
{
consoleWrite("<br><i>** Check initial in-band track states</i>");
[GStreamer] Support "chapter" text tracks https://bugs.webkit.org/show_bug.cgi?id=122000 Patch by Brendan Long <b.long@cablelabs.com> on 2013-10-01 Reviewed by Eric Carlson. Source/WebCore: Test: media/track/in-band/track-in-band-mkv-chapters.html * platform/graphics/gstreamer/InbandMetadataTextTrackPrivateGStreamer.h: New class, needed for cues that aren't associated with GstPads. * platform/graphics/gstreamer/MediaPlayerPrivateGStreamer.cpp: (WebCore::MediaPlayerPrivateGStreamer::handleMessage): Check for table of contents messages. (WebCore::MediaPlayerPrivateGStreamer::processTableOfContents): Parse table of contents messages into text tracks. (WebCore::MediaPlayerPrivateGStreamer::processTableOfContentsEntry): Recursive method for parsing all of the TOC entries. * platform/graphics/gstreamer/MediaPlayerPrivateGStreamer.h: Add the processTableOfContents functions. LayoutTests: * media/content/counting-subtitled-srt.mkv: Added chapters metadata. * media/track/in-band/track-in-band-mkv-chapters-expected.txt: New test, similar to cues-added-one, but uses "chapters" track. * media/track/in-band/track-in-band-mkv-chapters.html: Same. * media/in-band-cues.js: Updated tests to handle the new "chapters" track. * media/track/in-band/track-in-band-kate-ogg-cues-added-once-expected.txt: Same. * media/track/in-band/track-in-band-kate-ogg-cues-added-once.html: Same. * media/track/in-band/track-in-band-kate-ogg-mode-expected.txt: Same. * media/track/in-band/track-in-band-kate-ogg-mode.html: Same. * media/track/in-band/track-in-band-kate-ogg-style-expected.txt: Same. * media/track/in-band/track-in-band-kate-ogg-track-order-expected.txt: Same. * media/track/in-band/track-in-band-kate-ogg-track-order.html: Same. * media/track/in-band/track-in-band-srt-mkv-cues-added-once-expected.txt: Same. * media/track/in-band/track-in-band-srt-mkv-cues-added-once.html: Same. * media/track/in-band/track-in-band-srt-mkv-kind-expected.txt: Same. * media/track/in-band/track-in-band-srt-mkv-kind.html: Same. * media/track/in-band/track-in-band-srt-mkv-language-expected.txt: Same. * media/track/in-band/track-in-band-srt-mkv-language.html: Same. * media/track/in-band/track-in-band-srt-mkv-mode-expected.txt: Same. * media/track/in-band/track-in-band-srt-mkv-mode.html: Same. * media/track/in-band/track-in-band-srt-mkv-style-expected.txt: Same. * media/track/in-band/track-in-band-srt-mkv-track-order-expected.txt: Same. * media/track/in-band/track-in-band-srt-mkv-track-order.html: Same. Canonical link: https://commits.webkit.org/140209@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@156713 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2013-10-01 16:26:52 +00:00
testExpected("video.textTracks.length", numInBandTracks);
for (var i = 0; i < numInBandTracks; ++i)
run("inbandTrack" + (i + 1) + " = video.textTracks[" + i + "]");
Source/WebCore: [GStreamer] support in-band text tracks https://bugs.webkit.org/show_bug.cgi?id=103771 Patch by Brendan Long <b.long@cablelabs.com> on 2013-08-30 Reviewed by Eric Carlson. Tests: New tests added because existing tests were too specific. media/track/in-band/track-in-band-kate-ogg-cues-added-once.html media/track/in-band/track-in-band-kate-ogg-kind.html media/track/in-band/track-in-band-kate-ogg-language.html media/track/in-band/track-in-band-kate-ogg-mode.html media/track/in-band/track-in-band-kate-ogg-style.html media/track/in-band/track-in-band-kate-ogg-track-order.html media/track/in-band/track-in-band-srt-mkv-cues-added-once.html media/track/in-band/track-in-band-srt-mkv-kind.html media/track/in-band/track-in-band-srt-mkv-language.html media/track/in-band/track-in-band-srt-mkv-mode.html media/track/in-band/track-in-band-srt-mkv-style.html media/track/in-band/track-in-band-srt-mkv-track-order.html * CMakeLists.txt: Add InbandTextTrackPrivateGStreamer, InbandGenericTextTrack, InbandWebVTTTextTrack, and TextCombinerGStreamer files. * GNUmakefile.list.am: Same. * PlatformEfl.cmake: Same. * Target.pri: Same. * WebCore.vcxproj/WebCore.vcxproj: Same. * WebCore.vcxproj/WebCore.vcxproj.filters: Same. * WebCore.xcodeproj/project.pbxproj: Same. * html/track/InbandGenericTextTrack.cpp: Split out code for handling generic cues. (WebCore::GenericTextTrackCueMap::GenericTextTrackCueMap): Move from InbandTextTrack. (WebCore::GenericTextTrackCueMap::~GenericTextTrackCueMap): Same. (WebCore::GenericTextTrackCueMap::add): Same. (WebCore::GenericTextTrackCueMap::find): Same. (WebCore::GenericTextTrackCueMap::remove): Same. (WebCore::InbandGenericTextTrack::create): Same. (WebCore::InbandGenericTextTrack::updateCueFromCueData): Same. (WebCore::InbandGenericTextTrack::addGenericCue): Same. (WebCore::InbandGenericTextTrack::updateGenericCue): Same. (WebCore::InbandGenericTextTrack::removeGenericCue): Same. (WebCore::InbandGenericTextTrack::removeCue): Same. (WebCore::InbandGenericTextTrack::InbandGenericTextTrack): Empty. (WebCore::InbandGenericTextTrack::~InbandGenericTextTrack): Empty. * html/track/InbandGenericTextTrack.h: Copied from Source/WebCore/html/track/InbandTextTrack.h. The only addition is the ASSERT_NOT_REACHED() for WebVTT cues. * html/track/InbandTextTrack.cpp: Add label and language changed callbacks. (WebCore::InbandTextTrack::create): Return a generic or WebVTT text track based on the private CueFormat. (WebCore::InbandTextTrack::labelChanged): Added. (WebCore::InbandTextTrack::languageChanged): Added. * html/track/InbandTextTrack.h: Add label and language changed callbacks. * html/track/InbandWebVTTTextTrack.cpp: Added, based on InbandTextTrack. (WebCore::InbandWebVTTTextTrack::create): Same. (WebCore::InbandWebVTTTextTrack::InbandWebVTTTextTrack): Empty. (WebCore::InbandWebVTTTextTrack::~InbandWebVTTTextTrack): Empty. (WebCore::InbandWebVTTTextTrack::parseWebVTTCueData): Sends data to a WebVTTParser. (WebCore::InbandWebVTTTextTrack::newCuesParsed): Adds cues when WebVTTParser parses them. (WebCore::InbandWebVTTTextTrack::fileFailedToParse): Prints a warning when WebVTTParser has a problem. * html/track/InbandWebVTTTextTrack.h: Added. * platform/graphics/InbandTextTrackPrivate.h: (WebCore::InbandTextTrackPrivate::cueFormat): For determining if the track will have generic or WebVTT cues. (WebCore::InbandTextTrackPrivate::InbandTextTrackPrivate): Pass CueFormat in the constructor. * platform/graphics/InbandTextTrackPrivateClient.h: Same. * platform/graphics/avfoundation/InbandTextTrackPrivateAVF.cpp: (WebCore::InbandTextTrackPrivateAVF::InbandTextTrackPrivateAVF): Pass CueFormat (Generic) to InbandTextTrackPrivate. * platform/graphics/gstreamer/GRefPtrGStreamer.cpp: Add GRefPtr specializations for GstSample and GstEvent. * platform/graphics/gstreamer/GRefPtrGStreamer.h: Same. * platform/graphics/gstreamer/GStreamerUtilities.h: Add WARN_MEDIA_MESSAGE. * platform/graphics/gstreamer/GStreamerVersioning.h: Add a function to check GStreamer version at runtime. * platform/graphics/gstreamer/InbandTextTrackPrivateGStreamer.cpp: Added. (WebCore::textTrackPrivateEventCallback): Watches for tag and stream start events. (WebCore::textTrackPrivateSampleTimeoutCallback): See notifyTrackOfSample(); (WebCore::textTrackPrivateStreamTimeoutCallback): See notifyTrackOfStreamChanged(); (WebCore::textTrackPrivateTagsChangeTimeoutCallback): See notifyTrackOfTagsChanged(); (WebCore::InbandTextTrackPrivateGStreamer::InbandTextTrackPrivateGStreamer): Initializes tags and stream and sets up event callback. (WebCore::InbandTextTrackPrivateGStreamer::~InbandTextTrackPrivateGStreamer): Calls disconnect: (WebCore::InbandTextTrackPrivateGStreamer::disconnect): Removes signal handlers and frees some memory. (WebCore::InbandTextTrackPrivateGStreamer::handleSample): Adds samples to a list and sets up callback. (WebCore::InbandTextTrackPrivateGStreamer::streamChanged): Sets up callback. (WebCore::InbandTextTrackPrivateGStreamer::tagsChanged): Same. (WebCore::InbandTextTrackPrivateGStreamer::notifyTrackOfSample): Parses all queued samples with WebVTTParser. (WebCore::InbandTextTrackPrivateGStreamer::notifyTrackOfStreamChanged): Keeps track of current stream. (WebCore::InbandTextTrackPrivateGStreamer::notifyTrackOfTagsChanged): Sets label and language from tags. * platform/graphics/gstreamer/InbandTextTrackPrivateGStreamer.h: Added. (WebCore::InbandTextTrackPrivateGStreamer::create): Basic RefPtr create function. (WebCore::InbandTextTrackPrivateGStreamer::pad): Returns the pad this track is associated with (used to determine if a playbin text stream has already been associated with a text track). (WebCore::InbandTextTrackPrivateGStreamer::setIndex): Sets the track index (used for sorting). (WebCore::InbandTextTrackPrivateGStreamer::streamId): Returns the stream ID (used to handle new samples). * platform/graphics/gstreamer/MediaPlayerPrivateGStreamer.cpp: (WebCore::mediaPlayerPrivateTextChangedCallback): Called for playbin "text-changed" event. See textChanged(). (WebCore::mediaPlayerPrivateTextChangeTimeoutCallback): See notifyPlayerOfText(). (WebCore::mediaPlayerPrivateNewTextSampleCallback): See newTextSample(). (WebCore::MediaPlayerPrivateGStreamer::MediaPlayerPrivateGStreamer): Initialize m_textTimerHandler. (WebCore::MediaPlayerPrivateGStreamer::~MediaPlayerPrivateGStreamer): Disconnect text tracks (they don't necessarily get freed here, since a script could hold a reference). (WebCore::MediaPlayerPrivateGStreamer::textChanged): Setup callback for notifyPlayerOfText. (WebCore::MediaPlayerPrivateGStreamer::notifyPlayerOfText): Create text tracks. (WebCore::MediaPlayerPrivateGStreamer::newTextSample): Handle new samples by giving them to a text track with a matching stream. This method is syncryonous because we need to get the stream start sticky event immediately. * platform/graphics/gstreamer/MediaPlayerPrivateGStreamer.h: * platform/graphics/gstreamer/TextCombinerGStreamer.cpp: Added. This element forwards buffers from all of its input pads, but also converts plain text to WebVTT as needed. (webkit_text_combiner_init): Setup internal funnel. (webkitTextCombinerPadEvent): If the caps are plain text make sure we have a webvttenv, otherwise connect directly to the funnel. (webkitTextCombinerRequestNewPad): Setup ghostpad and event callback. (webkitTextCombinerReleasePad): Release pad and optional associated webvttenc. (webkit_text_combiner_class_init): Setup pad templates and request/release pad functions. (webkitTextCombinerNew): Returns a new WebKitTextCombiner. * platform/graphics/gstreamer/TextCombinerGStreamer.h: Added. * platform/graphics/gstreamer/TextSinkGStreamer.cpp: Added. (webkit_text_sink_init): Set sync=false. (webkitTextSinkGetProperty): Ignore sync property. (webkitTextSinkSetProperty): Same. (webkitTextSinkQuery): Ignore position and duration queries, forward everything else to appsink. (webkit_text_sink_class_init): Setup property and query functions. (webkitTextSinkNew): Return a new WebKitTextSink. * platform/graphics/gstreamer/TextSinkGStreamer.h: Added. LayoutTests: <https://webkit.org/b/103771> [GStreamer] support in-band text tracks Patch by Brendan Long <b.long@cablelabs.com> on 2013-08-30 Reviewed by Eric Carlson. * media/content/counting-subtitled-kate.ogv: Added. * media/content/counting-subtitled-srt.mkv: Added. * media/in-band-cues.js: Added. * media/track/in-band/track-in-band-kate-ogg-cues-added-once-expected.txt: Added. * media/track/in-band/track-in-band-kate-ogg-cues-added-once.html: Added. * media/track/in-band/track-in-band-kate-ogg-kind-expected.txt: Added. * media/track/in-band/track-in-band-kate-ogg-kind.html: Added. * media/track/in-band/track-in-band-kate-ogg-language-expected.txt: Added. * media/track/in-band/track-in-band-kate-ogg-language.html: Added. * media/track/in-band/track-in-band-kate-ogg-mode-expected.txt: Added. * media/track/in-band/track-in-band-kate-ogg-mode.html: Added. * media/track/in-band/track-in-band-kate-ogg-style-expected.txt: Added. * media/track/in-band/track-in-band-kate-ogg-style.html: Added. * media/track/in-band/track-in-band-kate-ogg-track-order-expected.txt: Added. * media/track/in-band/track-in-band-kate-ogg-track-order.html: Added. * media/track/in-band/track-in-band-srt-mkv-cues-added-once-expected.txt: Added. * media/track/in-band/track-in-band-srt-mkv-cues-added-once.html: Added. * media/track/in-band/track-in-band-srt-mkv-kind-expected.txt: Added. * media/track/in-band/track-in-band-srt-mkv-kind.html: Added. * media/track/in-band/track-in-band-srt-mkv-language-expected.txt: Added. * media/track/in-band/track-in-band-srt-mkv-language.html: Added. * media/track/in-band/track-in-band-srt-mkv-mode-expected.txt: Added. * media/track/in-band/track-in-band-srt-mkv-mode.html: Added. * media/track/in-band/track-in-band-srt-mkv-style-expected.txt: Added. * media/track/in-band/track-in-band-srt-mkv-style.html: Added. * media/track/in-band/track-in-band-srt-mkv-track-order-expected.txt: Added. * media/track/in-band/track-in-band-srt-mkv-track-order.html: Added. * platform/mac/TestExpectations: Skip MKV and OGG tests. Canonical link: https://commits.webkit.org/138536@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@154908 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2013-08-30 20:17:27 +00:00
consoleWrite("<br><i>** Add two tracks, check sort order<" + "/i>");
run("addTrack = video.addTextTrack('captions', 'Caption Track', 'en')");
run("trackElement = document.createElement('track')");
trackElement.label = '<track>';
run("video.appendChild(trackElement)");
[GStreamer] Support "chapter" text tracks https://bugs.webkit.org/show_bug.cgi?id=122000 Patch by Brendan Long <b.long@cablelabs.com> on 2013-10-01 Reviewed by Eric Carlson. Source/WebCore: Test: media/track/in-band/track-in-band-mkv-chapters.html * platform/graphics/gstreamer/InbandMetadataTextTrackPrivateGStreamer.h: New class, needed for cues that aren't associated with GstPads. * platform/graphics/gstreamer/MediaPlayerPrivateGStreamer.cpp: (WebCore::MediaPlayerPrivateGStreamer::handleMessage): Check for table of contents messages. (WebCore::MediaPlayerPrivateGStreamer::processTableOfContents): Parse table of contents messages into text tracks. (WebCore::MediaPlayerPrivateGStreamer::processTableOfContentsEntry): Recursive method for parsing all of the TOC entries. * platform/graphics/gstreamer/MediaPlayerPrivateGStreamer.h: Add the processTableOfContents functions. LayoutTests: * media/content/counting-subtitled-srt.mkv: Added chapters metadata. * media/track/in-band/track-in-band-mkv-chapters-expected.txt: New test, similar to cues-added-one, but uses "chapters" track. * media/track/in-band/track-in-band-mkv-chapters.html: Same. * media/in-band-cues.js: Updated tests to handle the new "chapters" track. * media/track/in-band/track-in-band-kate-ogg-cues-added-once-expected.txt: Same. * media/track/in-band/track-in-band-kate-ogg-cues-added-once.html: Same. * media/track/in-band/track-in-band-kate-ogg-mode-expected.txt: Same. * media/track/in-band/track-in-band-kate-ogg-mode.html: Same. * media/track/in-band/track-in-band-kate-ogg-style-expected.txt: Same. * media/track/in-band/track-in-band-kate-ogg-track-order-expected.txt: Same. * media/track/in-band/track-in-band-kate-ogg-track-order.html: Same. * media/track/in-band/track-in-band-srt-mkv-cues-added-once-expected.txt: Same. * media/track/in-band/track-in-band-srt-mkv-cues-added-once.html: Same. * media/track/in-band/track-in-band-srt-mkv-kind-expected.txt: Same. * media/track/in-band/track-in-band-srt-mkv-kind.html: Same. * media/track/in-band/track-in-band-srt-mkv-language-expected.txt: Same. * media/track/in-band/track-in-band-srt-mkv-language.html: Same. * media/track/in-band/track-in-band-srt-mkv-mode-expected.txt: Same. * media/track/in-band/track-in-band-srt-mkv-mode.html: Same. * media/track/in-band/track-in-band-srt-mkv-style-expected.txt: Same. * media/track/in-band/track-in-band-srt-mkv-track-order-expected.txt: Same. * media/track/in-band/track-in-band-srt-mkv-track-order.html: Same. Canonical link: https://commits.webkit.org/140209@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@156713 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2013-10-01 16:26:52 +00:00
testExpected("video.textTracks.length", numInBandTracks + 2);
Source/WebCore: [GStreamer] support in-band text tracks https://bugs.webkit.org/show_bug.cgi?id=103771 Patch by Brendan Long <b.long@cablelabs.com> on 2013-08-30 Reviewed by Eric Carlson. Tests: New tests added because existing tests were too specific. media/track/in-band/track-in-band-kate-ogg-cues-added-once.html media/track/in-band/track-in-band-kate-ogg-kind.html media/track/in-band/track-in-band-kate-ogg-language.html media/track/in-band/track-in-band-kate-ogg-mode.html media/track/in-band/track-in-band-kate-ogg-style.html media/track/in-band/track-in-band-kate-ogg-track-order.html media/track/in-band/track-in-band-srt-mkv-cues-added-once.html media/track/in-band/track-in-band-srt-mkv-kind.html media/track/in-band/track-in-band-srt-mkv-language.html media/track/in-band/track-in-band-srt-mkv-mode.html media/track/in-band/track-in-band-srt-mkv-style.html media/track/in-band/track-in-band-srt-mkv-track-order.html * CMakeLists.txt: Add InbandTextTrackPrivateGStreamer, InbandGenericTextTrack, InbandWebVTTTextTrack, and TextCombinerGStreamer files. * GNUmakefile.list.am: Same. * PlatformEfl.cmake: Same. * Target.pri: Same. * WebCore.vcxproj/WebCore.vcxproj: Same. * WebCore.vcxproj/WebCore.vcxproj.filters: Same. * WebCore.xcodeproj/project.pbxproj: Same. * html/track/InbandGenericTextTrack.cpp: Split out code for handling generic cues. (WebCore::GenericTextTrackCueMap::GenericTextTrackCueMap): Move from InbandTextTrack. (WebCore::GenericTextTrackCueMap::~GenericTextTrackCueMap): Same. (WebCore::GenericTextTrackCueMap::add): Same. (WebCore::GenericTextTrackCueMap::find): Same. (WebCore::GenericTextTrackCueMap::remove): Same. (WebCore::InbandGenericTextTrack::create): Same. (WebCore::InbandGenericTextTrack::updateCueFromCueData): Same. (WebCore::InbandGenericTextTrack::addGenericCue): Same. (WebCore::InbandGenericTextTrack::updateGenericCue): Same. (WebCore::InbandGenericTextTrack::removeGenericCue): Same. (WebCore::InbandGenericTextTrack::removeCue): Same. (WebCore::InbandGenericTextTrack::InbandGenericTextTrack): Empty. (WebCore::InbandGenericTextTrack::~InbandGenericTextTrack): Empty. * html/track/InbandGenericTextTrack.h: Copied from Source/WebCore/html/track/InbandTextTrack.h. The only addition is the ASSERT_NOT_REACHED() for WebVTT cues. * html/track/InbandTextTrack.cpp: Add label and language changed callbacks. (WebCore::InbandTextTrack::create): Return a generic or WebVTT text track based on the private CueFormat. (WebCore::InbandTextTrack::labelChanged): Added. (WebCore::InbandTextTrack::languageChanged): Added. * html/track/InbandTextTrack.h: Add label and language changed callbacks. * html/track/InbandWebVTTTextTrack.cpp: Added, based on InbandTextTrack. (WebCore::InbandWebVTTTextTrack::create): Same. (WebCore::InbandWebVTTTextTrack::InbandWebVTTTextTrack): Empty. (WebCore::InbandWebVTTTextTrack::~InbandWebVTTTextTrack): Empty. (WebCore::InbandWebVTTTextTrack::parseWebVTTCueData): Sends data to a WebVTTParser. (WebCore::InbandWebVTTTextTrack::newCuesParsed): Adds cues when WebVTTParser parses them. (WebCore::InbandWebVTTTextTrack::fileFailedToParse): Prints a warning when WebVTTParser has a problem. * html/track/InbandWebVTTTextTrack.h: Added. * platform/graphics/InbandTextTrackPrivate.h: (WebCore::InbandTextTrackPrivate::cueFormat): For determining if the track will have generic or WebVTT cues. (WebCore::InbandTextTrackPrivate::InbandTextTrackPrivate): Pass CueFormat in the constructor. * platform/graphics/InbandTextTrackPrivateClient.h: Same. * platform/graphics/avfoundation/InbandTextTrackPrivateAVF.cpp: (WebCore::InbandTextTrackPrivateAVF::InbandTextTrackPrivateAVF): Pass CueFormat (Generic) to InbandTextTrackPrivate. * platform/graphics/gstreamer/GRefPtrGStreamer.cpp: Add GRefPtr specializations for GstSample and GstEvent. * platform/graphics/gstreamer/GRefPtrGStreamer.h: Same. * platform/graphics/gstreamer/GStreamerUtilities.h: Add WARN_MEDIA_MESSAGE. * platform/graphics/gstreamer/GStreamerVersioning.h: Add a function to check GStreamer version at runtime. * platform/graphics/gstreamer/InbandTextTrackPrivateGStreamer.cpp: Added. (WebCore::textTrackPrivateEventCallback): Watches for tag and stream start events. (WebCore::textTrackPrivateSampleTimeoutCallback): See notifyTrackOfSample(); (WebCore::textTrackPrivateStreamTimeoutCallback): See notifyTrackOfStreamChanged(); (WebCore::textTrackPrivateTagsChangeTimeoutCallback): See notifyTrackOfTagsChanged(); (WebCore::InbandTextTrackPrivateGStreamer::InbandTextTrackPrivateGStreamer): Initializes tags and stream and sets up event callback. (WebCore::InbandTextTrackPrivateGStreamer::~InbandTextTrackPrivateGStreamer): Calls disconnect: (WebCore::InbandTextTrackPrivateGStreamer::disconnect): Removes signal handlers and frees some memory. (WebCore::InbandTextTrackPrivateGStreamer::handleSample): Adds samples to a list and sets up callback. (WebCore::InbandTextTrackPrivateGStreamer::streamChanged): Sets up callback. (WebCore::InbandTextTrackPrivateGStreamer::tagsChanged): Same. (WebCore::InbandTextTrackPrivateGStreamer::notifyTrackOfSample): Parses all queued samples with WebVTTParser. (WebCore::InbandTextTrackPrivateGStreamer::notifyTrackOfStreamChanged): Keeps track of current stream. (WebCore::InbandTextTrackPrivateGStreamer::notifyTrackOfTagsChanged): Sets label and language from tags. * platform/graphics/gstreamer/InbandTextTrackPrivateGStreamer.h: Added. (WebCore::InbandTextTrackPrivateGStreamer::create): Basic RefPtr create function. (WebCore::InbandTextTrackPrivateGStreamer::pad): Returns the pad this track is associated with (used to determine if a playbin text stream has already been associated with a text track). (WebCore::InbandTextTrackPrivateGStreamer::setIndex): Sets the track index (used for sorting). (WebCore::InbandTextTrackPrivateGStreamer::streamId): Returns the stream ID (used to handle new samples). * platform/graphics/gstreamer/MediaPlayerPrivateGStreamer.cpp: (WebCore::mediaPlayerPrivateTextChangedCallback): Called for playbin "text-changed" event. See textChanged(). (WebCore::mediaPlayerPrivateTextChangeTimeoutCallback): See notifyPlayerOfText(). (WebCore::mediaPlayerPrivateNewTextSampleCallback): See newTextSample(). (WebCore::MediaPlayerPrivateGStreamer::MediaPlayerPrivateGStreamer): Initialize m_textTimerHandler. (WebCore::MediaPlayerPrivateGStreamer::~MediaPlayerPrivateGStreamer): Disconnect text tracks (they don't necessarily get freed here, since a script could hold a reference). (WebCore::MediaPlayerPrivateGStreamer::textChanged): Setup callback for notifyPlayerOfText. (WebCore::MediaPlayerPrivateGStreamer::notifyPlayerOfText): Create text tracks. (WebCore::MediaPlayerPrivateGStreamer::newTextSample): Handle new samples by giving them to a text track with a matching stream. This method is syncryonous because we need to get the stream start sticky event immediately. * platform/graphics/gstreamer/MediaPlayerPrivateGStreamer.h: * platform/graphics/gstreamer/TextCombinerGStreamer.cpp: Added. This element forwards buffers from all of its input pads, but also converts plain text to WebVTT as needed. (webkit_text_combiner_init): Setup internal funnel. (webkitTextCombinerPadEvent): If the caps are plain text make sure we have a webvttenv, otherwise connect directly to the funnel. (webkitTextCombinerRequestNewPad): Setup ghostpad and event callback. (webkitTextCombinerReleasePad): Release pad and optional associated webvttenc. (webkit_text_combiner_class_init): Setup pad templates and request/release pad functions. (webkitTextCombinerNew): Returns a new WebKitTextCombiner. * platform/graphics/gstreamer/TextCombinerGStreamer.h: Added. * platform/graphics/gstreamer/TextSinkGStreamer.cpp: Added. (webkit_text_sink_init): Set sync=false. (webkitTextSinkGetProperty): Ignore sync property. (webkitTextSinkSetProperty): Same. (webkitTextSinkQuery): Ignore position and duration queries, forward everything else to appsink. (webkit_text_sink_class_init): Setup property and query functions. (webkitTextSinkNew): Return a new WebKitTextSink. * platform/graphics/gstreamer/TextSinkGStreamer.h: Added. LayoutTests: <https://webkit.org/b/103771> [GStreamer] support in-band text tracks Patch by Brendan Long <b.long@cablelabs.com> on 2013-08-30 Reviewed by Eric Carlson. * media/content/counting-subtitled-kate.ogv: Added. * media/content/counting-subtitled-srt.mkv: Added. * media/in-band-cues.js: Added. * media/track/in-band/track-in-band-kate-ogg-cues-added-once-expected.txt: Added. * media/track/in-band/track-in-band-kate-ogg-cues-added-once.html: Added. * media/track/in-band/track-in-band-kate-ogg-kind-expected.txt: Added. * media/track/in-band/track-in-band-kate-ogg-kind.html: Added. * media/track/in-band/track-in-band-kate-ogg-language-expected.txt: Added. * media/track/in-band/track-in-band-kate-ogg-language.html: Added. * media/track/in-band/track-in-band-kate-ogg-mode-expected.txt: Added. * media/track/in-band/track-in-band-kate-ogg-mode.html: Added. * media/track/in-band/track-in-band-kate-ogg-style-expected.txt: Added. * media/track/in-band/track-in-band-kate-ogg-style.html: Added. * media/track/in-band/track-in-band-kate-ogg-track-order-expected.txt: Added. * media/track/in-band/track-in-band-kate-ogg-track-order.html: Added. * media/track/in-band/track-in-band-srt-mkv-cues-added-once-expected.txt: Added. * media/track/in-band/track-in-band-srt-mkv-cues-added-once.html: Added. * media/track/in-band/track-in-band-srt-mkv-kind-expected.txt: Added. * media/track/in-band/track-in-band-srt-mkv-kind.html: Added. * media/track/in-band/track-in-band-srt-mkv-language-expected.txt: Added. * media/track/in-band/track-in-band-srt-mkv-language.html: Added. * media/track/in-band/track-in-band-srt-mkv-mode-expected.txt: Added. * media/track/in-band/track-in-band-srt-mkv-mode.html: Added. * media/track/in-band/track-in-band-srt-mkv-style-expected.txt: Added. * media/track/in-band/track-in-band-srt-mkv-style.html: Added. * media/track/in-band/track-in-band-srt-mkv-track-order-expected.txt: Added. * media/track/in-band/track-in-band-srt-mkv-track-order.html: Added. * platform/mac/TestExpectations: Skip MKV and OGG tests. Canonical link: https://commits.webkit.org/138536@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@154908 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2013-08-30 20:17:27 +00:00
compareTracks("video.textTracks[0]", "trackElement.track");
compareTracks("video.textTracks[1]", "addTrack");
[GStreamer] Support "chapter" text tracks https://bugs.webkit.org/show_bug.cgi?id=122000 Patch by Brendan Long <b.long@cablelabs.com> on 2013-10-01 Reviewed by Eric Carlson. Source/WebCore: Test: media/track/in-band/track-in-band-mkv-chapters.html * platform/graphics/gstreamer/InbandMetadataTextTrackPrivateGStreamer.h: New class, needed for cues that aren't associated with GstPads. * platform/graphics/gstreamer/MediaPlayerPrivateGStreamer.cpp: (WebCore::MediaPlayerPrivateGStreamer::handleMessage): Check for table of contents messages. (WebCore::MediaPlayerPrivateGStreamer::processTableOfContents): Parse table of contents messages into text tracks. (WebCore::MediaPlayerPrivateGStreamer::processTableOfContentsEntry): Recursive method for parsing all of the TOC entries. * platform/graphics/gstreamer/MediaPlayerPrivateGStreamer.h: Add the processTableOfContents functions. LayoutTests: * media/content/counting-subtitled-srt.mkv: Added chapters metadata. * media/track/in-band/track-in-band-mkv-chapters-expected.txt: New test, similar to cues-added-one, but uses "chapters" track. * media/track/in-band/track-in-band-mkv-chapters.html: Same. * media/in-band-cues.js: Updated tests to handle the new "chapters" track. * media/track/in-band/track-in-band-kate-ogg-cues-added-once-expected.txt: Same. * media/track/in-band/track-in-band-kate-ogg-cues-added-once.html: Same. * media/track/in-band/track-in-band-kate-ogg-mode-expected.txt: Same. * media/track/in-band/track-in-band-kate-ogg-mode.html: Same. * media/track/in-band/track-in-band-kate-ogg-style-expected.txt: Same. * media/track/in-band/track-in-band-kate-ogg-track-order-expected.txt: Same. * media/track/in-band/track-in-band-kate-ogg-track-order.html: Same. * media/track/in-band/track-in-band-srt-mkv-cues-added-once-expected.txt: Same. * media/track/in-band/track-in-band-srt-mkv-cues-added-once.html: Same. * media/track/in-band/track-in-band-srt-mkv-kind-expected.txt: Same. * media/track/in-band/track-in-band-srt-mkv-kind.html: Same. * media/track/in-band/track-in-band-srt-mkv-language-expected.txt: Same. * media/track/in-band/track-in-band-srt-mkv-language.html: Same. * media/track/in-band/track-in-band-srt-mkv-mode-expected.txt: Same. * media/track/in-band/track-in-band-srt-mkv-mode.html: Same. * media/track/in-band/track-in-band-srt-mkv-style-expected.txt: Same. * media/track/in-band/track-in-band-srt-mkv-track-order-expected.txt: Same. * media/track/in-band/track-in-band-srt-mkv-track-order.html: Same. Canonical link: https://commits.webkit.org/140209@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@156713 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2013-10-01 16:26:52 +00:00
for (var i = 1; i < numInBandTracks + 1; ++i)
compareTracks("video.textTracks[" + (i + 1) + "]", "inbandTrack" + i);
Source/WebCore: [GStreamer] support in-band text tracks https://bugs.webkit.org/show_bug.cgi?id=103771 Patch by Brendan Long <b.long@cablelabs.com> on 2013-08-30 Reviewed by Eric Carlson. Tests: New tests added because existing tests were too specific. media/track/in-band/track-in-band-kate-ogg-cues-added-once.html media/track/in-band/track-in-band-kate-ogg-kind.html media/track/in-band/track-in-band-kate-ogg-language.html media/track/in-band/track-in-band-kate-ogg-mode.html media/track/in-band/track-in-band-kate-ogg-style.html media/track/in-band/track-in-band-kate-ogg-track-order.html media/track/in-band/track-in-band-srt-mkv-cues-added-once.html media/track/in-band/track-in-band-srt-mkv-kind.html media/track/in-band/track-in-band-srt-mkv-language.html media/track/in-band/track-in-band-srt-mkv-mode.html media/track/in-band/track-in-band-srt-mkv-style.html media/track/in-band/track-in-band-srt-mkv-track-order.html * CMakeLists.txt: Add InbandTextTrackPrivateGStreamer, InbandGenericTextTrack, InbandWebVTTTextTrack, and TextCombinerGStreamer files. * GNUmakefile.list.am: Same. * PlatformEfl.cmake: Same. * Target.pri: Same. * WebCore.vcxproj/WebCore.vcxproj: Same. * WebCore.vcxproj/WebCore.vcxproj.filters: Same. * WebCore.xcodeproj/project.pbxproj: Same. * html/track/InbandGenericTextTrack.cpp: Split out code for handling generic cues. (WebCore::GenericTextTrackCueMap::GenericTextTrackCueMap): Move from InbandTextTrack. (WebCore::GenericTextTrackCueMap::~GenericTextTrackCueMap): Same. (WebCore::GenericTextTrackCueMap::add): Same. (WebCore::GenericTextTrackCueMap::find): Same. (WebCore::GenericTextTrackCueMap::remove): Same. (WebCore::InbandGenericTextTrack::create): Same. (WebCore::InbandGenericTextTrack::updateCueFromCueData): Same. (WebCore::InbandGenericTextTrack::addGenericCue): Same. (WebCore::InbandGenericTextTrack::updateGenericCue): Same. (WebCore::InbandGenericTextTrack::removeGenericCue): Same. (WebCore::InbandGenericTextTrack::removeCue): Same. (WebCore::InbandGenericTextTrack::InbandGenericTextTrack): Empty. (WebCore::InbandGenericTextTrack::~InbandGenericTextTrack): Empty. * html/track/InbandGenericTextTrack.h: Copied from Source/WebCore/html/track/InbandTextTrack.h. The only addition is the ASSERT_NOT_REACHED() for WebVTT cues. * html/track/InbandTextTrack.cpp: Add label and language changed callbacks. (WebCore::InbandTextTrack::create): Return a generic or WebVTT text track based on the private CueFormat. (WebCore::InbandTextTrack::labelChanged): Added. (WebCore::InbandTextTrack::languageChanged): Added. * html/track/InbandTextTrack.h: Add label and language changed callbacks. * html/track/InbandWebVTTTextTrack.cpp: Added, based on InbandTextTrack. (WebCore::InbandWebVTTTextTrack::create): Same. (WebCore::InbandWebVTTTextTrack::InbandWebVTTTextTrack): Empty. (WebCore::InbandWebVTTTextTrack::~InbandWebVTTTextTrack): Empty. (WebCore::InbandWebVTTTextTrack::parseWebVTTCueData): Sends data to a WebVTTParser. (WebCore::InbandWebVTTTextTrack::newCuesParsed): Adds cues when WebVTTParser parses them. (WebCore::InbandWebVTTTextTrack::fileFailedToParse): Prints a warning when WebVTTParser has a problem. * html/track/InbandWebVTTTextTrack.h: Added. * platform/graphics/InbandTextTrackPrivate.h: (WebCore::InbandTextTrackPrivate::cueFormat): For determining if the track will have generic or WebVTT cues. (WebCore::InbandTextTrackPrivate::InbandTextTrackPrivate): Pass CueFormat in the constructor. * platform/graphics/InbandTextTrackPrivateClient.h: Same. * platform/graphics/avfoundation/InbandTextTrackPrivateAVF.cpp: (WebCore::InbandTextTrackPrivateAVF::InbandTextTrackPrivateAVF): Pass CueFormat (Generic) to InbandTextTrackPrivate. * platform/graphics/gstreamer/GRefPtrGStreamer.cpp: Add GRefPtr specializations for GstSample and GstEvent. * platform/graphics/gstreamer/GRefPtrGStreamer.h: Same. * platform/graphics/gstreamer/GStreamerUtilities.h: Add WARN_MEDIA_MESSAGE. * platform/graphics/gstreamer/GStreamerVersioning.h: Add a function to check GStreamer version at runtime. * platform/graphics/gstreamer/InbandTextTrackPrivateGStreamer.cpp: Added. (WebCore::textTrackPrivateEventCallback): Watches for tag and stream start events. (WebCore::textTrackPrivateSampleTimeoutCallback): See notifyTrackOfSample(); (WebCore::textTrackPrivateStreamTimeoutCallback): See notifyTrackOfStreamChanged(); (WebCore::textTrackPrivateTagsChangeTimeoutCallback): See notifyTrackOfTagsChanged(); (WebCore::InbandTextTrackPrivateGStreamer::InbandTextTrackPrivateGStreamer): Initializes tags and stream and sets up event callback. (WebCore::InbandTextTrackPrivateGStreamer::~InbandTextTrackPrivateGStreamer): Calls disconnect: (WebCore::InbandTextTrackPrivateGStreamer::disconnect): Removes signal handlers and frees some memory. (WebCore::InbandTextTrackPrivateGStreamer::handleSample): Adds samples to a list and sets up callback. (WebCore::InbandTextTrackPrivateGStreamer::streamChanged): Sets up callback. (WebCore::InbandTextTrackPrivateGStreamer::tagsChanged): Same. (WebCore::InbandTextTrackPrivateGStreamer::notifyTrackOfSample): Parses all queued samples with WebVTTParser. (WebCore::InbandTextTrackPrivateGStreamer::notifyTrackOfStreamChanged): Keeps track of current stream. (WebCore::InbandTextTrackPrivateGStreamer::notifyTrackOfTagsChanged): Sets label and language from tags. * platform/graphics/gstreamer/InbandTextTrackPrivateGStreamer.h: Added. (WebCore::InbandTextTrackPrivateGStreamer::create): Basic RefPtr create function. (WebCore::InbandTextTrackPrivateGStreamer::pad): Returns the pad this track is associated with (used to determine if a playbin text stream has already been associated with a text track). (WebCore::InbandTextTrackPrivateGStreamer::setIndex): Sets the track index (used for sorting). (WebCore::InbandTextTrackPrivateGStreamer::streamId): Returns the stream ID (used to handle new samples). * platform/graphics/gstreamer/MediaPlayerPrivateGStreamer.cpp: (WebCore::mediaPlayerPrivateTextChangedCallback): Called for playbin "text-changed" event. See textChanged(). (WebCore::mediaPlayerPrivateTextChangeTimeoutCallback): See notifyPlayerOfText(). (WebCore::mediaPlayerPrivateNewTextSampleCallback): See newTextSample(). (WebCore::MediaPlayerPrivateGStreamer::MediaPlayerPrivateGStreamer): Initialize m_textTimerHandler. (WebCore::MediaPlayerPrivateGStreamer::~MediaPlayerPrivateGStreamer): Disconnect text tracks (they don't necessarily get freed here, since a script could hold a reference). (WebCore::MediaPlayerPrivateGStreamer::textChanged): Setup callback for notifyPlayerOfText. (WebCore::MediaPlayerPrivateGStreamer::notifyPlayerOfText): Create text tracks. (WebCore::MediaPlayerPrivateGStreamer::newTextSample): Handle new samples by giving them to a text track with a matching stream. This method is syncryonous because we need to get the stream start sticky event immediately. * platform/graphics/gstreamer/MediaPlayerPrivateGStreamer.h: * platform/graphics/gstreamer/TextCombinerGStreamer.cpp: Added. This element forwards buffers from all of its input pads, but also converts plain text to WebVTT as needed. (webkit_text_combiner_init): Setup internal funnel. (webkitTextCombinerPadEvent): If the caps are plain text make sure we have a webvttenv, otherwise connect directly to the funnel. (webkitTextCombinerRequestNewPad): Setup ghostpad and event callback. (webkitTextCombinerReleasePad): Release pad and optional associated webvttenc. (webkit_text_combiner_class_init): Setup pad templates and request/release pad functions. (webkitTextCombinerNew): Returns a new WebKitTextCombiner. * platform/graphics/gstreamer/TextCombinerGStreamer.h: Added. * platform/graphics/gstreamer/TextSinkGStreamer.cpp: Added. (webkit_text_sink_init): Set sync=false. (webkitTextSinkGetProperty): Ignore sync property. (webkitTextSinkSetProperty): Same. (webkitTextSinkQuery): Ignore position and duration queries, forward everything else to appsink. (webkit_text_sink_class_init): Setup property and query functions. (webkitTextSinkNew): Return a new WebKitTextSink. * platform/graphics/gstreamer/TextSinkGStreamer.h: Added. LayoutTests: <https://webkit.org/b/103771> [GStreamer] support in-band text tracks Patch by Brendan Long <b.long@cablelabs.com> on 2013-08-30 Reviewed by Eric Carlson. * media/content/counting-subtitled-kate.ogv: Added. * media/content/counting-subtitled-srt.mkv: Added. * media/in-band-cues.js: Added. * media/track/in-band/track-in-band-kate-ogg-cues-added-once-expected.txt: Added. * media/track/in-band/track-in-band-kate-ogg-cues-added-once.html: Added. * media/track/in-band/track-in-band-kate-ogg-kind-expected.txt: Added. * media/track/in-band/track-in-band-kate-ogg-kind.html: Added. * media/track/in-band/track-in-band-kate-ogg-language-expected.txt: Added. * media/track/in-band/track-in-band-kate-ogg-language.html: Added. * media/track/in-band/track-in-band-kate-ogg-mode-expected.txt: Added. * media/track/in-band/track-in-band-kate-ogg-mode.html: Added. * media/track/in-band/track-in-band-kate-ogg-style-expected.txt: Added. * media/track/in-band/track-in-band-kate-ogg-style.html: Added. * media/track/in-band/track-in-band-kate-ogg-track-order-expected.txt: Added. * media/track/in-band/track-in-band-kate-ogg-track-order.html: Added. * media/track/in-band/track-in-band-srt-mkv-cues-added-once-expected.txt: Added. * media/track/in-band/track-in-band-srt-mkv-cues-added-once.html: Added. * media/track/in-band/track-in-band-srt-mkv-kind-expected.txt: Added. * media/track/in-band/track-in-band-srt-mkv-kind.html: Added. * media/track/in-band/track-in-band-srt-mkv-language-expected.txt: Added. * media/track/in-band/track-in-band-srt-mkv-language.html: Added. * media/track/in-band/track-in-band-srt-mkv-mode-expected.txt: Added. * media/track/in-band/track-in-band-srt-mkv-mode.html: Added. * media/track/in-band/track-in-band-srt-mkv-style-expected.txt: Added. * media/track/in-band/track-in-band-srt-mkv-style.html: Added. * media/track/in-band/track-in-band-srt-mkv-track-order-expected.txt: Added. * media/track/in-band/track-in-band-srt-mkv-track-order.html: Added. * platform/mac/TestExpectations: Skip MKV and OGG tests. Canonical link: https://commits.webkit.org/138536@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@154908 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2013-08-30 20:17:27 +00:00
consoleWrite("<br><i>** Unload video file, check track count<" + "/i>");
run("video.src = ''");
testExpected("video.textTracks.length", 2);
consoleWrite("");
endTest();
}
findMediaElement();
video.src = uri;
waitForEvent('canplaythrough', canplaythrough);
}