Timeline & Playback Handling

content can be consumed in various ways, including live viewing or as video on demand (vod) after an event has taken place additionally, it is possible to watch content concurrently on multiple devices, preferably with synchronized playback across these devices for both live and vod content as a result of these diverse use cases, different requirements for playback handling arise each scenario demands specific behaviors to ensure a seamless experience, which we will explore in the next sections it is important to be mindful of terminology to avoid confusion the context in which timeline and playback modes and states are used may vary, whether from a technical perspective, timeline management standpoint, or the end user's content consumption experience as an illustration, the term "live" can carry different interpretations depending on the context from a player perspective, it may refer to the live edge or the current playback position from an event standpoint, it could pertain to an ongoing real time happening furthermore, it may signify that the timeline is in a live state or that the system is presently operating in live mode therefore, in the next sections we will try to be very specific when we talk about them the main experience timeline can be in 3 modes and 5 different states timeline & playback handling docid\ unv6a8vmlzqsyaou5kyx5 timeline & playback handling docid\ unv6a8vmlzqsyaou5kyx5 timeline & playback handling docid\ unv6a8vmlzqsyaou5kyx5 the system also uses two playback strategies in different states, namely timeline & playback handling docid\ unv6a8vmlzqsyaou5kyx5 timeline & playback handling docid\ unv6a8vmlzqsyaou5kyx5 timeline modes & states live mode (live and go live states) the live mode enables users to play content at the live edge, which is defined in the exp manifest this manifest includes a duration property that describes the virtual timeline, typically for an event the live edge latency is determined by the smep service docid\ bahfu3fbshk0ctqauif9e during the generation of the exp manifest docid\ rlfte lps44hdjfyopv2l to achieve this, the smep system estimates the latencies of all relevant content and selects an optimal live edge latency when an experience is launched and the exp manifest is still live, the system enters the live mode , switching to the live state additionally, there might be a desire to offer users the option of watching without spoilers, allowing them to commence the experience from the event's beginning in the go live state it is important not to confuse the term "live" with the live mode or live state in this context the timeline can be in sync mode while the event is still live, potentially introducing some live capabilities into the experience when in the live state, the timeline & playback handling docid\ unv6a8vmlzqsyaou5kyx5 is used for playback, where the exp engine attempts to synchronize with the live edge specified in the manifest and automatically catches up if playback falls behind due to network conditions or buffering in the go live state, the timeline & playback handling docid\ unv6a8vmlzqsyaou5kyx5 is used for playback, where content is not continually synchronized against any specific target additionally, in the live state, non seekable content (e g , webrtc based streams) may be preferred over seekable http based streams with higher latency once the user seeks back, exits the live state and switches to the go live state, the system starts using seekable content the system stays in the live state until either the exp manifest concludes, prompting a transition from live mode to vod mode and changing states, or until a user initiates manual interaction while in live mode , like seeking back for a replay during the live event in the latter case, the system switches to the go live state upon transitioning from live mode to vod mode , the system automatically shifts to the vod state when the user reaches the end of the timeline, a replay event button is displayed similarly, if the user is in vod mode and the event becomes live again (e g , due to changes in the event's end time), the system switches to live mode but enters the go live state instead of the live state to prevent skipping any content sync mode (synced and go sync states) the sync mode and the respective synced and go sync states functions similarly to the live mode and its states, but instead of synchronizing the content against the live edge while in synced state, it uses another target for synchronization for instance, for a second screen audio sync, the exp engine aligns with the audio being played on the tv similiar to the live mode , the timeline & playback handling docid\ unv6a8vmlzqsyaou5kyx5 is used for playback while in synced state and the timeline & playback handling docid\ unv6a8vmlzqsyaou5kyx5 while in go sync state the system transitions to the sync mode through user interaction, such as performing a second screen audio sync, and can be exited by another interaction the system won't leave the sync mode automatically only by manual interaction there is a key difference between the synced state and the live state when it comes to handling non seekable content when the synchronization target is in close proximity to the live edge, within a certain threshold, non seekable content becomes a viable option however, beyond this threshold, the system exclusively utilizes seekable content as mentioned above, the event may still be "live", introducing live capabilities to the system, despite the timeline being in sync mode for example, live notifications could be active while the event is live and the timeline is in sync mode which will be disabled once the event ends hence, managing the "live" status of an event is an additional consideration alongside the timeline mode and is independent of the live mode vod mode (vod state) the vod mode is the default state when the timeline is neither in live mode or sync mode this mode has only one state, the vod state in the vod state, the timeline & playback handling docid\ unv6a8vmlzqsyaou5kyx5 is used for playback, where content is not continually synchronized against any specific target playback strategies catch up strategy this playback strategy is used when in the live or synced state when using this strategy, the exp engine attempts to synchronize against a specific target e g the live edge or a sync target e g the tv audio and automatically catches up if playback falls behind due to network conditions or buffering some content may have higher latency, but the exp engine handles it by playing this content with an artificial delay when the network conditions consistently fail to support smooth content playback at the sync target for an extended duration, rather than just temporarily, the exp system can respond by issuing a warning to the user this notification informs the user that smooth playback is not feasible in the current state (e g live or synced ) and recommends falling back for a better experience, resulting in a switch to the go live or go sync state respectively to underline, it's important to note that while using this strategy the system doesn't automatically transition away from its current state, even during unfavorable network conditions the system's aim is to remain synced with the target, even if this results in ongoing buffering therefore, the above mentioned notification system is used, enabling users to easily transition from the current state to an alternative state that utilizes the default strategy for a more seamless playback experience default strategy this playback strategy is used when in the go live , go sync or vod state when using this strategy, content playback is not consistently synchronized against any specific target consequently, we allow content to buffer, minimizing or even completely avoiding interruptions or jumps after playback starts following a seek to better illustrate this concept, consider the example of being in the live state using the catch up strategy , where playback falls behind due to network request latencies the system attempts to catch up and speed up the playback, which is the intended behavior however, if the same catch up mechanism is applied in the vod state, seeking to a specific point in time would lead to an immediate start of the virtual playback timeline without waiting for the content this could cause issues, like playback speed ups, if the content requires additional time to initialize and buffer after a seek action or even re seeks which can result in playback freezes to avoid this, we adopt a different approach in the vod state by using the default strategy we allow the content to buffer together and pause the virtual playback timeline during this buffering period until the content is fully ready to be played back this strategy becomes particularly crucial in complex scenarios with multiple independent audio or video tracks e g for global audio or split view, where we aim to minimize any visible or audible catch up mechanic for content synchronization especially after a seek action on the timeline by the user