A bug in Timeline where the BGM/SE of Audio Track is out of sync with the video due to a drop in the game's frame rate.

In old movie theaters, the ``projector'' (image) that rotated the film and the ``gramophone (audio)'' that produced the sound were physically separated, and each was powered by a separate motor.

At the beginning of the movie (0 seconds), drop the projector's start button and the gramophone's needle at the same time. The timing is perfect at first. However, halfway through the project, the projector's motor became hot and the rotational speed suddenly dropped (frame drop). At this time, the gramophone continues to spin the record at a constant speed, saying, ``I'm not connected to the projector, I'm just spinning at my own pace.'' As a result, even though the video is delayed by one second, the audio continues to advance, resulting in a fatal "sound lag" in which the voice is heard before the actor starts speaking.

The standard setting in Unity (Update Mode = Game Time) is exactly the same. If the video (game frame) stutters due to a processing drop, the progress of the timeline will also be delayed, but the audio played inside the sound card (DSP clock) of your PC or smartphone will continue to play at a constant pitch without being affected by the processing drop. Due to these "two independent clocks", the discrepancy increases over time. To solve this problem, it is necessary to tie a string that allows you to manually advance or retard the projector's film to match the phonograph's playback pace (DSP clock) and keep it synchronized.

2. Root cause: Discrepancy between Game Time and DSP Clock

Technically speaking, Unity's normal timeline progress is evaluated based on Time.time (game clock). This is because the time update is calculated based on the processing interval (frame rate) of the CPU's main loop (Update), so if a processing drop occurs, the time per frame will be uneven.

On the other hand, Unity's audio system (internally known as FMOD) is powered by a separate hardware-specific, highly accurate clock called the DSP (Digital Signal Processing) clock, which keeps the OS's audio buffer filled at a constant sampling rate (e.g. 48kHz).

When the game's FPS drops from 60 to 20, the game clock progresses erratically, but the DSP clock continues to tick exactly one second according to physical time. Therefore, while the audio file placed in the AudioTrack of the timeline completes playback in real time (DSP), the evaluation of the video track lags behind the real time due to frame update delays, resulting in a "synchronization gap" between the two timeline evaluation positions.

3. Solution: Change Update Mode of PlayableDirector

In order to fundamentally prevent this synchronization deviation, change the time source of PlayableDirector that controls Timeline.

Step 1: Change Update Mode to DSP Clock

1. Select the game object in your scene that has the PlayableDirector component attached.
2. Look for the Update Mode property in the Inspector.
3. Change the value from Game Time (default) to `DSP Clock`.

*This will change the master clock for evaluating the progress of the entire timeline from the game frame time to the "number of audio playback samples of the sound card". Even if a video frame drops, the timeline will follow the audio playback position and skip (frame-by-frame), so lip sync etc. will never be out of sync.

Notes on using DSP Clock and C# code for pause processing

If you change Update Mode to DSP Clock, the timeline clock is separated from the game time (Time.timeScale), so "Time.timeScale = 0 A serious side effect is that even if you pause the entire game, the audio and animations in the timeline will continue to run unstoppably.

using UnityEngine;
using UnityEngine.Playables;

public class TimelinePauseManager : MonoBehaviour
{
    public PlayableDirector playableDirector;
    private bool isGamePaused = false;

    void Update()
    {
        // Toggle pausing the game with escape key
 if (Input.GetKeyDown(KeyCode.Escape))
 {
 TogglePause();
 }
 }

 public void TogglePause()
 {
 isGamePaused = !isGamePaused;

 if (isGamePaused)
 {
 // Pause physics and scripts for the entire game
 Time.timeScale = 0f;

 // Explicitly pause the PlayableDirector in DSP Clock mode (required)
 if (playableDirector != null && playableDirector.state == PlayState.Playing)
 {
 playableDirector.Pause();
 Debug.Log("Timeline paused via script.");
 }
 }
 else
 {
 // Resume game time
 Time.timeScale = 1.0f;

 // Resume PlayableDirector
 if (playableDirector != null)
 {
 playableDirector.Play();
 Debug.Log("Timeline resumed.");
 }
 }
 }
}