时间戳

首页 > 解决方案 > 如何使用时间戳注释标记和提取音频

audio - 如何使用时间戳注释标记和提取音频

问题描述

我想标记然后提取音频文件（audio.wav）的某些片段。段的开始和结束时间由另一个文件中的 DateTimeStamp（第一列）和以毫秒为单位的动作持续时间（第三列）给出，注释文件 (annot.csv)：

DateTimeStamp           Action  Duration of action in milliseconds
04/16/20 21:25:36:241   A       502
04/16/20 21:25:36:317   B       2253
04/16/20 21:25:36:734   X       118
04/16/20 21:25:36:837   C       10
04/16/20 21:25:37:537   D       797
04/16/20 21:25:37:606   X       70
04/16/20 21:25:37:874   A       1506
.                       .       .

audio.wav 文件从 annot.csv 文件的第一个 DateTimeStamp 开始。如何使用 annot.csv 文件中的信息从 audio.wav 文件中标记和提取某个片段（例如对应于 Action X）？

我试图用 librosa 和 pyAudioAnalysis 包解决它，但我找不到所需的信息。非常感谢任何帮助。

标签： audiotimestampextractlibrosalabeling

解决方案

这里的关键是计算每个指定片段的开始和结束（在音频样本索引中）。

这可以通过首先将毫秒转换为秒，然后通过乘以音频的采样率来采样索引来完成。

但总的来说，我建议在处理诸如此类的时间序列时使用 Pandas 的 datetime 和 timedelta 功能。下面是一些实现这一点的示例代码：

import io

import pandas
import numpy
import librosa


def read_data(f, date_format):
    df = pandas.read_csv(f, sep=',')

    # Use proper pandas datatypes
    df['Time'] = pandas.to_datetime(df['DateTimeStamp'], format=date_format)
    df['Duration'] = pandas.to_timedelta(df['Duration ms'], unit='ms')
    df = df.drop(columns=['DateTimeStamp', 'Duration ms'])

    # Compute start and end time of each segment
    # audio starts at time of first segment
    first = df['Time'].iloc[0]
    df['Start'] = df['Time'] - first
    df['End'] = df['Start'] + df['Duration']

    return df

def extract_segments(y, sr, segments):
    # compute segment regions in number of samples
    starts = numpy.floor(segments.Start.dt.total_seconds() * sr).astype(int)
    ends = numpy.ceil(segments.End.dt.total_seconds() * sr).astype(int)

    # slice the audio into segments
    for start, end in zip(starts, ends):
        audio_seg = y[start:end]
        print('extracting audio segment:', len(audio_seg), 'samples')

## Reproducible example
data = io.StringIO("""DateTimeStamp,Action,Duration ms
04/16/20 21:25:36:241,A,502
04/16/20 21:25:36:317,B,2253
04/16/20 21:25:36:734,X,118
04/16/20 21:25:36:837,C,10
04/16/20 21:25:37:537,D,797
04/16/20 21:25:37:606,X,70
04/16/20 21:25:37:874,A,1506
""")

segments = read_data(data, date_format="%m/%d/%y %H:%M:%S:%f")
print(segments)

path = librosa.util.example_audio_file()
y, sr = librosa.load(path, sr=16000, duration=10)
extract_segments(y, sr, segments)

应该输出类似

 Action                    Time        Duration           Start             End
0      A 2020-04-16 21:25:36.241 00:00:00.502000        00:00:00 00:00:00.502000
1      B 2020-04-16 21:25:36.317 00:00:02.253000 00:00:00.076000 00:00:02.329000
2      X 2020-04-16 21:25:36.734 00:00:00.118000 00:00:00.493000 00:00:00.611000
3      C 2020-04-16 21:25:36.837 00:00:00.010000 00:00:00.596000 00:00:00.606000
4      D 2020-04-16 21:25:37.537 00:00:00.797000 00:00:01.296000 00:00:02.093000
5      X 2020-04-16 21:25:37.606 00:00:00.070000 00:00:01.365000 00:00:01.435000
6      A 2020-04-16 21:25:37.874 00:00:01.506000 00:00:01.633000 00:00:03.139000
extracting audio segment: 8032 samples
extracting audio segment: 36048 samples
extracting audio segment: 1888 samples
extracting audio segment: 160 samples
extracting audio segment: 12752 samples
extracting audio segment: 1120 samples
extracting audio segment: 24097 samples

推荐阅读