渲染 Windows Phone 设备捕获的音频

引言

这个小型应用程序使用 XNA 框架在屏幕上显示 Windows Phone 设备麦克风捕获的音频，并将其显示为连续的波形。

可以在 Windows Phone Marketplace 中找到应用程序的一个稍作修改的版本（触摸屏幕时改变颜色）。

背景

要捕获 Windows Phone 设备上的音频，您需要一个指向默认麦克风的实例 (Microphone.Default)，决定您希望采样频率（使用 BufferDuration 属性），并挂钩 BufferReady 事件。然后，您可以通过 Start() 和 Stop() 方法来控制捕获。

麦克风以固定的 16000 Hz 速率为您提供采样，即每秒 16000 个采样。有一个 SampleRate 属性可以告诉您这个值。根据 采样定理，这意味着您将无法捕获高于 8000 Hz 的音频（而不失真）。

您对 BufferDuration 属性的值选择也有限制；它必须介于 0.1 到 1 秒（100 - 1000 毫秒）之间，步长为 10 毫秒。这意味着您必须选择 100、110、120、...、990、1000 毫秒中的一个值。

当麦克风事件 BufferReady 触发时，您应该调用 microphone.GetData(myBuffer) 方法，以便将采样从麦克风的内部缓冲区复制到您自己的缓冲区。录制的音频以字节数组的形式出现，但由于采样实际上是带符号的 16 位整数（即范围在 -32768 ... 32767 之间的整数），您可能需要在处理它们之前进行一些转换。 

使用代码

这个应用程序的工作方式是维护一个固定数量的窄图像，这里称为“（图像）切片”，并将其排列成一个链表。图像在屏幕上渲染，并从右向左平滑移动。当最左边的切片移出屏幕时，它会被移动到最右边（仍在屏幕外），以创造出无限图像的错觉。

每个切片都包含一个麦克风缓冲区内容的渲染样本。当麦克风机制填充缓冲区时，最右边的切片（屏幕外）将用这些新样本进行渲染，并开始向屏幕内移动。

切片在屏幕上移动的速度与缓冲区的持续时间相关，使得切片在麦克风捕获下一个缓冲区的时间内总共移动“一个切片宽度”。

由于捕获的音频缓冲区在接收到后会立即渲染成纹理上的图形，因此没有理由保留任何旧的缓冲区数据。因此，应用程序只在内存中保留一个缓冲区，并对其进行反复利用。

每次麦克风缓冲区准备就绪时，都会设置一个标志。由于 BufferReady 事件在主线程上触发，因此不需要任何锁机制。

在 XNA 应用程序的 Update() 方法中，会检查标志以确定是否有新数据到达，如果有，则绘制排队的切片。在 Draw() 方法中，切片会随着时间的推移绘制在屏幕上并稍作移动。 

这里是对主 Game 类结构的描述。

一些常量： 

// The size of the screen.
private const int LandscapeWidth = 800;
private const int LandscapeHeight = 480;
 
// The number of milliseconds per time slice.
private const int SliceMilliseconds = 100;

关于麦克风和捕获数据的字段： 

// The microphone and the sample data.
private readonly Microphone microphone;
private readonly byte[] microphoneData;

// The time it takes for a sample to pass over the screen.
private readonly TimeSpan screenMilliseconds = TimeSpan.FromSeconds(5);

选择一种几乎透明的颜色（四种参数中的最后一个；它是颜色的红色、绿色、蓝色和 Alpha 分量）。 原因是许多样本会绘制在彼此之上，而将每个单独的样本保持近乎透明会产生有趣的视觉效果。 

// The color of the samples.
private readonly Color sampleColor = new Color(0.4f, 0.9f, 0.2f, 0.07f);

绘图类。白色像素纹理负责所有绘图工作。 

// The sprite batch and a white single-pixel texture for drawing.
private SpriteBatch spriteBatch;
private Texture2D whitePixelTexture;

每个图像切片的大小。 

// The size in pixels of one time slice.
private int imageSliceWidth;
private int imageSliceHeight;

无需保留对链表本身的引用；只需第一个和最后一个节点。这些节点保留对其邻居的引用。 currentImageSlice 是下次要绘制的切片。 

// The first, last and current image slices.
private LinkedListNode<RenderTarget2D> firstImageSlice;
private LinkedListNode<RenderTarget2D> lastImageSlice;
private LinkedListNode<RenderTarget2D> currentImageSlice;

切片在屏幕上移动的速度。 

// The current speed of the samples.
private float pixelsPerSeconds;

为了知道当前样本应该移动多远，应用程序必须跟踪它们 出现的时间。 

// The time in seconds when the current microphone data appeared.
private float microphoneDataAppearedAtSeconds;

 指示 Update()-方法有新数据需要处理的信号。 

// A flag telling whether new data has been read.
private bool hasNewMicrophoneData;

 每像素采样密度。 

// The number of samples squeezed into the width of one pixel.
private int samplesPerPixel;

这是构造函数。在其中，图形模式被设置，麦克风被连接起来 并被要求开始监听。 

public Waveform()
{
    // Set the screen mode.
    new GraphicsDeviceManager(this)
    {
        PreferredBackBufferWidth = LandscapeWidth,
        PreferredBackBufferHeight = LandscapeHeight,
        IsFullScreen = true,
        SupportedOrientations =
            DisplayOrientation.Portrait |
            DisplayOrientation.LandscapeLeft |
            DisplayOrientation.LandscapeRight
    };

    // Standard setup.
    Content.RootDirectory = "Content";
    TargetElapsedTime = TimeSpan.FromTicks(333333);
    InactiveSleepTime = TimeSpan.FromSeconds(1);

    // Refer to the default microphone and hook the BufferReady-event.
    microphone = Microphone.Default;
    microphone.BufferReady += MicrophoneBufferReady;

    // Set the buffer duration to the length of one slice.
    microphone.BufferDuration = TimeSpan.FromMilliseconds(SliceMilliseconds);

    // Calculate the size in bytes of the sound buffer and create the byte array.
    var microphoneDataLength = microphone.GetSampleSizeInBytes(microphone.BufferDuration);
    microphoneData = new byte[microphoneDataLength];

    // Start listening.
    microphone.Start();
}

在 XNA 的 LoadContent 中，实际上没有任何内容被加载，因为应用程序不依赖于任何预先绘制的图像。 SpriteBatch 被创建，白色像素纹理被生成，图像切片被初始化（为黑色图像）。 

protected override void LoadContent()
{
    // Create a SpriteBatch for drawing.
    spriteBatch = new SpriteBatch(GraphicsDevice);

    // Create a 1x1 texture containing a white pixel.
    whitePixelTexture = new Texture2D(GraphicsDevice, 1, 1);
    var white = new[] { Color.White };
    whitePixelTexture.SetData(white);

    // Create the image slices.
    CreateSliceImages();
}

CreateSliceImages 计算了覆盖整个屏幕所需的切片数量（再加上两个，以便有移动的空间）。在方法末尾，调用了常规的 RenderSamples 方法来初始化所有图像。由于还没有数据（所有样本均为零），它将生成黑色图像。 

private void CreateSliceImages()
{
    // Calculate how many slices that fits the screen (rounding upwards).
    var imageSlicesOnScreenCount = (int)Math.Ceiling(screenMilliseconds.TotalMilliseconds / SliceMilliseconds);

    // Calculate the width of each slice.
    imageSliceWidth = (int)Math.Ceiling((float)LandscapeWidth / imageSlicesOnScreenCount);

    // Set the height of each slice to the largest screen size
    // (this way the full height is utilized in Portrait mode without stretching)
    imageSliceHeight = LandscapeWidth;

    // Create a linked list with the required number of slices, plus two
    // so that there's room for scrolling off-screen a bit.
    var imageSlices = new LinkedList<RenderTarget2D>();
    for (var i = 0; i < imageSlicesOnScreenCount + 2; i++)
    {
        var imageSlice = new RenderTarget2D(GraphicsDevice, imageSliceWidth, imageSliceHeight);
        imageSlices.AddLast(imageSlice);
    }

    // Reference the first, last and current slice.
    firstImageSlice = imageSlices.First;
    lastImageSlice = imageSlices.Last;
    currentImageSlice = imageSlices.Last;

    // Calculate the speed of the pixels for an image slice.
    pixelsPerSeconds = imageSliceWidth / (SliceMilliseconds / 1000f);

    // Since the byte-array buffer really holds 16-bit samples, the actual
    // number of samples in one buffer is the number of bytes divided by two.
    var sampleCount = microphoneData.Length / 2;

    // Calculate how many samples that should be squeezed in per pixel (width).
    samplesPerPixel = (int)Math.Ceiling((float)sampleCount / imageSliceWidth);

    // Iterate through all the image slices and render with the empty microphone buffer.
    var slice = firstImageSlice;
    while (slice != null)
    {
        RenderSamples(slice.Value);
        slice = slice.Next;
    }
}

protected override void UnloadContent()
{
    // Dispose the SpriteBatch.
    spriteBatch.Dispose();

    // Dispose the white pixel.
    whitePixelTexture.Dispose();

    // Dispose all the image slices.
    var slice = firstImageSlice;
    while (slice != null)
    {
        slice.Value.Dispose();
        slice = slice.Next;
    }
}

麦克风的 BufferReady 事件的事件处理程序。它从麦克风缓冲区复制数据并 引发新数据已到达的标志。 

private void MicrophoneBufferReady(object sender, EventArgs e)
{
    // New microphone data can now be fetched from its buffer.

    // Copy the samples from the microphone buffer to our buffer.
    microphone.GetData(microphoneData);

    // Raise the flag that new data has come.
    hasNewMicrophoneData = true;
}

XNA 的 Update 方法 检查手机的 Back 按钮，看是否到了退出的时候。之后，它检查标志以查看是否有新数据被录制。如果有，则通过调用 RenderSamles 方法来渲染新样本。 

 protected override void Update(GameTime gameTime)
{
    // Exit the app if the user presses the back-button.
    if (GamePad.GetState(PlayerIndex.One).Buttons.Back == ButtonState.Pressed)
    {
        Exit();
    }

    // If new data has been captured, a new slice should be drawn.
    if (hasNewMicrophoneData)
    {
        // Reset the flag.
        hasNewMicrophoneData = false;

        // Express the current point in time as "seconds passed since start of app".
        var currentSeconds = (float)gameTime.TotalGameTime.TotalSeconds;

        // Remember the current time as "when the mic data appeared".
        microphoneDataAppearedAtSeconds = currentSeconds;

        // Render the new samples on the current image slice.
        RenderSamples(currentImageSlice.Value);

        // Select the next image slice as the new "current".
        currentImageSlice = currentImageSlice.Next ?? firstImageSlice;
    }

    base.Update(gameTime);
}

XNA 的 Draw 方法负责绘制渲染后的切片。它处理两种屏幕方向模式；横向和纵向，通过相应地缩放图像。如果处于横向模式，图像的高度会被压缩；如果处于纵向模式，图像的宽度会被压缩。 

所有设置完成后，该方法会逐一迭代图像并在屏幕上渲染它们，并在 X 轴上稍微调整，以弥补已过去的时间。 

protected override void Draw(GameTime gameTime)
{
    // Clear the device. (Actually unnecessary since the whole screen will be painted below.)
    GraphicsDevice.Clear(Color.Black);

    // Calculate the "screen width-scale", to allow the app to be drawn both in
    // Landscape and Portrait mode.
    // In Landscape mode, the screenWidthScale will be 1.0 (i.e. original scale)
    // In Portrait mode, the screen must be squeezed.
    var screenWidthScale = (float)GraphicsDevice.Viewport.Width / LandscapeWidth;

    // Calculate the scaled width of one slice.
    var scaledWidth = (int)Math.Ceiling(imageSliceWidth * screenWidthScale);

    // Express the current point in time as "seconds passed since start of app".
    var currentSeconds = (float)gameTime.TotalGameTime.TotalSeconds;

    // Calculate how many seconds that has passed since the current microphone data was captured.
    var secondsPassed = currentSeconds - microphoneDataAppearedAtSeconds;

    // For a smooth move of the pixels, calculate the offset of the current microphone data
    // (where the offset is zero at the time of the new data arrived, and then growing up
    // one full width of a slice.
    var drawOffsetX = secondsPassed * pixelsPerSeconds;

    // Since it is not certain that the next microphone data will come before the current
    // slice has moved its full distance, the offset needs to be truncated so it doesn't
    // move too far.
    if (drawOffsetX > scaledWidth)
    {
        drawOffsetX = scaledWidth;
    }

    try
    {
        // Start draw the slices
        spriteBatch.Begin();

        // Start with one slice before the current one, wrap if necessary.
        var imageSlice = currentImageSlice.Previous ?? lastImageSlice;

        // Prepare the rectangle to draw within, starting with the newest
        // slice far to the right of the screen (a bit outside, even).
        var destinationRectangle = new Rectangle(
            (int)(GraphicsDevice.Viewport.Width + scaledWidth - drawOffsetX),
            0,
            scaledWidth,
            GraphicsDevice.Viewport.Height);

        // Draw the slices in the linked list one by one from the right
        // to the left (from the newest sample to the oldest)
        // and move the destinationRectangle one slice-width at a time
        // until the full screen is covered.
        while (destinationRectangle.X > -scaledWidth)
        {
            // Draw the current image slice.
            spriteBatch.Draw(imageSlice.Value, destinationRectangle, Color.White);

            // Move the destinationRectangle one step to the left.
            destinationRectangle.X -= scaledWidth;

            // Select the previous image slice to draw next time, wrap if necessary.
            imageSlice = imageSlice.Previous ?? lastImageSlice;
        }
    }
    finally
    {
        // Drawing done.
        spriteBatch.End();
    }

    base.Draw(gameTime);
}

private void RenderSamples(RenderTarget2D target)
{
    try
    {
        // Redirect the drawing to the given target.
        GraphicsDevice.SetRenderTarget(target);

        // Clear the target slice.
        GraphicsDevice.Clear(Color.Black);

        // Begin to draw. Use Additive for an interesting effect.
        spriteBatch.Begin(SpriteSortMode.Deferred, BlendState.Additive);

        // The X-variable points out to which column of pixels to
        // draw on.
        var x = 0;

        // Since the byte-array buffer really holds 16-bit samples, the actual
        // number of samples in one buffer is the number of bytes divided by two.
        var sampleCount = microphoneData.Length / 2;

        // The index of the current sample in the microphone buffer.
        var sampleIndex = 0;

        // The vertical mid point of the image (the Y-position
        // of a zero-sample and the height of the loudest sample).
        var halfHeight = imageSliceHeight / 2;

        // The maximum number of a 16-bit signed integer.
        // Dividing a signed 16-bit integer (the range -32768..32767)
        // by this value will give a value in the range of -1 (inclusive) to 1 (exclusive).
        const float SampleFactor = 32768f;

        // Iterate through the samples and render them on the image.
        for (var i = 0; i < sampleCount; i++)
        {
            // Increment the X-coordinate each time 'samplesPerPixel' pixels
            // has been drawn.
            if ((i > 0) && ((i % samplesPerPixel) == 0))
            {
                x++;
            }

            // Convert the current sample (16-bit value) from the byte-array to a
            // floating point value in the range of -1 (inclusive) to 1 (exclusive).
            var sampleValue = BitConverter.ToInt16(microphoneData, sampleIndex) / SampleFactor;

            // Scale the sampleValue to its corresponding height in pixels.
            var sampleHeight = (int)Math.Abs(sampleValue * halfHeight);

            // The top of the column of pixels.
            // A positive sample should be drawn from the center and upwards,
            // and a negative sample from the center and downwards.
            // Since a rectangle is used to describe the "pixel column", the
            // top must be modified depending on the sign of the sample (positive/negative).
            var y = (sampleValue < 0)
                ? halfHeight
                : halfHeight - sampleHeight;

            // Create the 1 pixel wide rectangle corresponding to the sample.
            var destinationRectangle = new Rectangle(x, y, 1, sampleHeight);

            // Draw using the white pixel (stretching it to fill the rectangle).
            spriteBatch.Draw(
                whitePixelTexture,
                destinationRectangle,
                sampleColor);

            // Step the two bytes-sample.
            sampleIndex += 2;
        }
    }
    finally
    {
        // Drawing done.
        spriteBatch.End();

        // Restore the normal rendering target (the screen).
        GraphicsDevice.SetRenderTarget(null);
    }
}