Silverlight Audio

It might not be the current favourite fad using Silverlight, but as my tool of choice, it's just so easy to hack out some interesting ideas. Stimulated by the previous posting about Gibber I resurrected some of the code I'd been playing about with a while back to mess around with some musical ideas and thought it'd be useful to capture the basics of the audio playback framework which can then be built on to generate some neat audio synthesis ideas.

Here's the bare-bones:

First of all, create a basic application and in the MainPage.xaml drop in the following line:

<MediaElement x:Name="media" AutoPlay="False"/>

You then need a new class below which does the bulk of the work:

// PcmStreamSource.cs adapted from 
// http://www.charlespetzold.com/blog/2009/07/Simple-Electronic-Music-Sequencer-for-Silverlight.html
// and from SilverSynth
// http://silversynth.codeplex.com/
 
// the silversynth library is much more tidily packaged and contains a lot of good features 
// this is intended as a much simpler base to get started with some synthesis ideas so 
// strips everything back to a much more minimal base

using System;
using System.Collections.Generic;
using System.IO;
using System.Text;
using System.Windows.Media;

namespace AgSynth
{
    public interface ISampleSource
    {
        short tick();
    }

    public class PcmStreamSource : MediaStreamSource
    {
        public const int SampleRate = 44100;
        public const int BitsPerSample = 16;
        public const int ChannelCount = 1; 

        MediaStreamDescription mediaStreamDescription;
        long startPosition;
        long currentPosition;
        long currentTimeStamp;
        int byteRate;
        short blockAlign;
        MemoryStream memoryStream;
        Dictionary<MediaSampleAttributeKeys, string> emptySampleDict = 
            new Dictionary<MediaSampleAttributeKeys, string>();
        int bufferByteCount;
        const int numSamples = 2 * 256;

        public PcmStreamSource()
        {
            byteRate = SampleRate * ChannelCount * BitsPerSample / 8;
            blockAlign = (short)(BitsPerSample * ChannelCount / 8);
            memoryStream = new MemoryStream();
            bufferByteCount = ChannelCount * BitsPerSample / 8 * numSamples;
            this.BufferLength = 600;
        }

        public int BufferLength
        {
            get { return this.AudioBufferLength; }
            set { this.AudioBufferLength = value; }
        }

        public ISampleSource Input
        {
            get;
            set;
        }

        protected override void OpenMediaAsync()
        {
            startPosition = currentPosition = 0;

            Dictionary<MediaStreamAttributeKeys, string> streamAttributes = new Dictionary<MediaStreamAttributeKeys, string>();
            Dictionary<MediaSourceAttributesKeys, string> sourceAttributes = new Dictionary<MediaSourceAttributesKeys, string>();
            List<MediaStreamDescription> availableStreams = new List<MediaStreamDescription>();

            string format = "";
            format += ToLittleEndianString(string.Format("{0:X4}", 1)); //PCM
            format += ToLittleEndianString(string.Format("{0:X4}", ChannelCount)); 
            format += ToLittleEndianString(string.Format("{0:X8}", SampleRate));
            format += ToLittleEndianString(string.Format("{0:X8}", byteRate));
            format += ToLittleEndianString(string.Format("{0:X4}", blockAlign));
            format += ToLittleEndianString(string.Format("{0:X4}", BitsPerSample));
            format += ToLittleEndianString(string.Format("{0:X4}", 0));

            streamAttributes[MediaStreamAttributeKeys.CodecPrivateData] = format;


            mediaStreamDescription = new MediaStreamDescription(MediaStreamType.Audio, streamAttributes);
            availableStreams.Add(mediaStreamDescription);
            sourceAttributes[MediaSourceAttributesKeys.Duration] = "0"; 
            sourceAttributes[MediaSourceAttributesKeys.CanSeek] = "false";
            ReportOpenMediaCompleted(sourceAttributes, availableStreams);
        }

        protected override void GetSampleAsync(MediaStreamType mediaStreamType)
        {


            for (int i = 0; i < numSamples; i++)
            {
                short sample = 0;
                if (this.Input != null)
                {
                    sample = this.Input.tick();
                }

                for (int c = 0; c < ChannelCount; c++)
                {
                    //channel
                    memoryStream.WriteByte((byte)(sample & 0xFF));
                    memoryStream.WriteByte((byte)(sample >> 8));
                }
            }

            MediaStreamSample mediaStreamSample =
                new MediaStreamSample(mediaStreamDescription, memoryStream, currentPosition,
                                      bufferByteCount, currentTimeStamp, emptySampleDict);

            currentTimeStamp += bufferByteCount * 10000000L / byteRate;
            currentPosition += bufferByteCount;

            ReportGetSampleCompleted(mediaStreamSample);
        }

        protected override void SeekAsync(long seekToTime)
        {
            this.ReportSeekCompleted(seekToTime);
        }

        protected override void CloseMedia()
        {
            startPosition = currentPosition = 0;
            mediaStreamDescription = null;
        }

        protected override void GetDiagnosticAsync(MediaStreamSourceDiagnosticKind diagnosticKind)
        {
            throw new NotImplementedException();
        }

        protected override void SwitchMediaStreamAsync(MediaStreamDescription mediaStreamDescription)
        {
            throw new NotImplementedException();
        }

        string ToLittleEndianString(string bigEndianString)
        {
            StringBuilder builder = new StringBuilder();

            for (int i = 0; i < bigEndianString.Length; i += 2)
            {
                builder.Insert(0, bigEndianString.Substring(i, 2));
            }

            return builder.ToString();
        }


    }
}

I've then stripped down the rest of the framework that is used in the referenced code from Petzold and Mike Hodnick (kindohm).

We can now build the simplest kind of synth generator to make a middle-C (C4) sine-wave signal:

using System;
using System.Net;
using System.Windows;
using System.Windows.Controls;
using System.Windows.Documents;
using System.Windows.Ink;
using System.Windows.Input;
using System.Windows.Media;
using System.Windows.Media.Animation;
using System.Windows.Shapes;

namespace AgSynth
{
    public class Generator : ISampleSource
    {
        public virtual short tick()
        {
            return 0;
        }

    }


    public class Sine : Generator
    {
        int n = 0;
        const double C4 = 261.6; // middle C in Hz
        double f = C4; // Hz

        double vol = 1.0;

        public override short tick()
        {
            short sample = (short)(short.MaxValue * vol * Math.Sin(f * (2 * Math.PI) * n / PcmStreamSource.SampleRate));
            n++;

            return sample;
        }

    }
}

Effectively what happens here is each tick a new sample is generated for the desired signal. Please take a look at SilverSynth to see how this basic framework can be built on to make more advanced processing. There's a great example of this done very nicely here.

All we need to do is wire this up to the MainPage now. So, firstly create the PcmStreamSource object:

PcmStreamSource source = new PcmStreamSource();

Set the MediaSource when the control is loaded:

private void UserControl_Loaded(object sender, RoutedEventArgs e)
{
    media.SetSource(source);
}

Don't forget to make sure you have the loaded event in the XAML.

And last but not least, the fun bit, make a button to start (like a play-button) and in the event create the generator object, assign it to the MediaElement and play away:

Sine sine = new Sine();

private void Button_Click(object sender, RoutedEventArgs e)
{
    source.Input = sine;
    media.Play();
}

Not hugely exciting to blast out a single sine-wave frequency, but as Fourier analysis shows all music can be made from the assembly of suitable sine-waves, so it's a start!

Note - one thing I did notice in this was that there is a gotcha here that had me bashing my head a little bit. I was getting the OpenMediaAsync called but not the GetSampleAsync. I think it is also the cause of some similar problems that have been reported. I tracked my problem down to the annoying issue that if you put the lines in Button_Click into the UserControl_Loaded event, e.g. to kick-off playing at the start, the initialisation does not seem to happen properly.