Wednesday, 9 July 2014

Bringing the Delay classes closer to STK

I'm pretty pleased with where things are going with the Silverlight KS dabbling and want to continue putting in some more improvements. Before taking this further as I'm using STK as a basis if only for a commentary reference it seems sensible to harmonise some of the terms a little more with it so that going forwards I can keep the description of the code at least using familiar terms. So, this morning I've added a generic Filter class and then derived the Delay classes from it and changed some of the terms to match those used in STK. Here is a run through of the changes: New Filter class added as below. Some of the attributes and methods don't get used much at the moment, but we'll use them in the building blocks to put some more of the filters later on:

Filter.cs 

using System;
using System.Net;
using System.Collections.Generic;

namespace AgSynth
{
    public class Filter
    {
        protected double gain = 1.0;
        protected double last = 0.0;
        protected List<double> a = new List<double>();
        protected List<double> b = new List<double>();

        protected List<double> outputs = new List<double>();
        protected List<double> inputs = new List<double>();


        public double Gain
        {
            set
            {
                gain = value;
            }

            get
            {
                return gain;
            }
        }

        public virtual double phaseDelay(double frequency)
        {
            if (frequency <= 0.0 || frequency > 0.5 * PcmStreamSource.SampleRate)
            {
                throw new Exception();
            }

            double omegaT = 2 * Math.PI * frequency / PcmStreamSource.SampleRate;
            double reala = 0.0;
            double imaga = 0.0;
            double realb = 0.0;
            double imagb = 0.0;

            for (int i = 0; i < a.Count; i++)
            {
                reala += a[i] * Math.Cos(i * omegaT);
                imaga -= a[i] * Math.Sin(i * omegaT);
            }

            for (int i = 0; i < b.Count; i++)
            {
                realb += b[i] * Math.Cos(i * omegaT);
                imagb -= b[i] * Math.Sin(i * omegaT);
            }


            reala *= gain;
            imaga *= gain;
            realb *= gain;
            imagb *= gain;

            double phase = Math.Atan2(imagb, realb) - Math.Atan2(imaga, reala);
            phase = -phase % (2 * Math.PI);

            return phase / omegaT;
        }



        public double Last
        {
            get
            {
                return last;
            }
        }

        public virtual double tick(double input)
        {
            return 0.0;
        }
    }
}

The majority of this is the phaseDelay method which is calculating the phase delay for the filter at a particular frequency. We're not using that at all at the moment, but will become useful a bit later. I'll maybe blog about that separately if I get a chance and keep this just to the code changes.

The STK contains Delay, DelayL and DelayA classes which follow the code so far. However, as I wanted to use Delay as the attribute for the Delay instead of Length as I've been using (I don't like using _xxx for attributes, it just makes code messy IMHO), it means I could not use Delay for the class name.I decided instead to use DelayN for the integer delay following the pattern used in SuperCollider.

DelayN
This replaces the previous Delay class with the following major changes:

  • get has been changed to tapOut
  • set has been changed to tapIn
  • Length has been changed to Delay and made to set a double. For DelayN this now truncates the value to make the delay an integer length
  • the tick operation has been added which calculates the next sample from the filter and pushes the next sample in which is much closer to how STK works
This is how it now looks:


using System;
using System.Net;


namespace AgSynth
{
    public class DelayN : Filter
    {
        protected const int MAXLENGTH = 1000;
        private double[] buffer = new double[MAXLENGTH + 1];

        private int idx = 0;
        protected int length = 0;

        public DelayN()
        {
            for (int i = 0; i < MAXLENGTH + 1; i++)
            {
                buffer[i] = 0;
            } 
        }

        public void increment()
        {
            lock (buffer)
            {
                idx = (idx + 1 > length) ? 0 : idx + 1;
            }
        }

        public override double tick(double input)
        {
            double last = pop();
            push(input * gain);

            return last;
        }

        public virtual double pop()
        {
            double value = tapOut(length);
            

            return value;
        }

        // increment now on pushing into delay
        public void push(double value)
        {
            tapIn(0, value);
            increment();
        }

        public double tapOut(int index)
        {
            double result = 0.0;

            if (buffer != null)
            {
                lock (buffer)
                {
                    int pos = (idx + index >= length) ? idx + index - length : index + idx;
                    result = buffer[pos];
                }
            }

            return result;
        }

        public void tapIn(int index, double value)
        {
            if (buffer != null)
            {
                lock (buffer)
                {
                    int pos = (idx + index >= length) ? idx + index - length : index + idx;
                    buffer[pos] = value;
                }
            }
        }

        public virtual double Delay
        {
            set
            {
                length = Helpers.Truncate(value);

                if (length < MAXLENGTH)
                {
                    lock (buffer)
                    {
                        double[] newbuffer = new double[MAXLENGTH];

                        for (int i = 0; i < length; i++)
                        {
                            newbuffer[i] = tapOut(i);
                        }

                        buffer = newbuffer;
                        idx = 0;
                    }
                }
            }

            get
            {
                return length;
            }
        }

    }
}

And on to the two fractional delays:

DelayL
Only needs some relatively small tweaks to the change in new method names:


using System;
using System.Net;


namespace AgSynth
{
    public class DelayL : DelayN
    {

        private double alpha = 0;

        public DelayL() : base()
        {
        }

        double xdel = 0;

        public override double pop()
        {
            double x = tapOut(length);

            double y = ((1-alpha) * x) + (alpha*xdel);
            xdel = x;

            last = y;

            return y;
        }

        public override double Delay
        {
            set
            {
                base.Delay = value;
                alpha = value - length;
            }
        }

    }
}

DelayA
And pretty much similar modifications for DelayA too:


using System;
using System.Net;


namespace AgSynth
{
    public class DelayA : DelayN
    {
        private double alpha = 0;
        private double coeff = 0;

        public DelayA() : base()
        {
        }

 
        double ydel = 0;
        double xdel = 0;

        public override double pop()
        {
            double x = tapOut(length);

            double y = (-coeff * ydel) + xdel + (coeff * x);

            xdel = x;
            ydel = y;

            last = y;

            return y;
        }

        public override double Delay
        {
            set
            {
                int l = Helpers.Truncate(value);
                alpha = 1.0 + value - l;

                // optimal allpass range is between  0.5 - 1.5 to achieve flattest phase delay response
                if (alpha < 0.5)
                {
                    l++;
                    alpha += 1.0;
                }

                // coefficient for filter
                coeff = (1.0 - alpha) / (1.0 + alpha);

                base.Delay = l;
            }
        }

    }
}


And given our current implementation of the Karp Generator no changes are needed there. et-voila, we're much closer to talking like STK.

Tuesday, 8 July 2014

Plucking some Guitar with Silverlight

Following on from the improvements to the basic KS algorithm covered in my previous two posts to address the fine-tuning this only really makes sense listening to something more representative, so I wanted to add in some code that would easily allow putting together some basic guitar patterns.

Firstly, building on the earlier simple Theory class which contained the various different pitch frequencies I've made a very simple Fretboard class that returns the correct pitch mapping for a traditionally tuned (EADGBE) 6-string guitar:

    public class Fretboard
    {
        static double[,] fretboard = new double[6, 16] 
        {
            //1
            { 
                Theory.E4, Theory.F4, Theory.Fs4, Theory.G4, Theory.Gs4, Theory.A4, Theory.As4, Theory.B5, Theory.C5,
                Theory.Cs5, Theory.D5, Theory.Ds5, Theory.E5, Theory.F5, Theory.Fs5, Theory.G5
            
            },

            //2
            { 
                Theory.B3, Theory.C4, Theory.Cs4, Theory.D4, Theory.Ds4, Theory.E4, Theory.F4, Theory.Fs4, Theory.G4 ,
                Theory.Gs4, Theory.A4, Theory.As4, Theory.B4, Theory.C5, Theory.Cs5, Theory.D5
            },

            //3
            { 
                Theory.G3, Theory.Gs3, Theory.A3, Theory.As3, Theory.B3, Theory.C4, Theory.Cs4, Theory.D4, Theory.Ds4,
                Theory.E4, Theory.F4, Theory.Fs4, Theory.G4, Theory.Gs4, Theory.A4, Theory.As4
            },
            
            //4
            { 
                Theory.D3, Theory.Ds3, Theory.E3, Theory.F3, Theory.Fs3, Theory.G3, Theory.Gs3, Theory.A3, Theory.As3,
                Theory.B3, Theory.C4, Theory.Cs4, Theory.D4, Theory.Ds4, Theory.E4, Theory.F4
            },
            
            //5
            { 
                Theory.A2, Theory.As2, Theory.B2, Theory.C3, Theory.Cs3, Theory.D4, Theory.Ds4, Theory.E4, Theory.F4,
                Theory.Fs4, Theory.G4, Theory.Gs4, Theory.A4, Theory.As4, Theory.B4, Theory.C5
            },
            
            //6
            { 
                Theory.E2, Theory.F2, Theory.Fs2, Theory.G2, Theory.Gs2, Theory.A2, Theory.As2, Theory.B2, Theory.C3,
                Theory.Cs3, Theory.D3, Theory.Ds3, Theory.E3, Theory.F3, Theory.Fs3, Theory.G3
            }
        };

        static public double Pitch(int s, int f)
        {
            double result = 0;

            if (s <= 6 && f <= 15)
            {
                result = fretboard[s, f];
            }

            return result;
        }
    }

I've also added a new Note class which brings together the pitch and duration:

        public class Note
        {
            public double pitch;
            public double duration;

            public Note(double p, double d)
            {
                pitch = p;
                duration = d;
            }
        }

And the PlayMusic function has been improved now to use this to use the new Note and Fretboard classes.


        private void PlayMusic()
        {
            double bpm = 120;

            double barduration = 60 * 1000 * 4 / bpm; // in ms - assuming 4/4

            beat = barduration / 4;


            List<Note> tune = new List<Note>() 
            {

                new Note(Fretboard(0,1), 1.0),
                new Note(Fretboard(0,2), 1.0),
                new Note(Fretboard(0,3), 1.0),
                new Note(Fretboard(0,4), 1.0),
            };

            while (true)
            {
                foreach (Note n in tune)
                {
                    karp.pitch(n.pitch);
                    karp.pluck();
                    Thread.Sleep((int)(beat * n.duration));
                }
            }
        }

All good, but so far, not so interesting, so lets take some music - Silent Night - here in score and tab:


Which looks now like this in code:


            List<Note> tune = new List<Note>() 
            {
                new Note(Fretboard.Pitch(2,0), 0.75),
                new Note(Fretboard.Pitch(2,2), 0.25),
                new Note(Fretboard.Pitch(2,0), 0.5),
                new Note(Fretboard.Pitch(3,2), 1.5),

                new Note(Fretboard.Pitch(2,0), 0.75),
                new Note(Fretboard.Pitch(2,2), 0.25),
                new Note(Fretboard.Pitch(2,0), 0.5),
                new Note(Fretboard.Pitch(3,2), 1.5),

                new Note(Fretboard.Pitch(1,3), 1),
                new Note(Fretboard.Pitch(1,3), 0.5),
                new Note(Fretboard.Pitch(1,0), 1.5),

                new Note(Fretboard.Pitch(1,1), 1),
                new Note(Fretboard.Pitch(1,1), 0.5),
                new Note(Fretboard.Pitch(2,0), 1.5),

                new Note(Fretboard.Pitch(2,2), 0.5),
                new Note(Fretboard.Pitch(2,2), 0.5),
                new Note(Fretboard.Pitch(2,2), 0.5),
                new Note(Fretboard.Pitch(1,1), 0.75),
                new Note(Fretboard.Pitch(1,0), 0.25),
                new Note(Fretboard.Pitch(2,2), 0.5),

                new Note(Fretboard.Pitch(2,0), 0.75),
                new Note(Fretboard.Pitch(2,2), 0.25),
                new Note(Fretboard.Pitch(2,0), 0.5),
                new Note(Fretboard.Pitch(3,2), 1.5),

                new Note(Fretboard.Pitch(2,2), 1),
                new Note(Fretboard.Pitch(2,2), 0.5),
                new Note(Fretboard.Pitch(1,1), 0.75),
                new Note(Fretboard.Pitch(1,1), 0.25),
                new Note(Fretboard.Pitch(2,2), 0.5),

                new Note(Fretboard.Pitch(2,0), 0.75),
                new Note(Fretboard.Pitch(2,2), 0.25),
                new Note(Fretboard.Pitch(2,0), 0.5),
                new Note(Fretboard.Pitch(3,2), 1.5),

                new Note(Fretboard.Pitch(1,3), 0.5),
                new Note(Fretboard.Pitch(1,3), 0.5),
                new Note(Fretboard.Pitch(1,3), 0.5),
                new Note(Fretboard.Pitch(0,1), 0.75),
                new Note(Fretboard.Pitch(1,3), 0.25),
                new Note(Fretboard.Pitch(1,0), 0.5),

                new Note(Fretboard.Pitch(1,1), 1.5),
                new Note(Fretboard.Pitch(0,0), 1.5),

                new Note(Fretboard.Pitch(1,1), 0.75),
                new Note(Fretboard.Pitch(2,0), 0.25),
                new Note(Fretboard.Pitch(3,2), 0.5),
                new Note(Fretboard.Pitch(2,0), 0.75),
                new Note(Fretboard.Pitch(3,3), 0.25),
                new Note(Fretboard.Pitch(3,0), 0.5),

                new Note(Fretboard.Pitch(4,3), 1.5),
                new Note(0, 1.5),

                new Note(0, 3)
            };

A couple of little changes need to be made to the Karp class to accomodate DelayA and DelayL:

DelayL delay = new DelayL();


        public void pitch(double p)
        {
            if (p == 0)
            {
                delay.Length = 0;
            }
            else
            {
                Fo = p; // frequency
                double Fs = PcmStreamSource.SampleRate;

                double No = Fs / Fo;
                
                length = Helpers.Truncate(No - 0.5);
                excite = Helpers.Truncate(length * 0.95);

                delay.Length = No - 0.5;
            }
        }

And, remember to remove the delay.increment from after the delay.push as that has now been combined into that operation.

The other little tweak is that the decay value should be set to 0.9999 or even 0.99999 to make the notes ring out longer.

And it sounds like this..... a little buzzy, but fun!



Karplus Strong Fine Tuning in Silverlight

Following on from my previous post which had a basic implementation of KS in Silverlight I wanted to take this a bit further and address some of the problems in the simple implementation. Firstly as is well referenced, the problem with the basic KS implementation using an integer length delay line is that as the frequencies get higher there is an increasing problem with the pitch of the notes being out of tune.

There are two typical ways of addressing the integer delay line problem:
  • Linear Interpolation
  • All-pass filter
Linear Interpolation
A linear interpolation delay filter can be made by simply taking a portion equivalent to the fractional delay between the two last samples. Having already taken the lead from the STK implementation, we're going to continue to implement this using a similar class called DelayL (header, source).

As referenced above a Linearly Interpolated delay line (1st order FIR) has the following block diagram:






With a transfer function and difference equation for the fractional delay part of:






Where the delay fractional delay (n) above we will reference as alpha in the code implementation.

For more information on Interpolated Delay lines take a look here.

Firstly a few tidying up operations need to be made to the Delay class:

protected const int MAXLENGTH = 1000;

protected int length = 0;

public virtual double pop()

In addition, the increment operation has also been moved into the push operation:


        // increment now on pushing into delay
        public void push(double value)
        {
            set(0, value);
            increment();
        }


DelayL.cs
The new DelayL class inherits from Delay and modifies the pop operation to implement the Linear Interpolation algorithm and the Length setter now handles setting a non-integer value.

using System;
using System.Net;


namespace AgSynth
{
    public class DelayL : Delay
    {

        private double alpha = 0;

        public DelayL() : base()
        {
        }

        double xdel = 0;

        public override double pop()
        {
            double x = get(length);

            double y = ((1-alpha) * x) + (alpha*xdel);
            xdel = x;

            return y;
        }


        public double Length
        {
            set
            {
                base.Length = Helpers.Truncate(value);
                alpha = value - length;
            }
        }

    }
}

This all works very nicely, and sounds pretty good, but the unfortunate effect of Linear Interpolation is that it adjusts the pitch and can sharpen and flatten some of the notes.

Allpass Filter
Again taking the lead from STK to implement a First-Order All-pass Interpolation we can follow the implementation in the class DelayA (header, source).

As referenced above a First-Order All-pass Interpolation has the following block diagram:





With a transfer function and difference equation of the fractional part of:






In this case to calculate the delay, the parameters need to be calculated as follows:





Where n should be between 0.5 - 1.5 for the best characteristic. Refer here again for more treatment of the theory behind this.


DelayA.cs
The new DelayA class inherits from Delay and modifies the pop operation to implement the all-pass algorithm and the Length setter now handles setting a non-integer value.


using System;
using System.Net;


namespace AgSynth
{
    public class DelayA : Delay
    {
        private double alpha = 0;
        private double coeff = 0;

        public DelayA() : base()
        {
        }

 
        double ydel = 0;
        double xdel = 0;

        public override double pop()
        {
            double x = get(length);

            double y = (-coeff * ydel) + xdel + (coeff * x);

            xdel = x;
            ydel = y;

            return y;
        }

        public double Length
        {
            set
            {
                int l = Helpers.Truncate(value);
                alpha = 1.0 + value - l;

                // optimal allpass range is between  0.5 - 1.5 to achieve flattest phase delay response
                if (alpha < 0.5)
                {
                    l++;
                    alpha += 1.0;
                }

                // coefficient for filter
                coeff = (1.0 - alpha) / (1.0 + alpha);

                base.Length = l;
            }
        }

    }
}

Again, this works very well and overcomes the previous problems. To show this off working to it's best some additions and modifications need to be made to the other code that I will cover in a follow-on post.

Monday, 7 July 2014

Karplus-Strong in Silverlight

Following on from my previous post making audio signals with Silverlight, I thought it would be good to do something a bit more interesting to explore the possibilities. I had a bit of a play about with this last summer, so have dug out the code again to break it into some steps and hopefully can then show some more interesting examples.

One of the easiest algorithms to get an interesting sound with very little coding is the Karplus-Strong delay-line which makes a very basic simulation of a plucked string through a simplification of the more generalised concept of digital waveguide synthesis.

Basically a signal, usually white noise, is used to simulate the string pluck and then the signal is delayed and put into a low-pass filter to simulate the decay of the string oscillations.

The basic filter circuit is shown below:


There are already some good examples of this online. Probably the most accessible and closest to what we're doing is by Andre-Michelle who has posted his AS3 implementation. He also has taken this a bit further to simulate all six strings along with resonance, which we'll get onto in a future post hopefully.

The other notable, accessible way to see this algorithm and more is the well-know Synthesis Toolkit in C++ (STK). The source is described and there is a lot of reference material explaining this and the other concepts that are built on it. Andy Li has also done an excellent job of porting much of this over into AS3 (stk-in-as3).

And back to some code here... we're going to build on the previous posting by making a new Generator class called Karp:


using System;
using System.Net;

namespace AgSynth
{
    public class Karp: Generator
    {
        int excite = 70; // excitation duration in samples
        int noise = 0; // gate if noise is being generated or not
        int length = 0; // length of the delay line
        Random r = new Random();

        double Fo = 0; // fundamental frequency (pitch requested)
        double vlpfdelay = 0; // low pass filter internal delay value

        Delay delay = new Delay();

        public Karp()
        {
        }

        public void pluck()
        {
            noise += excite;
        }

        // helper function
        static int Truncate(double d)
        {
            return (int)(d > 0 ? Math.Floor(d) : -Math.Floor(-d));
        }

        public void pitch(double p)
        {
            if (p == 0)
            {
                delay.Length = 0;
            }
            else
            {
                Fo = p; // frequency
                double Fs = PcmStreamSource.SampleRate;

                double No = Fs / Fo;
                length = Truncate(No - 0.5);
                excite = Truncate(length * 0.95);

                double fraction = No - 0.5 - length;

                delay.Length = length;
            }
        }



        public override short tick()
        {
            short sample = 0;

            if (length > 0)
            {
                double amp = 0.99; // amplification

                double vin = 0;

                // push noise into circuit if still generating noise
                if (noise > 0) 
                {
                    vin = (r.NextDouble() * 2.0) - 1.0; // generate noise sample
                    noise--;
                }

                double decay = 0.999;

                double vinlpf = delay.pop();  // get the delayed value

                // filter the output with simple LPF and decay
                double voutlpf = ((vlpfdelay)  + (vinlpf)) * 0.5 * decay;
                vlpfdelay = voutlpf;

                // output = mix input with filter
                double vout = voutlpf + vin;

                // push the output to the start of the delay-line
                delay.push(vout);
                delay.increment();

                // amplify
                sample = (short)(vout * short.MaxValue * amp);
            }

            return sample;
        }
    }
}

There are three main methods here:

  • pitch - which sets the pitch (or frequency of the note)
  • pluck - which sets the counter for the gate for noise to be pumped into the algorithm
  • tick - which is our main algorithm
In the Karplus-Strong algorithm the pitch is set by the length of the delay-line. I'll get onto this a bit more in a follow-on post. The Karp class therefore uses a generic ring-buffer class called delay (emulating the approach used by STK).

In this implementation push puts the sample on the end of the delay-line, pop takes the last sample off the delay line and increment cycles the delay-line by a sample.



using System;
using System.Net;


namespace AgSynth
{
    public class Delay
    {
        private const int MAXLENGTH = 1000;
        private double[] buffer = new double[MAXLENGTH + 1];

        private int idx = 0;
        private int length = 0;

        public Delay()
        {
            for (int i = 0; i < MAXLENGTH + 1; i++)
            {
                buffer[i] = 0;
            } 
        }

        public void increment()
        {
            lock (buffer)
            {
                idx = (idx + 1 > length) ? 0 : idx + 1;
            }
        }


        public double pop()
        {
            double value = get(length);
            //increment();

            return value;
        }

        public void push(double value)
        {
            set(0, value);
        }

        public double get(int index)
        {
            double result = 0.0;

            if (buffer != null)
            {
                lock (buffer)
                {
                    int pos = (idx + index >= length) ? idx + index - length : index + idx;
                    result = buffer[pos];
                }
            }

            return result;
        }

        public void set(int index, double value)
        {
            if (buffer != null)
            {
                lock (buffer)
                {
                    int pos = (idx + index >= length) ? idx + index - length : index + idx;
                    buffer[pos] = value;
                }
            }
        }


        public int Length
        {
            set
            {
                length = value;

                if (length < MAXLENGTH)
                {
                    lock (buffer)
                    {
                        double[] newbuffer = new double[MAXLENGTH];

                        for (int i = 0; i < length; i++)
                        {
                            newbuffer[i] = get(i);
                        }

                        buffer = newbuffer;
                        idx = 0;
                    }
                }
            }
        }

    }
}


The tick() function then implements the algorithm. Firstly noise is generated for a certain number of samples (which is scaled by the frequency). A basic two-point moving averager low-pass filter is then implemented to progressively smooth out the high-frequencies and decay the signal.

The MainPage algorithm to make this do something interesting then just needs a few tweaks to get it to play some notes:


        Karp karp = new Karp();

        private void PlayTune()
        {
            double bpm = 120;

            double barduration = 60 * 1000 * 4 / bpm; // in ms - assuming 4/4

            beat = barduration / 4;

            List<double> tune = new List<double>() { Theory.C4, Theory.D4, Theory.E4, Theory.F4, Theory.E4, Theory.D4 };

            while (true)
            {
                foreach (double p in tune)
                {
                    karp.pitch(p);
                    karp.pluck();
                    Thread.Sleep((int)(beat));
                }
            }
        }


        private void ButtonPlay_Click(object sender, RoutedEventArgs e)
        {
            source.Input = karp;
            media.Play();

            if (player == null)
            {
                player = new Thread(new ThreadStart(PlayTune));

                player.Start();
            }
        }

The Theory class puts together the note pitches:


      
    public class Theory
    {  

        public const double C4 = 261.63;
        public const double Cs4 = 277.2;
        public const double D4 = 293.67;
        public const double Ds4 = 311.1;
        public const double E4 = 329.63;
        public const double F4 = 349.23;
        public const double Fs4 = 370.0;
        public const double G4 = 392.00;
        public const double Gs4 = 415.3;
        public const double A4 = 440.00;
        public const double As4 = 466.2;
        public const double B4 = 493.88;
    }


And here is is






Thursday, 3 July 2014

Propellerheads Figure + UX

I know Propellerheads Figure been kicking around for a while (see the review here), but I only got whiff of this last month and have been playing around a bit with it in idle moments. I really like the clean software UX design:



























There's a good review with the developer Kalle Paulsson here.

Since the UI is well-lauded, it's good to see what could have been if they had not taken an alternative approach.... just think how many other apps would have a completely different breath of life with some different GUI design:


What I particularly like about Propellerheads is that they took on the UX design themselves with an understanding of the underlying application and software. It's not just a skinning exercise.


Silverlight Audio

It might not be the current favourite fad using Silverlight, but as my tool of choice, it's just so easy to hack out some interesting ideas. Stimulated by the previous posting about Gibber I resurrected some of the code I'd been playing about with a while back to mess around with some musical ideas and thought it'd be useful to capture the basics of the audio playback framework which can then be built on to generate some neat audio synthesis ideas.

Here's the bare-bones:

First of all, create a basic application and in the MainPage.xaml drop in the following line:


<MediaElement x:Name="media" AutoPlay="False"/>

You then need a new class below which does the bulk of the work:


// PcmStreamSource.cs adapted from 
// http://www.charlespetzold.com/blog/2009/07/Simple-Electronic-Music-Sequencer-for-Silverlight.html
// and from SilverSynth
// http://silversynth.codeplex.com/
 
// the silversynth library is much more tidily packaged and contains a lot of good features 
// this is intended as a much simpler base to get started with some synthesis ideas so 
// strips everything back to a much more minimal base

using System;
using System.Collections.Generic;
using System.IO;
using System.Text;
using System.Windows.Media;

namespace AgSynth
{
    public interface ISampleSource
    {
        short tick();
    }

    public class PcmStreamSource : MediaStreamSource
    {
        public const int SampleRate = 44100;
        public const int BitsPerSample = 16;
        public const int ChannelCount = 1; 

        MediaStreamDescription mediaStreamDescription;
        long startPosition;
        long currentPosition;
        long currentTimeStamp;
        int byteRate;
        short blockAlign;
        MemoryStream memoryStream;
        Dictionary<MediaSampleAttributeKeys, string> emptySampleDict = 
            new Dictionary<MediaSampleAttributeKeys, string>();
        int bufferByteCount;
        const int numSamples = 2 * 256;

        public PcmStreamSource()
        {
            byteRate = SampleRate * ChannelCount * BitsPerSample / 8;
            blockAlign = (short)(BitsPerSample * ChannelCount / 8);
            memoryStream = new MemoryStream();
            bufferByteCount = ChannelCount * BitsPerSample / 8 * numSamples;
            this.BufferLength = 600;
        }

        public int BufferLength
        {
            get { return this.AudioBufferLength; }
            set { this.AudioBufferLength = value; }
        }

        public ISampleSource Input
        {
            get;
            set;
        }

        protected override void OpenMediaAsync()
        {
            startPosition = currentPosition = 0;

            Dictionary<MediaStreamAttributeKeys, string> streamAttributes = new Dictionary<MediaStreamAttributeKeys, string>();
            Dictionary<MediaSourceAttributesKeys, string> sourceAttributes = new Dictionary<MediaSourceAttributesKeys, string>();
            List<MediaStreamDescription> availableStreams = new List<MediaStreamDescription>();

            string format = "";
            format += ToLittleEndianString(string.Format("{0:X4}", 1)); //PCM
            format += ToLittleEndianString(string.Format("{0:X4}", ChannelCount)); 
            format += ToLittleEndianString(string.Format("{0:X8}", SampleRate));
            format += ToLittleEndianString(string.Format("{0:X8}", byteRate));
            format += ToLittleEndianString(string.Format("{0:X4}", blockAlign));
            format += ToLittleEndianString(string.Format("{0:X4}", BitsPerSample));
            format += ToLittleEndianString(string.Format("{0:X4}", 0));

            streamAttributes[MediaStreamAttributeKeys.CodecPrivateData] = format;


            mediaStreamDescription = new MediaStreamDescription(MediaStreamType.Audio, streamAttributes);
            availableStreams.Add(mediaStreamDescription);
            sourceAttributes[MediaSourceAttributesKeys.Duration] = "0"; 
            sourceAttributes[MediaSourceAttributesKeys.CanSeek] = "false";
            ReportOpenMediaCompleted(sourceAttributes, availableStreams);
        }

        protected override void GetSampleAsync(MediaStreamType mediaStreamType)
        {


            for (int i = 0; i < numSamples; i++)
            {
                short sample = 0;
                if (this.Input != null)
                {
                    sample = this.Input.tick();
                }

                for (int c = 0; c < ChannelCount; c++)
                {
                    //channel
                    memoryStream.WriteByte((byte)(sample & 0xFF));
                    memoryStream.WriteByte((byte)(sample >> 8));
                }
            }

            MediaStreamSample mediaStreamSample =
                new MediaStreamSample(mediaStreamDescription, memoryStream, currentPosition,
                                      bufferByteCount, currentTimeStamp, emptySampleDict);

            currentTimeStamp += bufferByteCount * 10000000L / byteRate;
            currentPosition += bufferByteCount;

            ReportGetSampleCompleted(mediaStreamSample);
        }

        protected override void SeekAsync(long seekToTime)
        {
            this.ReportSeekCompleted(seekToTime);
        }

        protected override void CloseMedia()
        {
            startPosition = currentPosition = 0;
            mediaStreamDescription = null;
        }

        protected override void GetDiagnosticAsync(MediaStreamSourceDiagnosticKind diagnosticKind)
        {
            throw new NotImplementedException();
        }

        protected override void SwitchMediaStreamAsync(MediaStreamDescription mediaStreamDescription)
        {
            throw new NotImplementedException();
        }

        string ToLittleEndianString(string bigEndianString)
        {
            StringBuilder builder = new StringBuilder();

            for (int i = 0; i < bigEndianString.Length; i += 2)
            {
                builder.Insert(0, bigEndianString.Substring(i, 2));
            }

            return builder.ToString();
        }


    }
}

I've then stripped down the rest of the framework that is used in the referenced code from Petzold and Mike Hodnick (kindohm).

We can now build the simplest kind of synth generator to make a middle-C (C4) sine-wave signal:


using System;
using System.Net;
using System.Windows;
using System.Windows.Controls;
using System.Windows.Documents;
using System.Windows.Ink;
using System.Windows.Input;
using System.Windows.Media;
using System.Windows.Media.Animation;
using System.Windows.Shapes;

namespace AgSynth
{
    public class Generator : ISampleSource
    {
        public virtual short tick()
        {
            return 0;
        }

    }


    public class Sine : Generator
    {
        int n = 0;
        const double C4 = 261.6; // middle C in Hz
        double f = C4; // Hz

        double vol = 1.0;

        public override short tick()
        {
            short sample = (short)(short.MaxValue * vol * Math.Sin(f * (2 * Math.PI) * n / PcmStreamSource.SampleRate));
            n++;

            return sample;
        }

    }
}

Effectively what happens here is each tick a new sample is generated for the desired signal. Please take a look at SilverSynth to see how this basic framework can be built on to make more advanced processing. There's a great example of this done very nicely here.

All we need to do is wire this up to the MainPage now. So, firstly create the PcmStreamSource object:

PcmStreamSource source = new PcmStreamSource();

Set the MediaSource when the control is loaded:


private void UserControl_Loaded(object sender, RoutedEventArgs e)
{
    media.SetSource(source);
}

Don't forget to make sure you have the loaded event in the XAML.

And last but not least, the fun bit, make a button to start (like a play-button) and in the event create the generator object, assign it to the MediaElement and play away:


Sine sine = new Sine();

private void Button_Click(object sender, RoutedEventArgs e)
{
    source.Input = sine;
    media.Play();
}

Not hugely exciting to blast out a single sine-wave frequency, but as Fourier analysis shows all music can be made from the assembly of suitable sine-waves, so it's a start!

Note - one thing I did notice in this was that there is a gotcha here that had me bashing my head a little bit. I was getting the OpenMediaAsync called but not the GetSampleAsync. I think it is also the cause of some similar problems that have been reported. I tracked my problem down to the annoying issue that if you put the lines in Button_Click into the UserControl_Loaded event, e.g. to kick-off playing at the start, the initialisation does not seem to happen properly.



65daysofstatic Gibber - yrCodeIsBadAndYouShouldFeelBad

Been a bit distracted recently with a number of things so only finally getting round to jotting this down. I got a Facebook update from my fav band of 2014 - 65daysofstatic - with some performance code using Gibber. It now looks like it is front page on their web-site.

Before we get to Gibber, if you haven't heard Wild Light yet,take a listen to some of the tracks, then go and order it from your favourite purveyor of music and take your ears onto their Soundcloud page. Thanks to ComfortConspiracy for playing this on the excellent last Christmas podcast (when is the next one, it was just teasing to say Part 1!).

Back to Gibber, this seems to be the brain-child of Charlie Roberts at UCSB (a general big-up for UC as a Bruin myself) in their Media Arts and Technology Program. Take a look at Charlie's page for additional information and a link to the source code on GitHub.

65dos have written a song (yrCodeIsBadAndYouShouldFeelBad) entirely in code. The 65dos style is there to see - love the use of pluckArp for the guitar sound and coldSynthAnUndefinableSadnessCheerUpThoughEh?

I could only get this to run in Chrome, but ran on Win and Mac and spent a wonderful evening of tinkerage playing around with the sounds myself. There is a great Vimeo vid of Gibber being used as a performance musical tool.