The Lms Widrow Algorithm

Published Date: 02 Nov 2017

This is an abstract of the notch this is an abstract of a digital filter this is an abstract of a digital filter is an abstract of the notch this is an abstract of a digital filter this is an abstract of a digital filter

Introduction

This report will cover the design of an IIR Notch filter, by using the Pole/Zero placement method, and an analysis of an adaptive filter that is based on the LMS Widrow algorithm.

Notch filter design

Table 1.1 Filter specification

Frequency notch

50Hz

Frequency sampling

8KHz

Bandwidth

5Hz

Theory

A graphical method is used to compute the filter coefficients [1], the transfer function of a 2nd order filter is shown in figure 1.1 the design requires the computation of the coefficients a(k) and b(k).

\* MERGEFORMAT (.)

Design equations

\* MERGEFORMAT (.)

\* MERGEFORMAT (.)

\* MERGEFORMAT (.)

The bandwidth of the filter is controlled by the poles

\* MERGEFORMAT (.)

\* MERGEFORMAT (.)

\* MERGEFORMAT (.)

A discrete system can in general be described by its transfer function is given below.

\* MERGEFORMAT (.)

\* MERGEFORMAT (.)

Figure 1.1 Z-plane showing complex conjugate poles and zeros.

From equation 1.9 the location of zeros are place on the unit circle of the z-plane at the points + pi/80 and – pi/80, a complex conjugate pair, and the poles are also place at the same angle with magnitude R, note the magnitude of R determines the bandwidth of the filter.

Now with algebraic manipulation the filter coefficients can be computed.

\* MERGEFORMAT (.)

\* MERGEFORMAT (.)

\* MERGEFORMAT (.)

Now divide by top and bottom of equation.

\* MERGEFORMAT (.)

From equation (1.13) the coefficients are

\* MERGEFORMAT (.)

\* MERGEFORMAT (.)

Note the highest power is z-2 in the transfer function H(z), therefore a 2nd order filter, it is possible to implement this filter using a minimum of delay elements with a canonical structure as shown below.

Figure 1.2 second order canonical filter structure.

Testing the Notch Filter

MATLAB is used to determine the frequency response of the filter and to confirm the notch frequency response.

The MATLAB generated frequency and phase response of the notch filter is shown below in figure 1.3 and figure 1.4.

Figure 1.3 Notch filter frequency response.

Figure 1.4 Notch filter phase response.

A test signal will be applied to the filter, composed of two waveforms, one at 50Hz and the other at 120Hz, the magnitude spectrum of these two input test signals is shown in figure 1.5.

An input signal is applied as the input of the filter where is given by

\* MERGEFORMAT (.)

\* MERGEFORMAT (.)

MATLAB generated magnitude spectrum of the Notch filter test signal.

Figure 1.5 Magnitude spectrum of Notch filter input and output test signals.

The input signal has two components one at 50Hz and the other at 120Hz; these will now be applied as the input signal to the digital notch filter.

Filter output

The expectation is that the 50Hz signal will be attenuated

Drawbacks of IIR Notch Filter

The IIR filter removes noise by attenuating all the signals that are at 50 Hz. If the noise signal frequency deviates and enters the spectrum of the wanted signal, there will be a loss of information. The filter bandwidth can be made narrower but would require a higher order filter with added computational

time, but the same problem could occur when noise and desired signal occupy the same bandwidth.

Introduction Least Mean Square (LMS) Algorithm

The adaptive LMS Widrow algorithm was developed by Widrow [2]

The performance of an adaptive filter using the adaptive LMS algorithm will be investigated using MATLAB, in a noise cancellation application. The analysis begins by using the Wiener filter model, the block diagram is shown in figure 1.6. The objective is to minimize the output error signal e(n) by using feedback. The mathematics that follows describes how this is achieved.

Figure 1.6 Wiener filter

Adaptive noise cancellation configuration.

\* MERGEFORMAT (.)

Where the filter is an FIR filter then:

\* MERGEFORMAT (.)

\* MERGEFORMAT (.)

Where MSE

\* MERGEFORMAT (.)

Now taking the derivative of with respect to

Leads to the Wiener-Hopf equation

\* MERGEFORMAT (.)

From the Wiener-Hopf equation the optimum coefficients that minimizes the MSE is found by using an the

respective auto-correlation and cross-correlation coefficients.

The Wiener-Hopf equation (1.22) form the basis of adaptive filter as used in the LMS algorithm, the drawback is that it requires the values of the correlation functions as this is not possible in a real-time application, this leads onto the LMS Widrow algorithm.

Figure 1.7 shows a block diagram of an adaptive filter that uses the LMS algorithm in a noise cancellation application, where d(n) is the noise affected signal which contains the wanted signal, represented by s(n), and the interfering noise, given by v(n). There is A reference signal x(n), which is highly correlated to v(n) the noise.

Figure 1.7 LMS Widrow adaptive filter.

Figure 1.8 Input and output spectrum of adaptive filter.

Adaptive filter noise cancellation.

Figure 1.9 LMS Widrow filter used in noise cancellation process.

The filter has two main parts, the FIR digital filter with coefficients that can be adjusted coefficients, and the adaptive LMS algorithm that controls the value of the coefficients of the FIR filter. The input and delayed signal d(n) and d(n-D) are both applied at the same time to the filter. Because of the delaying d(n), d(n-D) are less correlated with the noise signal s(n) while it is still correlated with the interference. The delayed input, d(n-D), will be processed by the filter to generate an estimate signal of the noise, y(n). This estimation

of the wanted signal, e(n), will be obtained by the subtraction of the filter output, y(n), from the noise contaminated input , d(n):

\* MERGEFORMAT (.)

\* MERGEFORMAT (.)

The estimate signal, e(n), has two functions, it acts as an estimation of the wanted signal and then as an error signal that can be feedback to update the digital filter coefficients. This updating of filter coefficients is carried out by the adaptive algorithm, therefore acts to minimize the noise and then produce the minimum error estimation of the wanted signal, s(n).

This MSE can be found by the squaring of equation (1.24) and then taking the expectation as follows.

Equation 1.24 squared (1.24) we have.

\* MERGEFORMAT (.)

The expectations on both sides of equation (1.25).

\* MERGEFORMAT (.)

For an desired signal that is not correlated with the noise, s(n) and v(n) will then be uncorrelated, therefore E[v(n)s(n)] = 0 and the last term will be -2E[s(n)y(n)] and equation (1.26) will becomes.

\* MERGEFORMAT (.)

And then s(n) and y(n) are also uncorrelated, which results in E[y(n)s(n)] = 0, the last term also being zero and then equation (1.27) is.

\* MERGEFORMAT (.)

= MSE

It can be seen from equation (1.28) if the adaptive filter is adjusted towards an optimum, the noise and the total output power will then be minimized. The wanted signal, E[s(n)2], is not affected by the adjustments where s(n) is not correlated with v(n). Thus

\* MERGEFORMAT (.)

Now the S/N ratio in the desired output will be maximized and the filter has adaptively learned to generate the noise (i.e. y(n) ï‚» v(n)).

Digital Filter

The filter used in the noise cancellation process is realized using a FIR, The FIR filter is selected for its simplicity and stability. Figure 1.27 shows the FIR filter.

This filter produces the estimation of v(n), y(n), and its output is given by

\* MERGEFORMAT (.)

\* MERGEFORMAT (.)

Where the b(k) are the adjustable coefficients of the filter, and x[n] and y[n] are the input and the output of the filter, respectively.

Figure 1.10 FIR filter structure.

Adaptive algorithms

In the adaptive noise cancelation filter, as shown in Figure 1.26, the coefficients of the filter are updated using the LMS adaptive algorithm. This updating is carried out such that the estimate of the wanted output signal, e(n), is then minimized in accordance to the criterion, e.g. the least squares sense.

LMS Algorithm

The LMS algorithm tries to minimize the expected value of, E[e2k]. It is based on steepest decent algorithm where the weight vectors will be updated on a sample by sample basis as follows:

\* MERGEFORMAT (.)

Now where B(k) and k are weight and gradient vectors of the error function, and  is the step size that governs the rate of convergence.

According to Wiener filter theory, the gradient vector, , of the MSE can be obtained by the differentiation of the MSE equation, , with respect to the weight vector, W That is.

(1.33)

Now the value of P is known as the cross correlation between the input, xk, and the output d(n), and R is the autocorrelation of the input, xk.

Now P and R are not available in a real time basis, then the instantaneous values should be used for .

Thus,

\* MERGEFORMAT (.)

\* MERGEFORMAT (.)

Substituting equation (1.31) for in equation (1.35) gives:

\* MERGEFORMAT (.)

Substituting equation (1.23) for in equation (1.36) gives:

\* MERGEFORMAT (.)

Substituting equation (1.37) for gradient in equation (1.32) gives:

(1.38)

the Widrow-Hopf algorithm that is used for updating the weights of the filter

The coefficient values which are obtained by the LMS algorithm will be an estimate only that should improve over time, when the filter can learn the features of the input signals. For improved performance it will be required that the coefficient values must converge to the optimum value, Wopt, which occurs when the gradient, , in equation (1.33) is towards zero or when . In a real situation, B(k) will not reach the theoretical optimum solution, but will fluctuates around it as shown in Figure 1.28.

Appendix A MATLAB code listings

Notch filter frequency response

clear;

Fs=8000;

R=0.998;

b1=2*cos(pi/80);

a1=2*R*cos(pi/80);

a2=R*R;

b=[1,-b1,1];

a=[1,-a1,a2];

[h,f]=freqz(b,a,2048,Fs);

H=abs(h);

HdB=20*log10(H);

phase=angle(h);

subplot(3,1,1);

plot(f,phase);

title('Phase Response');

xlabel('Frequency');

grid;

subplot(3,1,2);

plot(f,HdB);

title('Frequency Response')

ylabel('Magnitude (dB)');

xlabel('Frequency');

grid;

subplot(3,1,3);

zplane(b,a);

grid;

Notch filter test signal

clear; %Clear the work areas

N=1024; % Total Number of Samples

Fs=1000; % Sampling frequency set at 1000Hz

f1=120;

f2=50;

n=0:N-1;

x=sin(2*pi*(f1/Fs)*n)+sin(2*pi*(f2/Fs)*n);

[pxx,fx]=psd(x,2*N,Fs);

subplot(2,1,1);

plot(fx,20*log10(pxx));

axis([0 250 -40 40])

grid;

title('Magnitude Spectrum of x(n)');

xlabel('Frequency, Hz');

ylabel('Magnitude, dB');

R=0.998;

b1=2*cos(pi/80);

a1=2*R*cos(pi/80);

a2=R*R;

b=[1,-b1,1];

a=[1,-a1,a2];

y=filter(b, a, x);

[pyy, fy]=psd(y, 2*N, Fs);

subplot(2,1,2);

plot(fy, 20*log10(pyy));

grid;

MATLAB code LMS Algorithm

function [B,y,e,J]= Mylms(x, dn, mu, M)

% Use of the function:

% function [B,y,e,J]= Mylms(x, dn, mu, M)

N = length(x);

y = zeros(1, N);

B = zeros(1, M);

for n=M: N

x1=x(n: -1: n-M+1);

y(n) = B * x1';

e(n) = dn(n) - y(n);

B = B + 2*mu*e(n)*x1;

end;

J = e.^2;

% Note J is the learning Curve of the Adaptive Process.

% Adaptive Noise Cancellation Demonstration

% Using the Adaptive LMS Algorithm

clear all;

A=1;

N=128;

f0=1000; % The frequency of the wanted signal s(n)

f1=2000; % The frequency of the noisy signal

Fs=8000;

n=0:N-1;

mu=0.01;

M=10;

% Generate a Sinusoidal of Frequency f0

s=sin(2*pi*(f0/Fs)*n);

% Generate another Sinewave to simulate a noisy tone and add it to

the wanted signal

v=sin(2*pi*(f1/Fs)*n);

% Construct the d(n)=s(n)+v(n)

d=s+v;

% Construct the x(n) to be the same noisy tone

x=sin(2*pi*(f1/Fs)*n);

% Call the LMS Algorithm

[B,y,e,J]= Mylms(x, d, mu, M);

% Plot the noisy signal and the Error Signal as well as their PSD

figure(1);

subplot(2,2,1);

plot(d), title('The Noisy Signal'); grid;

[psdd,f]=psd(d,length(d),Fs);

subplot(2,2,3);

plot(f,psdd), title('The Spectrum of the Noisy Signal'); grid;

subplot(2,2,2);

plot(e), title('The cleared Signal (Error Signal!)'); grid;

[psde,f]=psd(e,length(e),Fs);

subplot(2,2,4);

plot(f,psde), title('The Spectrum of the Error Signal (i.e s(n)');

grid;