LTI system theory

Encyclopedia : L : LT : LTI : LTI system theory

In electrical engineering, specifically in circuits, signal processing, and control theory, LTI system theory investigates the response of a linear, time-invariant system to an arbitrary input signal. Though the standard independent variable is time, it could just as easily be space (as in image processing and field theory) or some other coordinate. Thus a better, albeit less common, term is linear translation-invariant. The term linear shift-invariant is the corresponding concept for a discrete-time (sampled) system.

Contents

Overview

The defining properties of any linear time-invariant system are, of course, linearity and time invariance:

Linearity means that the relationship between the input and the output of the system satisfies the scaling and superposition properties. Formally, a linear system is a system which exhibits the following property: if the input of the system is

:[x(t) = Ax_1(t) + Bx_2(t) \, ]

then the output of the system will be

:[y(t) = Ay_1(t) + By_2(t) \, ]

for any constants A and B, where [y_i(t)] is the output when the input is [x_i(t)].

Time invariance means that whether we apply an input to the system now or T seconds from now, the output will be identical, except for a time delay of the T seconds. More specifically, an input affected by a time delay should effect a corresponding time delay in the output, hence time-invariant.

The fundamental result in LTI system theory is that any LTI system can be characterized entirely by a single function called the system's impulse response. The output of the system is simply the convolution of the input to the system with the system's impulse response. This method of analysis is often called the time domain point-of-view. The same result is true of discrete-time linear shift-invariant systems, in which signals are discrete-time samples, and convolution is defined on sequences.

320px

Equivalently, any LTI system can be characterized in the frequency domain by the system's transfer function, which is the Laplace transform of the system's impulse response (or Z transform in the case of discrete-time systems). As a result of the properties of these transforms, the output of the system in the frequency domain is the product of the transfer function and the transform of the input. In other words, convolution in the time domain is equivalent to multiplication in the frequency domain.

For all LTI systems, the eigenfunctions, and the basis functions of the transforms, are complex exponentials. This is, if the input to a system is the complex waveform [A\exp()] for some complex amplitude [A] and complex frequency [s], the output will be some complex constant times the input, say [B\exp()] for some new complex amplitude [B]. The ratio [B/A] is the transfer function at frequency [s].

Because sinusoids are a sum of complex exponentials with complex-conjugate frequencies, if the input to the system is a sinusoid, then the output of the system will also be a sinusoid, perhaps with a different amplitude and a different phase, but always with the same frequency.

LTI system theory is good at describing many important systems. Most LTI systems are considered "easy" to analyze, at least compared to the time-varying and/or nonlinear case. Any system that can be modeled as a linear homogeneous differential equation with constant coefficients is an LTI system. Examples of such systems are electrical circuits made up of resistors, inductors, and capacitors (RLC circuits). Ideal spring–mass–damper systems are also LTI systems, and are mathematically equivalent to RLC circuits.

Most LTI system concepts are similar between the continuous-time and discrete-time (linear shift-invariant) cases. In image processing, the time variable is replaced with 2 space variables, and the notion of time invariance is replaced by two-dimensional shift invariance. When analyzing filter banks and MIMO systems, it is often useful to consider vectors of signals.

Continuous-time systems

Time invariance and linear transformation

Let us start with a time-varying system whose impulse response is a 2-dimensional function and see how the condition of time invariance helps us reduce it to one dimension. For example, suppose the input signal is [x(t)] where its index set is the real line, i.e., [t \in \mathbb]. The linear operator [\mathcal] represents the system operating on the input signal. The appropriate operator for this index set is a 2-dimensional function

[h(t_1, t_2) \mbox t_1, t_2 \in \mathbb.]

Since [\mathcal] is a linear operator, the action of the system on the input signal [x(t)] is a linear transformation represented by the following superposition integral

[y(t_1) = \int_^ h(t_1, t_2) \, x(t_2) \, d t_2.]

If the linear operator [\mathcal] is also time-invariant, then

[ h(t_1, t_2) = h(t_1 + \tau, t_2 + \tau) \qquad \forall \, \tau \in \mathbb.]

If we let

[ \tau = -t_2, \, ]

then it follows that

[h(t_1, t_2) = h(t_1 - t_2, 0). \, ]

We usually drop the zero second argument to [h (t_1, t_2)] for brevity of notation so that the superposition integral now becomes the familiar convolution integral used in filtering

[y(t_1) = \int_^ h(t_1 - t_2) \, x(t_2) \, d t_2 = (h * x) (t_1).]

Thus, the convolution integral represents the effect of a linear, time-invariant system on any input function. For a finite-dimensional analog, see the article on a circulant matrix.

Impulse response

If we input a Dirac delta function to this system, the result of the LTI transformation is known as the impulse response because the delta function is an ideal impulse. We illustrate this idea as follows:

[ (h * \delta) (t) = \int_^ h(t - \tau) \, \delta (\tau) \, d \tau = h(t),]

(by the sifting property of the delta function).

Note that

[h(t) = h(t, 0) \ (\mbox t = t_1 - t_2)]

so that [h(t)] is the impulse response of the system.

The impulse response can be used to find the response of any input in the following way. Again using the sifting property of the [\delta(t)], we can write any input as a superposition of deltas:

[x(t) = \int_^\infty x(\tau) \delta(t-\tau) \,d\tau]

Applying the system to the input,

[\mathcal x(t) = \mathcal \int_^\infty x(\tau) \delta(t-\tau) \,d\tau]

[\quad = \int_^\infty \mathcal x(\tau) \delta(t-\tau) \,d\tau] (because [\mathcal] is linear and can pass inside the integral)

[\quad = \int_^\infty x(\tau) \mathcal \delta(t-\tau) \,d\tau] (because [x(\tau)] is constant in t and [\mathcal] is linear)

[\quad = \int_^\infty x(\tau) h(t-\tau) \,d\tau] (by definition of [h(t)])

All information about the system is contained in the impulse response [h(t)].

Exponentials as eigenfunctions

An eigenfunction is a function for which the output of the operator is the same function, just scaled by some amount. In symbols,

[\mathcalf = \lambda f],

where f is the eigenfunction and [\lambda] is the eigenvalue, a constant.

The exponential functions [e^], where [s \in \mathbb], are eigenfunctions of a linear, time-invariant operator. A simple proof illustrates this concept.

Suppose the input is [x(t) = e^]. The output of the system with impulse response [h(t)] is then

[\int_^ h(t - \tau) e^ d \tau]

which is equivalent to the following by the commutative property of convolution

[\int_^ h(\tau) \, e^ \, d \tau]

[ \quad = e^ \int_^ h(\tau) \, e^ \, d \tau]

[ \quad = e^ H(s)],

where

[H(s) = \int_^\infty h(t) e^ d t]

is dependent only on the parameter s.

So, [e^] is an eigenfunction of an LTI system because the system response is the same as the input times the constant [H(s)].

Fourier and Laplace Transforms

The eigenfunction property of exponentials is very useful for both analysis and insight into LTI systems. The Laplace transform

[H(s) = \mathcal\ = \int_^\infty h(t) e^ d t]

is exactly the way to get the eigenvalues from the impulse response. Of particular interest are pure sinusoids, i.e. exponentials of the form [\exp()] where [\omega \in \mathbb] and [j = \sqrt]. These are generally called complex exponentials even though the argument is purely imaginary. The Fourier transform [H(j \omega) = \mathcal\] gives the eigenvalues for pure complex sinusoids. Both of [H(s)] and [H(j\omega)] are called the system function, system response, or transfer function.

The Laplace transform is usually used in the context of one-sided signals, i.e. signals that are zero for all values of t less than some value. Usually, this "start time" is set to zero, for convenience and without loss of generality, with the transform integral being taken from zero to infinity (the transform shown with lower limit of integration of negative infinity is formally known as the bilateral Laplace transform).

The Fourier transform is used for analyzing systems that process signals that are infinite in extent, such as modulated sinusoids, even though it can not be directly applied to input and output signals that are not square integrable. The Laplace transform actually works directly for these signals if they are zero before a start time, even if they are not square integrable, for stable systems. The Fourier transform is often applied to spectra of infinite signals via the Wiener–Khinchin theorem even when Fourier transforms of the signals do not exist.

Due to the convolution property of both of these transforms, the convolution that gives the output of the system can be transformed to a multiplication in the transform domain, given signals for which the transforms exist

[y(t) = (h*x)(t) = \int_^\infty h(t - \tau) x(\tau) d \tau]

[\quad = \mathcal^\]

Not only is it often easier to do the transforms, multiplication, and inverse transform than the original convolution, but one can also gain insight into the behavior of the system from the system response. One can look at the modulus of the system function |H(s)| to see whether the input [\exp()] is passed (let through) the system or rejected or attenuated by the system (not let through).

Examples

A simple example of an LTI operator is the derivative:

[ \frac \left( c_1 x_1(t) + c_2 x_2(t) \right) = c_1 x'_1(t) + c_2 x'_2(t), ]

[ \frac x(t-\tau) = x'(t-\tau). ]

When the Laplace transform of the derivative is taken, it transforms to a simple multiplication by the Laplace variable s.

[ \mathcal\left\x(t)\right\} = s X(s) ]

That the derivative has such a simple Laplace transform partly explains the utility of the transform.

Another simple LTI operator is an averaging operator

[ \mathcal\left\ = \int_^ x(\lambda) d \lambda ].

It is linear because of the linearity of integration

[ \mathcal\left\ ]

[ = \int_^ \left( c_1 x_1(\lambda) + c_2 x_2(\lambda) \right) d \lambda ]

[ = c_1 \int_^ x_1(\lambda) d \lambda + c_2 \int_^ x_2(\lambda) d \lambda ]

[ = c_1 \mathcal\left\ + c_2 \mathcal\left\ ].

It is time invariant too

[ \mathcal\left\ ]

[ = \int_^ x(\lambda-\tau) d \lambda ]

[ = \int_^ x(\xi) d \xi ]

[ = \mathcal\(t-\tau) ].

Indeed, [\mathcal] can be written as a convolution with the box function [\Pi(t)].

[ \mathcal\left\ = \int_^\infty \Pi\left(\frac\right) x(\lambda) d \lambda ],

where the box function is

[\Pi(t) = \left\ 1 & |t| < 1/2 \\ 0 & |t| > 1/2 \end \right. ].

Important System Properties

Some of the most important properties of a system are causality and stability. It is more or less necessary for a system to be causal in order for it to be implemented in the real world. Non-stable systems can be built and can be useful in many circumstances. Even non-real systems can be built and are very useful in many contexts.

Causality

A system is causal if the output depends only on present and past inputs. A necessary and sufficient condition for causality is

[h(t) = 0 \quad \forall t < 0,]

where [h(t)] is the impulse response. It is not possible in general to determine causality from the Laplace transform, because the inverse transform is not unique. When a region of convergence is specified, then causality can be determined.

Stability

A system is bounded input, bounded output stable (BIBO stable) if, for every bounded input, the output is finite. Mathematically, if

[||x(t)||_\infty < \infty]

and

[||y(t)||_\infty < \infty]

(i.e., the maximum absolute values of [x(t)] and [y(t)] are finite), then the system is stable. A necessary and sufficient condition is that [h(t)], the impulse response, satisfies

[||h(t)||_1 = \int_^\infty |h(t)| dt < \infty.]

In the frequency domain, the region of convergence must contain the imaginary axis [s=j\omega].

Discrete-time systems

Almost everything in continuous time systems has an analog in discrete time systems.

Discrete-time systems from continuous-time systems

In many contexts, a discrete time (DT) system is really part of a larger continuous time (CT) system. For example, a digital recording system takes an analog sound, digitizes it, possibly processes the digital signals, and plays back an analog sound for people to listen to.

Formally, the DT signals studied are almost always uniformly sampled versions of CT signals. If [x(t)] is a CT signal, then an analog to digital converter will transform it to the DT signal [x[n]], with

[x[n] = x(nT)],

where T is the sampling period. It is very important to limit the range of frequencies in the input signal for faithful representation in the DT signal. Due to the sampling theorem, a DT signal can only contain a frequency range of [1/(2T)]. Other frequencies are aliased to the same range.

Time invariance and linear transformation

Let us start with a time-varying system whose impulse response is a two dimensional function and see how the condition of time-invariance helps us reduce it to one dimension. For example, suppose the input signal is [x[n]] where its index set is the integers, i.e., [n \in \mathbb]. The linear operator [\mathcal] represents the system operating on the input signal. The appropriate operator for this index set is a two-dimensional function

[h[n_1, n_2] \mbox n_1, n_2 \in \mathbb.]

Since [\mathcal] is a linear operator, the action of the system on the input signal [x[n]] is a linear transformation represented by the following superposition sum

[y[n_1] = \sum_^ h[n_1, n_2] \, x[n_2],]

If the linear operator [\mathcal] is also time-invariant, then

[ h[n_1, n_2] = h[n_1 + m, n_2 + m] \qquad \forall \, m \in \mathbb.]

If we let

[ m = -n_2, \, ]

then it follows that

[h[n_1, n_2] = h[n_1 - n_2, 0]. \, ]

We usually drop the zero second argument to [h[n_1, n_2]] for brevity of notation so that the superposition integral now becomes the familiar convolution sum used in filtering

[y[n_1] = \sum_^ h[n_1 - n_2] \, x[n_2] = (h * x) [n_1].]

Thus, the convolution sum represents the effect of a linear, time-invariant system on any input function. For a finite-dimensional analog, see the article on a circulant matrix.

Impulse response

If we input a discrete delta function to this system, the result of the LTI transformation is known as the impulse response because the delta function is an ideal impulse. We illustrate this idea as follows:

[ (h * \delta) [n] = \sum_^ h[n - m] \, \delta [m] = h[n],]

(by the sifting property of the delta function).

Note that

[h[n] = h[n_1 - n_2, 0] \,\!\mbox n = n_1 - n_2,]

so that [h[n]] is the impulse response of the system.

The impulse response can be used to find the response of any input in the following way. Again using the sifting property of the [\delta[n]], we can write any input as a superposition of deltas:

[x[n] = \sum_^\infty x[m] \delta[n-m].]

Applying the system to the input,

[\mathcal x[n] = \mathcal \sum_^\infty x[m] \delta[n-m]]

[\quad = \sum_^\infty \mathcal x[m] \delta[n-m]] (because [\mathcal] is linear and can pass inside the sum)

[\quad = \sum_^\infty x[n] \mathcal \delta[n-m]] (because [x[m]] is constant in n and [\mathcal] is linear)

[\quad = \sum_^\infty x[m] h[n-m]] (by definition of [h[n]])

All information about the system is contained in the impulse response [h[n]].

Exponentials as eigenfunctions

An eigenfunction is a function for which the output of the operator is the same function, just scaled by some amount. In symbols,

[\mathcalf = \lambda f],

where f is the eigenfunction and [\lambda] is the eigenvalue, a constant.

The exponential functions [z^n = e^], where [n \in \mathbb], are eigenfunctions of a linear, time-invariant operator. [T \in \mathbb] is the sampling interval, and [z = e^, \ z,s \in \mathbb]. A simple proof illustrates this concept.

Suppose the input is [x[n] = \,\!z^n]. The output of the system with impulse response [h[n]] is then

[\sum_^ h[n-m] \, z^m]

which is equivalent to the following by the commutative property of convolution

[\sum_^ h[m] \, z^]

[ \quad = z^n \sum_^ h[m] \, z^]

[ \quad = z^n H(z)],

where

[H(z) = \sum_^\infty h[n] z^]

is dependent only on the parameter z.

So, [z^n] is an eigenfunction of an LTI system because the system response is the same as the input times the constant [H(z)].

Z and Discrete-time Fourier transforms

The eigenfunction property of exponentials is very useful for both analysis and insight into LTI systems. The Z transform

[H(z) = \mathcal\ = \sum_^\infty h[n] z^]

is exactly the way to get the eigenvalues from the impuse response. Of particular interest are pure sinusoids, i.e. exponentials of the form [e^], where [\omega \in \mathbb]. These can also be written as [z^n] with [z = e^]. These are generally called complex exponentials even though the argument is purely imaginary. The Discrete-time Fourier transform (DTFT) [H(e^) = \mathcal\] gives the eigenvalues of pure sinusoids. Both of [H(z)] and [H(e^)] are called the system function, system response, or transfer function.

The Z transform is usually used in the context of one-sided signals, i.e. signals that are zero for all values of t less than some value. Usually, this "start time" is set to zero, for convenience and without loss of generality. The Fourier transform is used for analyzing signals that are infinite in extent.

Due to the convolution property of both of these transforms, the convolution that gives the output of the system can be transformed to a multiplication in the transform domain.

[y[n] = (h*x)[n] = \sum_^\infty h[n-m] x[m]]

[\quad = \mathcal^\]

Not only is it often easier to do the transforms, multiplication, and inverse transform than the original convolution, one can gain insight into the behavior of the system from the system response. One can look at the modulus of the system function |H(z)| to see whether the input [z^n] is passed (let through) by the system, or rejected or attenuated by the system (not let through).

Examples

A simple example of an LTI operator is the delay operator [D\[n]:=x[n-1]].

[ D \left( c_1 x_1[n] + c_2 x_2[n] \right) = c_1 x_1[n-1] + c_2 x_2[n-1] = c_1 Dx_1[n] + c_2 Dx_2[n], ]

[ D\ = x[n-m-1] = x[(n-1)-m] = D\[n-m]. \,]

When the Z transform of the difference is taken, it transforms to a simple multiplication by z:

[ \mathcal\left\ = z X(z). ]

That the difference has such a simple Z transform partly explains the utility of the transform.

Another simple LTI operator is an averaging operator

[ \mathcal\left\ = \sum_^ x[k]].

It is linear because of the linearity of sums:

[ \mathcal\left\ ]

[ = \sum_^ \left( c_1 x_1[k] + c_2 x_2[k] \right) ]

[ = c_1 \sum_^ x_1[k] + c_2 \sum_^ x_2[k] ]

[ = c_1 \mathcal\left\ + c_2 \mathcal\left\ ].

It is time invariant too:

[ \mathcal\left\ ]

[ = \sum_^ x[k-m] ]

[ = \sum_^ x[k'] ]

[ = \mathcal\left\[n-m] ].

Important system properties

Some of the most important properties of a system are causality and stability. Unlike CT systems, non-causal DT systems can be realized. It is trivial to make an acausal FIR system causal by adding delays. It is even possible to make acausal IIR systems (See Vaidyanathan and Chen, 1995). Non-stable systems can be built and can be useful in many circumstances. Even non-real systems can be built and are very useful in many contexts.

Causality

A system is causal if the output depends only on present and past inputs. A necessary and sufficient condition for causality is

[h[n] = 0 \ \forall n < 0,]

where [h[n]] is the impulse response. It is not possible in general to determine causality from the Z transform, because the inverse transform is not unique. When a region of convergence is specified, then causality can be determined.

Stability

A system is bounded input, bounded output stable (BIBO stable) if, for every bounded input, the output is finite. Mathematically, if

[||x[n]||_\infty < \infty]

and

[||y[n]||_\infty < \infty]

(i.e., the maximum absolute values of [x[n]] and [y[n]] are finite), then the system is stable. A necessary and sufficient condition is that [h[n]], the impulse response, satisfies

[||h[n]||_1 = \sum_^\infty |h[n]| < \infty.]

In the frequency domain, the region of convergence must contain the unit circle [|z|=1].

LTI system theory

Overview

Continuous-time systems

Time invariance and linear transformation

Impulse response

Exponentials as eigenfunctions

Fourier and Laplace Transforms

Examples

Important System Properties

Causality

Stability

Discrete-time systems

Discrete-time systems from continuous-time systems

Time invariance and linear transformation

Impulse response

Exponentials as eigenfunctions

Z and Discrete-time Fourier transforms

Examples

Important system properties

Causality

Stability

See also