Linear Time-invariant Systems

Linearity

A system is linear if it can be defined by linear differential equations. In particulars, if the functions $\mathbf{f}$ and $g$ in its state-space model are linear functions of the state variables $\mathbf{x}(t)$ and input $u(t)$.

Linear systems have two properties:

1. Homogeneity: Scalar multiplication works as we expect.
   • If we have a system that takes $x(t)$ as input and outputs $y(t)$, inputting $ax(t)$ will instead return $ay(t)$.
2. Additivity/superposition: Addition works as we expect it to.
   • If we have a system that has $x_1(t)\to \cdots \to y_1(t)$ and $x_2(t)\to \cdots \to y_2(t)$, passing in $x_1(t)+x_2(t)$ will result in $y_1(t)+y_2(t)$

Time-Invariance

A system is said time-invariant if it can expressed be by differential equations with constant coefficients. In particular, if the functions $\mathbf{f}$ and $g$ in its state space model do not depend on the time $t$ explicitly.

• See some examples in Time-invariant and time varying systems

Assume that a time-invariant system has zero initial conditions, and zero input generates zero output. If input $u(t)$ produces output $y(t)$, then input $u(t-\tau )$ for all $\tau \ge 0$.

So if a system has $x(t)\to \cdots \to y(t)$, but instead we input $x(t-t_0)$, we will have

$$x(t-t_0)\to \cdots \to y(t,t_0)$$

If $y(t,t_0)=y(t-t_0)$, then the system is time invariant.

Examples

Example 1

Let’s say we have $y(t)=2t^{2}x(t)$. Then, we have

$$\begin{array}{rl}y(t,t_0)& =2t^{2}x(t-t_0)\\ y(t-t_0)& =2(t-t_0{)}^{2}x(t-t_0)\end{array}$$

Since the two are not equal, the system is not time-invariant.

However, we have

$$\begin{array}{rl}ay(t)& =2t^{2}ax(t)\\ y_1(t)& =2t^2{x}_1(t)\\ y_2(t)& =2t^2{x}_2(t)\\ y_1(t)+y_2(t)& =2t^2(x_1(t)+x_2(t))\end{array}$$

so this system is linear.

Example 2

Let’s say we have $y(t)=3e^{3x(t)}$. Then:

$$\begin{array}{rl}y(t,t_0)& =3e^{3x(t-t_0)}\\ y(t-t_0)& =3e^{3x(t-t_0)}\end{array}$$

Thus, this is time invariant.

However, we have

$$3a{e}^{3x(t)}\ne 3e^{3ax(t)}$$

Thus this system is not linear.

Example 3

Let’s say we have

$$5t^2\frac{d^{2}y(t)}{d{t}^2}+4y(t)=2x(t)$$

Because of the $t^2$ term, this is time-varying.

Example 4

Let’s say we have

$$y[n]+5ny[n-1]=x[n-2]+x[n]$$

Because of the $n$ term, we know that this is not linear (similar to Example 1).

Form

A LTI system has the following form of state space model:

$$\begin{array}{rl}\dot{\mathbf{x}}& =\mathbf{Ax}(t)+\mathbf{b}u(t)\\ y(t)& =\mathbf{cx}(t)+du(t)\end{array}$$

where $\mathbf{A}\in {\mathbb{R}}^{n\times n}$, $\mathbf{b}\in {\mathbb{R}}^{n\times 1}$, $c\in {\mathbb{R}}^{1\times n}$, and $d\in \mathbb{R}$ are constant matrices.

State Machine Interpretation

Linear time-invariant systems can be seen as a state machine where $S={\mathbb{R}}^m,X={\mathbb{R}}^1,Y={\mathbb{R}}^n$, and $f$ and $g$ are linear functions of their input. In discrete time, they can be described as a linear difference equation, like

$$y[t]=3y[t-1]+6y[t-2]+5x[t]+3x[t-2]$$

where $y[t]$ is $y$ at time $t$. LTI systems can be implemented using state to store relevant previous input and output information.

Recurrent Neural Networks are a lot like a non-linear version of LTIs.