Complex (Cognitive) Systems  Autonomous University of the State of Morelos
work in progress...
This marvelous course by Prof. Markus Müller revealed like a hidden treasure in the most unexpected of places: a socialsciences and humanitiesoriented postgrad program, in random underfunded university (the cognitive science masters degree from the Autonomous State University of Morelos, Mexico). The aim of selfpublishing notes and assignments, and assembling them into a coherent whole, goes beyond giving everyone brave enough^{1} a shot to enjoy it. It also sits as a personal reminder of my interest at the intersection of mathematical modeling, biophysics and the mind.
the brain meets the operational definition of what a complex^{2} system is:
 hierarchical organisation, scalefree structure, manytomany relations among components, interconnectedness (recurrence)
 nonlinear behaviour, emergent properties (hard to determine from the observation of components alone, which number in the millions)
 chaotic to initial conditions (slight differences lead to widely different states)
 nonstationary measurements (dynamical system whose model constants/parameters actually vary; system doesn't stay in an attractor)
 phase transitions, equilibrium depends on critical states
 noisy measurements, plus some rather stochastic processes (spiking), whose measurable fluctuations may be difficult to distinguish from chaotic determinism
• dynamical systems basics
we'll distinguish between two kinds of systems:
 stochastic
 dynamical (deterministic). e.g. the harmonic oscillator:
• harmonic oscillator
consider the mechanics of a mass attached to a spring (no other forces involved, e.g. friction, gravity). let's say this is a very simple brain model (or of one of its components thereof) whose activity behaves like an oscillating variable:
from Newton's second law and Hooke's law:
$$F=ma=kx$$
therefore^{3}
$$m\frac{{d}^{2}x}{d{t}^{2}}=kx$$
$$\stackrel{\u0308}{x\left(t\right)}=\frac{k}{m}x$$
$$\stackrel{\u0308}{x\left(t\right)}={\omega}_{0}^{2}x$$
in order to solve the secondorder linear ordinary homogeneous differential equation, we partition it into a system of two firstorder equations. let $v=\stackrel{\u0307}{x}$ , therefore:
$$\{\begin{array}{c}\hfill \stackrel{\u0307}{v}={\omega}_{0}^{2}x\hfill \\ \hfill \stackrel{\u0307}{x}=v\hfill \\ \hfill \hfill \end{array}$$
$$\{\begin{array}{c}\hfill \stackrel{\u0307}{v}={\omega}_{0}^{2}\int v\hfill \\ \hfill \stackrel{\u0307}{x}=\int {\omega}_{0}^{2}x={\omega}_{0}^{2}\int x\hfill \\ \hfill \hfill \end{array}$$
notice that with the last substitutions both equations are the same, except in terms of different variables. both sin(ω_{0}t) and cos(ω_{0}t) satisfy it, because:
$$\frac{d}{dt}sin\left({\omega}_{0}t\right)={\omega}_{0}cos\left({\omega}_{0}t\right)={\omega}_{0}^{2}\int sin\left({\omega}_{0}t\right)dt$$
and $$\frac{d}{dt}cos\left({\omega}_{0}t\right)={\omega}_{0}sin\left({\omega}_{0}t\right)={\omega}_{0}^{2}\int cos\left({\omega}_{0}t\right)dt$$
more generally, solutions will conform to the linear superposition principle:
$$x\left(t\right)=asen\left({\omega}_{0}t\right)+bcos\left({\omega}_{0}t\right);\phantom{\rule{0.278em}{0ex}}a,b\in \mathbb{R}$$
any possible function of position on time will look sinusoidal, of varying frequency (depending on $\sqrt{k/m}$ ), amplitude and phase (a and b together). we rewrite it as:
$$x\left(t\right)=\sqrt{{a}^{2}+{b}^{2}}[\frac{a}{\sqrt{{a}^{2}+{b}^{2}}}sen\left({\omega}_{0}t\right)+\frac{b}{\sqrt{{a}^{2}+{b}^{2}}}cos\left({\omega}_{0}t\right)]=A[{c}_{1}sen\left({\omega}_{0}t\right)+{c}_{2}sen\left({\omega}_{0}t\right)]$$
this has the property that $1<={c}_{1},{c}_{2}<=1$ , and ${c}_{1}^{2}+{c}_{2}^{2}=1$ .
let ${\phi}_{0}$ be the initial phase. then $\exists {\phi}_{0}\phantom{\rule{0.278em}{0ex}}(\frac{a}{\sqrt{{a}^{2}+{b}^{2}}}=cos\left({\phi}_{0}\right)\wedge \frac{b}{\sqrt{{a}^{2}+{b}^{2}}}=cos\left({\phi}_{0}\right))$
so:
$$x\left(t\right)=A[cos\left({\phi}_{0}\right)sen\left({\omega}_{0}t\right)+sen\left({\phi}_{0}\right)sen\left({\omega}_{0}t\right)]=$$
$$A\left[sen({\omega}_{0}t+{\phi}_{0})\right]$$
graphically:
• initial conditions
finally, let's say the spring is totally relaxed at t=0, so $x\left(0\right)=A=A\left[sen\left({\phi}_{0}\right)\right]$ . this implies that ${\phi}_{0}=\pi /2$ :
$$x\left(t\right)=A\left[sen({\omega}_{0}t+\pi /2)\right]$$
• complex polar notation
prove that $z={e}^{i\phi}$ is the unit circle.
following Euler's formula, ${e}^{i\phi}=cos\left(\phi \right)+isin\left(\phi \right)\in (\mathbb{R}\to \u2102)$ . therefore $\left\overrightarrow{z}\right=\sqrt{{e}^{i\phi}{e}^{i\phi}}=\sqrt{{e}^{i\phi i\phi}}=\sqrt{{e}^{0}}=1.$ ^{4}
solutions of the form $A{e}^{\lambda t}$ include the aforementioned trigonometric solutions when λ, A ∈ ℂ. so the #original differential equation can be rewritten as:
$${\lambda}^{2}A{e}^{\lambda t}+{\omega}_{0}^{2}A{e}^{\lambda t}=0=({\lambda}^{2}+{\omega}_{0}^{2})A{e}^{\lambda t}$$
$$\u22a2{\lambda}^{2}+{\omega}_{0}^{2}=0$$
$$\u22a2\lambda =\sqrt{{\omega}_{0}^{2}}=\pm i{\omega}_{0}$$
the general solution becomes:
$$x\left(t\right)=A{e}^{i{\omega}_{0}t}+B{e}^{i{\omega}_{0}t}$$
if we restrict x(t) to the type ℝ→ℝ (position is real), it follows that x(t) equals its complex conjugate.
moreover, $\overline{x\left(t\right)}=\overline{A{e}^{i{\omega}_{0}t}+B{e}^{i{\omega}_{0}t}}=\overline{A{e}^{i{\omega}_{0}t}}+\overline{B{e}^{i{\omega}_{0}t}}=\overline{A}{e}^{i{\omega}_{0}t}+\overline{B}{e}^{i{\omega}_{0}t}$ . this implies that $A=\overline{B}$ and $B=\overline{A}$ . so:
$$x\left(t\right)=(a+ib)[cos\left({\omega}_{0}t\right)+isin\left({\omega}_{0}t\right)]+(aib)[cos\left({\omega}_{0}t\right)isin\left({\omega}_{0}t\right)]$$
$$=2acos\left(\omega t\right)2bsin\left(\omega t\right)\in (\mathbb{R}\to \mathbb{R})$$
compare this expression against the one using trigonometric functions.
• energy loss
(also see #dissipative systems and attractors)
now we will damp the oscillator by adding a term for friction, whose force is proportional to velocity (but in opposite direction):
$${F}_{T}=\sum F$$
$$\stackrel{\u0308}{x\left(t\right)}={\omega}_{0}^{2}x\left(t\right)2\beta \stackrel{\u0307}{x\left(t\right)}$$
assume a family of solutions of the form $x\left(t\right)=A{e}^{\lambda t}$ . therefore:
$${\lambda}^{2}A{e}^{\lambda t}+2\beta \lambda A{e}^{\lambda t}+{\omega}_{0}^{2}A{e}^{\lambda t}=0$$
$$A{e}^{\lambda t}({\lambda}^{2}+2\beta \lambda +{\omega}_{0}^{2})=0$$
$${\lambda}^{2}+2\beta \lambda ={\omega}_{0}^{2}$$
$${\lambda}^{2}+2\beta \lambda +{\beta}^{2}={\omega}_{0}^{2}+{\beta}^{2}={(\lambda +\beta )}^{2}$$
$$\lambda =\beta \pm i\sqrt{{\omega}_{0}^{2}{\beta}^{2}}$$
plug that back into the general solution:
$$x\left(t\right)=A{e}^{(\beta +i\sqrt{{\omega}_{0}^{2}{\beta}^{2}})t}+B{e}^{(\beta i\sqrt{{\omega}_{0}^{2}{\beta}^{2}})t}$$
$$=A{e}^{\beta t}{e}^{i\sqrt{{\omega}_{0}^{2}{\beta}^{2}}t}+B{e}^{\beta t}{e}^{i\sqrt{{\omega}_{0}^{2}{\beta}^{2}}t}$$
$$={e}^{\beta t}(A{e}^{i\sqrt{{\omega}_{0}^{2}{\beta}^{2}}t}+B{e}^{i\sqrt{{\omega}_{0}^{2}{\beta}^{2}}t})$$
asymptotically, the system will inevitably evolve to the most entropic dynamic state. let's show this on a casebycase basis, taking note on the different (non)monotonic dynamics:
• underdamped: ω_{0} > β
let $\omega =\sqrt{{\omega}_{0}^{2}{\beta}^{2}}$ :
$$x\left(t\right)={e}^{\beta t}(A{e}^{i\omega t}+B{e}^{i\omega t})$$
by restricting x(t) to ℝ→ℝ, and from our previous knowledge of the simple harmonic oscillator solution:
$$\overline{x\left(t\right)}={e}^{\beta t}Dsin(\omega t+{\phi}_{0})$$
• ω_{0} = β
$${\lambda}_{1}=\beta \pm i\sqrt{0}=\beta ={\lambda}_{2}$$
we leverage the fact that amplitude isn't constant anymore:
$$x\left(t\right)=A\left(t\right){e}^{\beta t}$$
$$\stackrel{\u0307}{x}\left(t\right)=\stackrel{\u0307}{A}\left(t\right){e}^{\beta t}A\left(t\right)\beta {e}^{\beta t}$$
$$\stackrel{\u0308}{x}\left(t\right)=\stackrel{\u0308}{A}\left(t\right){e}^{\beta t}\stackrel{\u0307}{A}\left(t\right)\beta {e}^{\beta t}\stackrel{\u0307}{A}\left(t\right)\beta {e}^{\beta t}+A\left(t\right){\beta}^{2}{e}^{\beta t}$$
and the damped harmonic oscillator equation becomes:
$${e}^{\beta t}[\stackrel{\u0308}{A}2\stackrel{\u0307}{A}\beta +A{\beta}^{2}+2\beta \stackrel{\u0307}{A}2{\beta}^{2}A+{\beta}^{2}A]=0$$
$${e}^{\beta t}\left[\stackrel{\u0308}{A}\right]=0$$
$$\u22a2\stackrel{\u0308}{A}\left(t\right)=0$$
$$\u22a2\stackrel{\u0308}{A}\left(t\right)=0\leftrightarrow A\left(t\right)={a}_{1}+{a}_{2}t$$
i.e., amplitude is a linear function. so the solution must be of the form:
$$x\left(t\right)={a}_{1}{e}^{\beta t}+{a}_{2}t{e}^{\beta t}$$
graphically, the spring oscillates only once:
• overdamped: ω_{0} < β
$$\lambda =\beta \pm i\sqrt{{\omega}_{0}^{2}{\beta}^{2}}=\beta \pm {i}^{2}\sqrt{{\beta}^{2}{\omega}_{0}^{2}}\in \mathbb{R}$$
$$x\left(t\right)=A{e}^{\beta t}[A{e}^{\omega t}+B{e}^{\omega t}]$$
because $\beta >\omega $ , $A{e}^{\omega t}$ is overshadowed by $B{e}^{\omega t}$ . the plot never gets to complete an oscillation:
• energy input
the damped model will get augmented with the influence of external stimuli, driving our cognitiveunit model to activity. this is known as a driven harmonic oscillator. let's go with an engine rotating at frequency $\stackrel{\u0303}{\omega}$ and amplitude F, which in turn pushes and pulls the rest of the system:
$$\stackrel{\u0308}{x\left(t\right)}+2\beta \stackrel{\u0307}{x\left(t\right)}+{\omega}_{0}^{2}x\left(t\right)=Fcos\left(\stackrel{\u0303}{\omega}t\right)$$
the equation isn't homogeneous anymore. the solution is the sum of the general and particular solutions: the homogeneous one is of the type ${e}^{\beta t}$ and eventually dissipates. the particular one oscillates forever:
$${x}_{p}\left(t\right)=A{e}^{i\stackrel{\u0303}{\omega}t}$$
• resonance
starting from an exploration of the steady state ("final conditions"), we will show that amplitude is a function of the engine's frequency, and that there are privileged frequencies for which steadystate amplitude forms maxima or singularities: resonance frequencies.
$$\stackrel{\u0307}{{x}_{p}\left(t\right)}=i\stackrel{\u0303}{\omega}A{e}^{i\stackrel{\u0303}{\omega}t}$$
$$\stackrel{\u0308}{{x}_{p}\left(t\right)}={\stackrel{\u0303}{\omega}}^{2}A{e}^{i\stackrel{\u0303}{\omega}t}$$
so we rewrite the full differential equation:
$$[{\stackrel{\u0303}{\omega}}^{2}A+2\beta iA\stackrel{\u0303}{\omega}+{\omega}_{0}^{2}A]{e}^{i\stackrel{\u0303}{\omega}t}=F{e}^{i\stackrel{\u0303}{\omega}t}$$
$$[{\omega}_{0}^{2}{\stackrel{\u0303}{\omega}}^{2}+2\beta i\stackrel{\u0303}{\omega}]A=F$$
$$A\left(\stackrel{\u0303}{\omega}\right)=\frac{F}{[{\omega}_{0}^{2}{\stackrel{\u0303}{\omega}}^{2}+i2\beta \stackrel{\u0303}{\omega}]}$$
$$=\frac{F}{[({\omega}_{0}^{2}{\stackrel{\u0303}{\omega}}^{2})+i2\beta \stackrel{\u0303}{\omega}]}\frac{[({\omega}_{0}^{2}{\stackrel{\u0303}{\omega}}^{2})i2\beta \stackrel{\u0303}{\omega}]}{[({\omega}_{0}^{2}{\stackrel{\u0303}{\omega}}^{2})i2\beta \stackrel{\u0303}{\omega}]}$$
$$=\frac{F[({\omega}_{0}^{2}{\stackrel{\u0303}{\omega}}^{2})2\beta i\stackrel{\u0303}{\omega}]}{[{({\omega}_{0}^{2}{\stackrel{\u0303}{\omega}}^{2})}^{2}+{\left(2\beta \stackrel{\u0303}{\omega}\right)}^{2}]}$$
(...magic happens here...)
$$\leftA\left(\stackrel{\u0303}{\omega}\right)\right=\frac{F}{\sqrt{{({\omega}_{0}^{2}{\stackrel{\u0303}{\omega}}^{2})}^{2}+{\left(2\beta \stackrel{\u0303}{\omega}\right)}^{2}}}$$
• phase space
phase space contains all information of a dynamical system, with a different axis for each variable of interest. each point in it represents a state of the whole system, and a trajectory in this virtual space corresponds to the evolution of the system.
signal: disregarding noise, taking a series of measurements is equivalent to observing variation in a single dimension of phase space. graphically, this is a projection of the system upon some vector in phase space.
• Fourier analysis
• linear independence of Fourier terms
let ${v}_{n}\left(x\right)=\frac{1}{\sqrt{2a}}{e}^{i\frac{n\pi}{a}x}$ with $n=0,\pm 1,\pm 2,...$ be functions on the interval $[a,a]$ . prove that ${v}_{n}\xb7{v}_{m}={\delta}_{nm}$ , given the proper definition of the dot product. ${\delta}_{nm}=\{\begin{array}{cc}\hfill 1\hfill & \hfill \iff n=m\hfill \\ \hfill 0\hfill & \hfill \iff n\ne m\hfill \\ \hfill \hfill \end{array}$ is Dirac's delta distribution.
let ${V}_{n}\xb7{V}_{m}={\int}_{a}^{a}dx\phantom{\rule{0.278em}{0ex}}{V}_{n}^{\star}\left(x\right){V}_{m}\left(x\right)$ be the scalar product for functions of the vector basis defined by ${V}_{n}\left(x\right)$ .
proof by modus ponens follows:
 restatement of conditional premise
given the above definitions; if ${V}_{n}\xb7{V}_{m}={\delta}_{nm}$ , then ${V}_{n}\left(x\right)$ is an orthogonal basis. that is:
$$\forall n\forall m\phantom{\rule{0.278em}{0ex}}(({V}_{n}\xb7{V}_{n}=1)\wedge ({V}_{n}\xb7{V}_{m}=0;n\ne m)\Rightarrow {V}_{n}\left(x\right)\phantom{\rule{0.333em}{0ex}}\mathrm{\text{is orthogonal}})$$
 proof of the antecedent premise
indeed: $\forall n\forall m\phantom{\rule{0.278em}{0ex}}(({V}_{n}\xb7{V}_{n}=1)\wedge ({V}_{n}\xb7{V}_{m}=0))$
because:
$$\u22a2({\int}_{a}^{a}dx\phantom{\rule{0.278em}{0ex}}{V}_{n}^{\star}\left(x\right){V}_{n}\left(x\right)=1)\wedge ({\int}_{a}^{a}dx\phantom{\rule{0.278em}{0ex}}{V}_{n}^{\star}\left(x\right){V}_{m}\left(x\right)=0)$$
$$\u22a2({\int}_{a}^{a}dx\phantom{\rule{0.278em}{0ex}}\frac{1}{\sqrt{2a}}{e}^{in\pi x/a}\frac{1}{\sqrt{2a}}{e}^{in\pi x/a}=1)\wedge ({\int}_{a}^{a}dx\phantom{\rule{0.278em}{0ex}}\frac{1}{\sqrt{2a}}{e}^{in\pi x/a}\frac{1}{\sqrt{2a}}{e}^{im\pi x/a}=0)$$
$$\u22a2(\frac{1}{2a}{\int}_{a}^{a}dx\phantom{\rule{0.278em}{0ex}}{e}^{in\pi x/a}{e}^{in\pi x/a}=1)\wedge (\frac{1}{2a}{\int}_{a}^{a}dx\phantom{\rule{0.278em}{0ex}}{e}^{in\pi x/a}{e}^{im\pi x/a}=0)$$
$$\u22a2(\frac{1}{2a}{\int}_{a}^{a}dx\phantom{\rule{0.278em}{0ex}}{e}^{\left(in\pi x/a\right)\left(in\pi x/a\right)}=1)\wedge (\frac{1}{2a}{\int}_{a}^{a}dx\phantom{\rule{0.278em}{0ex}}{e}^{\left(in\pi x/a\right)\left(im\pi x/a\right)}=0)$$
$$\u22a2(\frac{1}{2a}{\int}_{a}^{a}dx\phantom{\rule{0.278em}{0ex}}{e}^{0}=1)\wedge (\frac{1}{2a}{\int}_{a}^{a}dx\phantom{\rule{0.278em}{0ex}}{e}^{i\pi x(nm)/a}=0)$$
from Euler's formula:
$$\u22a2(\frac{1}{2a}{\int}_{a}^{a}dx=1)\wedge (\frac{1}{2a}{\int}_{a}^{a}dx\phantom{\rule{0.278em}{0ex}}(cos\left(\frac{\pi x(nm)}{a}\right)+i\phantom{\rule{0.278em}{0ex}}sen\left(\frac{\pi x(nm)}{a}\right))=0)$$
$$\u22a2({\frac{x}{2a}}_{a}^{a}=1)\wedge (\frac{1}{2a}{\int}_{a}^{a}dx\phantom{\rule{0.278em}{0ex}}(cos\left(\frac{\pi x(nm)}{a}\right)+i\phantom{\rule{0.278em}{0ex}}sen\left(\frac{\pi x(nm)}{a}\right))=0)$$
from the integration of the odd function $sen\left(\frac{\pi x(nm)}{a}\right)$ over a zerocentered interval, $[a,a]$ :
$$\u22a2(\frac{a}{2a}\frac{a}{2a}=1)\wedge (\frac{1}{2a}{\int}_{a}^{a}dx\phantom{\rule{0.278em}{0ex}}(cos\left(\frac{\pi x(nm)}{a}\right)+0)=0)$$
$$\u22a2(1=1)\wedge (\frac{1}{2a}{\int}_{a}^{a}dx\phantom{\rule{0.278em}{0ex}}cos\left(\frac{\pi x(nm)}{a}\right)=0)$$
let $u=\frac{\pi x(nm)}{a}$ ; then $\frac{du}{dx}=\frac{\pi (mn)}{a}$ .
$$\u22a2\top \wedge (\frac{1}{2a}{\int}_{a}^{a}cos\left(u\right)\frac{\pi (nm)}{a}\phantom{\rule{0.278em}{0ex}}dx\phantom{\rule{0.278em}{0ex}}\frac{a}{\pi (nm)}=0)$$
$$\u22a2\top \wedge (\frac{a}{2a\pi (nm)}{\int}_{a}^{a}cos\left(u\right)\phantom{\rule{0.278em}{0ex}}du=0)$$
$$\u22a2\top \wedge (\frac{a}{2a\pi (nm)}{\left[sen\left(u\right)\right]}_{a}^{a}=0)$$
$$\u22a2\top \wedge (\frac{a}{2a\pi (nm)}[sen\left(\frac{\pi \left(a\right)(nm)}{a}\right)sen\left(\frac{\pi (a)(nm)}{a}\right)]=0)$$
note that $(nm)\in \mathbb{Z}$ . $\u22a2\forall n\forall m\phantom{\rule{0.278em}{0ex}}(sen\left(\pi (nm)\right)=0)$
$$\u22a2\top \wedge (\frac{a}{2a\pi (nm)}[00]=0)$$
$$\u22a2\top \wedge \top $$
 conclusion
$\u22a2{V}_{n}\left(x\right)$ is an orthogonal basis.
• Fourier transform
the Fourier series represents a periodic function as a discrete linear combination of sines and cosines . on the other hand, the Fourier transform is capable of representing a more general set of functions, not restricted to being periodic. how is that achieved?
(for convenience, we will start from the series in complex exponential notation)
$$f\left(x\right){f}_{0}=\sum _{n=1}^{\infty}[{a}_{n}cos\left(\frac{n\pi x}{a}\right)+{b}_{n}sen\left(\frac{n\pi x}{a}\right)]$$
$$=\sum _{\infty}^{\infty}\underset{{\alpha}_{n}}{\underset{\u23df}{\left(\frac{1}{2a}{\int}_{a}^{a}f\left(x\right){e}^{in\pi x/a}\phantom{\rule{0.278em}{0ex}}dx\right)}}{e}^{in\pi x/a}$$
if $f\left(x\right)$ weren't periodic, the series would be representing it only at $[a,a]$ . nonetheless, we can make the interval arbitrarily large:
$$=\underset{a\to \infty}{lim}\sum _{\infty}^{\infty}\underset{{\alpha}_{n}}{\underset{\u23df}{\left(\frac{1}{2a}{\int}_{a}^{a}f\left(x\right){e}^{in\pi x/a}\phantom{\rule{0.278em}{0ex}}dx\right)}}{e}^{in\pi x/a}$$
let ${k}_{n}=\frac{n\pi}{a}$ , then $\Delta k=\frac{\pi}{a}$ . substituting in the last expression yields:
$$=\underset{a\to \infty}{lim}\sum _{\infty}^{\infty}\Delta k{\left(\Delta k\right)}^{1}\left(\frac{1}{2a}\right)\left({\int}_{a}^{a}f\left(x\right){e}^{i{k}_{n}x}\phantom{\rule{0.278em}{0ex}}dx\right)\left({e}^{i{k}_{n}x}\right)$$
$$=\underset{a\to \infty}{lim}\sum _{\infty}^{\infty}\left(\frac{\pi}{a}\right)\left(\frac{a}{2a\pi}\right)\left({\int}_{a}^{a}f\left(x\right){e}^{i{k}_{n}x}\phantom{\rule{0.278em}{0ex}}dx\right)\left({e}^{i{k}_{n}x}\right)$$
$$=\underset{\Delta k\to 0}{lim}\phantom{\rule{0.278em}{0ex}}\frac{1}{2\pi}\sum _{\infty}^{\infty}\Delta k\left({\int}_{\infty}^{\infty}f\left(x\right){e}^{i{k}_{n}x}\phantom{\rule{0.278em}{0ex}}dx\right)\left({e}^{i{k}_{n}x}\right)$$
which is a Riemman sum for variable ${k}_{n}$ :
$$=\frac{1}{2\pi}{\int}_{\infty}^{\infty}d{k}_{n}\phantom{\rule{0.278em}{0ex}}\left({\int}_{\infty}^{\infty}f\left(x\right){e}^{i{k}_{n}x}\phantom{\rule{0.278em}{0ex}}dx\right)\left({e}^{i{k}_{n}x}\right)$$
$$={\left(\sqrt{\frac{1}{2\pi}}\right)}^{2}{\int}_{\infty}^{\infty}\left({\int}_{\infty}^{\infty}f\left(x\right){e}^{i{k}_{n}x}\phantom{\rule{0.278em}{0ex}}dx\right){e}^{i{k}_{n}x}\phantom{\rule{0.278em}{0ex}}d{k}_{n}$$
$$=\underset{\mathrm{\text{Inverse Fourier transform}}}{\underset{\u23df}{\frac{1}{\sqrt{2\pi}}{\int}_{\infty}^{\infty}\underset{\mathrm{\text{Fourier transform}}}{\underset{\u23df}{\left(\frac{1}{\sqrt{2\pi}}{\int}_{\infty}^{\infty}f\left(x\right){e}^{i{k}_{n}x}\phantom{\rule{0.278em}{0ex}}dx\right)}}{e}^{i{k}_{n}x}\phantom{\rule{0.278em}{0ex}}d{k}_{n}}}$$
• properties
which symmetry properties does H(f) has if h(t) is:
 odd real
 odd imaginary
 odd complex
and which symmetry properties does h(t) has if H(f ) is:
 odd real
 even complex
 even imaginary
recall that:
$$H\left(f\right)={\int}_{\infty}^{\infty}[r\left(t\right)+ij\left(t\right)][cos\left(2\pi ft\right)+isen\left(2\pi ft\right)]dt$$
$$={\int}_{\infty}^{\infty}[r\left(t\right)cos\left(2\pi ft\right)+j\left(t\right)sen\left(2\pi ft\right)]dt+i{\int}_{\infty}^{\infty}[j\left(t\right)cos\left(2\pi ft\right)r\left(t\right)sen\left(2\pi ft\right)]dt;$$
$$h\left(t\right)={\int}_{\infty}^{\infty}[R\left(f\right)+iJ\left(f\right)][cos\left(2\pi ft\right)+isen\left(2\pi ft\right)]dt$$
$$={\int}_{\infty}^{\infty}[R\left(f\right)cos\left(2\pi ft\right)+J\left(f\right)sen\left(2\pi ft\right)]dt+i{\int}_{\infty}^{\infty}[J\left(f\right)cos\left(2\pi ft\right)R\left(f\right)sen\left(2\pi ft\right)]dt.$$
 odd real h(t)
we get rid of the imaginary part of H(f):
$$H\left(f\right)={\int}_{\infty}^{\infty}[r\left(t\right)cos\left(2\pi ft\right)+0)]dt+i{\int}_{\infty}^{\infty}[0r\left(t\right)sen\left(2\pi ft\right)]dt$$
from integration of an odd function (r(t)cos(2πft)) at a symmetric interval:
$$H\left(f\right)=0+i{\int}_{\infty}^{\infty}\left[r\left(t\right)sen\left(2\pi ft\right)\right]dt$$
$sen\left(2\pi ft\right)$ is odd. $\therefore H\left(f\right)$ is odd and imaginary.
 odd imaginary h(t)
same reasoning:
$$H\left(f\right)={\int}_{\infty}^{\infty}[0+j\left(t\right)sen\left(2\pi ft\right)]dt+i{\int}_{\infty}^{\infty}[j\left(t\right)cos\left(2\pi ft\right)0]dt$$
$$H\left(f\right)={\int}_{\infty}^{\infty}\left[j\left(t\right)sen\left(2\pi ft\right)\right]dt+0$$
$sen\left(2\pi ft\right)$ is odd. $\therefore H\left(f\right)$ is odd and real.
 odd h(t)
$$H\left(f\right)={\int}_{\infty}^{\infty}[0+j\left(t\right)sen\left(2\pi ft\right)]dt+i{\int}_{\infty}^{\infty}[0r\left(t\right)sen\left(2\pi ft\right)]dt$$
$sen\left(2\pi ft\right)$ is odd. $\therefore H\left(f\right)$ is odd and complex.
 odd real H(f)
$$h\left(t\right)={\int}_{\infty}^{\infty}[R\left(f\right)cos\left(2\pi ft\right)+0]dt+i{\int}_{\infty}^{\infty}[0R\left(f\right)sen\left(2\pi ft\right)]dt$$
$$h\left(t\right)=0+i{\int}_{\infty}^{\infty}R\left(f\right)sen\left(2\pi ft\right)dt$$
$sen\left(2\pi ft\right)$ is odd. $\therefore h\left(t\right)$ is imaginary and odd.
 complex even H(f)
$$h\left(t\right)={\int}_{\infty}^{\infty}[R\left(f\right)cos\left(2\pi ft\right)+0]dt+i{\int}_{\infty}^{\infty}[J\left(f\right)cos\left(2\pi ft\right)0]dt$$
$cos\left(2\pi ft\right)$ is even. $\therefore h\left(t\right)$ is even and complex.
 imaginary even H(f)
$$h\left(t\right)={\int}_{\infty}^{\infty}[0+J\left(f\right)sen\left(2\pi ft\right)]dt+i{\int}_{\infty}^{\infty}[J\left(f\right)cos\left(2\pi ft\right)0]dt$$
$$h\left(t\right)=0+i{\int}_{\infty}^{\infty}J\left(f\right)cos\left(2\pi ft\right)dt$$
$cos\left(2\pi ft\right)$ is even. $\therefore h\left(t\right)$ is even and imaginary.
• uncertainty principle
the more shortlived the wave in the time domain, the more widespread its power spectrum (a.k.a. frequency spectrum). vice versa.
• convolution theorem
$y\left(t\right)=x\left(t\right)\ast h\left(t\right)={\int}_{\infty}^{\infty}x\left(\tau \right)h(t\tau )d\tau $ is the convolution of functions x(t) y h(t). F[y(t)] denotes its Fourier transform:
$$F\left[y\left(t\right)\right]={\int}_{\infty}^{\infty}y\left(t\right){e}^{2\pi ift}dt$$
$$={\int}_{\infty}^{\infty}\left[{\int}_{\infty}^{\infty}x\left(\tau \right)h(t\tau )d\tau \right]{e}^{2\pi ift}dt$$
$$={\int}_{\infty}^{\infty}x\left(\tau \right)\left[{\int}_{\infty}^{\infty}h(t\tau ){e}^{2\pi ift}dt\right]d\tau $$
let u=tτ.
$$={\int}_{\infty}^{\infty}x\left(\tau \right)\left[{\int}_{\infty}^{\infty}h\left(u\right){e}^{2\pi if(u+\tau )}du\right]d\tau $$
$$={\int}_{\infty}^{\infty}x\left(\tau \right)\left[{\int}_{\infty}^{\infty}h\left(u\right){e}^{2\pi ifu}{e}^{2\pi if\tau}du\right]d\tau $$
$$=\left[{\int}_{\infty}^{\infty}x\left(\tau \right){e}^{2\pi if\tau}d\tau \right]\left[{\int}_{\infty}^{\infty}h\left(u\right){e}^{2\pi ifu}du\right]$$
$$=X\left(f\right)H\left(f\right)$$
therefore:
$$x\left(t\right)\ast h\left(t\right)={F}^{1}\left[X\left(f\right)H\left(f\right)\right]$$
timedomain convolution is equivalent to frequencydomain product!
• crosscorrelation theorem
${z}_{x,h}\left(t\right)={\int}_{\infty}^{\infty}x\left(\tau \right)h(t+\tau )d\tau $ is the correlation (covariance, actually) of functions x(t) y h(t) as a function of a time lag between them. F[z(t)] denotes its Fourier transform:
$$F\left[z\left(t\right)\right]={\int}_{\infty}^{\infty}z\left(t\right){e}^{2\pi ift}dt$$
$$={\int}_{\infty}^{\infty}\left[{\int}_{\infty}^{\infty}x\left(\tau \right)h(t+\tau )d\tau \right]{e}^{2\pi ift}dt$$
$$={\int}_{\infty}^{\infty}x\left(\tau \right)\left[{\int}_{\infty}^{\infty}h(t+\tau ){e}^{2\pi ift}dt\right]d\tau $$
let u=t+τ.
$$={\int}_{\infty}^{\infty}x\left(\tau \right)\left[{\int}_{\infty}^{\infty}h\left(u\right){e}^{2\pi if(u\tau )}du\right]d\tau $$
$$={\int}_{\infty}^{\infty}x\left(\tau \right)\left[{\int}_{\infty}^{\infty}h\left(u\right){e}^{2\pi ifu}{e}^{2\pi if\tau}du\right]d\tau $$
$$=\left[{\int}_{\infty}^{\infty}x\left(\tau \right){e}^{2\pi if\tau}d\tau \right]\left[{\int}_{\infty}^{\infty}h\left(u\right){e}^{2\pi ifu}du\right]$$
$$={X}^{*}\left(f\right)H\left(f\right)$$
therefore:
$${z}_{x,h}={F}^{1}\left[{X}^{*}\left(f\right)H\left(f\right)\right]$$
crosscorrelation is equivalent to frequency domain product (with conjugate X(f)).
• discrete Fourier transform
the computation of a DFT is done segment by segment. the signal is multiplied by some "window" function (e.g. a square pulse) which is zero outside the window. this is known as windowing.
the effect of using a finite window is that unexisting frequencies appear at the power spectrum (around the original frequencies). this phenomenon is known as leakage. the explanation is evident from the #convolution theorem: one can use similar reasoning to show that $h\left(t\right)x\left(t\right)={F}^{1}[H\left(f\right)\ast X\left(f\right)]$ . therefore, multiplication by a window will induce a convolution in the frequency domain.
low frequencies (and therefore #nonstationary systems) are the most problematic to compute a DFT for, since the window may not be long enough to capture them. there's no single #test for stationarity. it can be tested for nonetheless, measuring the effect of varying window sizes on different parameters (centrality and spread statistics, Lyapunov exponent, etc.).
• Whittaker–Shannon (sinc) interpolation
what function should we convolve the signal's Fourier transform with in order to obtain the same result as windowing it with a rectangular function? we will analyze the closelyrelated case of a square pulse window:
calculate the first three terms of the Fourier series for function $f\left(x\right)=\{\begin{array}{cc}\hfill 1\hfill & \hfill \iff 2n\pi \le x\le (2n+1)\pi \hfill \\ \hfill 1\hfill & \hfill \iff (2n+1)\pi \le x\le (2n+2)\pi \hfill \\ \hfill \hfill \end{array}$ , with $n=0,\pm 1,\pm 2,...$
in terms of the Fourier series:
$$f\left(x\right)=\underset{{f}_{0}}{\underset{\u23df}{\frac{1}{2a}{\int}_{a}^{a}f\left(x\right)dx}}+\sum _{n=1}^{\infty}[\underset{{a}_{n}}{\underset{\u23df}{\left(\frac{1}{a}{\int}_{a}^{a}dx\phantom{\rule{0.278em}{0ex}}f\left(x\right)cos\left(\frac{n\pi x}{a}\right)\right)}}cos\left(\frac{n\pi x}{a}\right)+\underset{{b}_{n}}{\underset{\u23df}{\left(\frac{1}{a}{\int}_{a}^{a}dx\phantom{\rule{0.278em}{0ex}}f\left(x\right)sen\left(\frac{n\pi x}{a}\right)\right)}}sen\left(\frac{n\pi x}{a}\right)]$$
from the integration of odd functions ( $f\left(x\right)cos\left(\frac{n\pi x}{a}\right)$ and $f\left(x\right)$ ) at symmetrical interval ( $[a,a]$ ):
$$\u22a2f\left(x\right)=\underset{{f}_{0}}{\underset{\u23df}{0}}+\sum _{n=1}^{\infty}[\underset{{a}_{n}}{\underset{\u23df}{\left(0\right)}}\left(cos\left(\frac{n\pi x}{a}\right)\right)+\underset{{b}_{n}}{\underset{\u23df}{\left(\frac{1}{a}{\int}_{a}^{a}dx\phantom{\rule{0.278em}{0ex}}f\left(x\right)sen\left(\frac{n\pi x}{a}\right)\right)}}sen\left(\frac{n\pi x}{a}\right)]$$
$$\u22a2f\left(x\right)=\sum _{n=1}^{\infty}\underset{{b}_{n}}{\underset{\u23df}{\left(\frac{1}{a}{\int}_{a}^{a}dx\phantom{\rule{0.278em}{0ex}}f\left(x\right)sen\left(\frac{n\pi x}{a}\right)\right)}}sen\left(\frac{n\pi x}{a}\right)$$
we will consider the interval $[a,a]=[\pi ,\pi ]$ , which includes a full period of $f\left(x\right)sen\left(n\pi x/\pi \right)$ for any n. therefore:
$$\u22a2f\left(x\right)=\sum _{n=1}^{\infty}\underset{{b}_{n}}{\underset{\u23df}{\left(\frac{1}{\pi}{\int}_{\pi}^{\pi}dx\phantom{\rule{0.278em}{0ex}}f\left(x\right)sen\left(\frac{n\pi x}{\pi}\right)\right)}}sen\left(\frac{n\pi x}{\pi}\right)$$
$$\u22a2f\left(x\right)=\sum _{n=1}^{\infty}\underset{{b}_{n}}{\underset{\u23df}{\frac{1}{\pi}({\int}_{\pi}^{0}dx\phantom{\rule{0.278em}{0ex}}(1)sen\left(nx\right)+{\int}_{0}^{\pi}dx\phantom{\rule{0.278em}{0ex}}(+1)sen\left(nx\right))}}sen\left(nx\right)$$
$$\u22a2f\left(x\right)=\sum _{n=1}^{\infty}\underset{{b}_{n}}{\underset{\u23df}{\frac{1}{\pi}(\frac{1}{n}{\left[cos\left(nx\right)\right]}_{\pi}^{0}\frac{1}{n}{\left[cos\left(nx\right)\right]}_{0}^{\pi})}}sen\left(nx\right)$$
$$\u22a2f\left(x\right)=\sum _{n=1}^{\infty}\underset{{b}_{n}}{\underset{\u23df}{\frac{1}{n\pi}(cos\left(0\right)cos(n\pi )cos\left(n\pi \right)+cos\left(0\right))}}sen\left(nx\right)$$
$$\u22a2f\left(x\right)=\sum _{n=1}^{\infty}\underset{{b}_{n}}{\underset{\u23df}{\frac{1}{n\pi}(2cos\left(0\right)2cos\left(n\pi \right))}}sen\left(nx\right)$$
$$\u22a2f\left(x\right)=\sum _{n=1}^{\infty}\underset{{b}_{n}}{\underset{\u23df}{\frac{1}{n\pi}(22cos\left(n\pi \right))}}sen\left(nx\right)$$
$$\u22a2f\left(x\right)=\sum _{n=1}^{\infty}\underset{{b}_{n}}{\underset{\u23df}{\frac{1}{n\pi}\left(\{\begin{array}{cc}\hfill 22\hfill & \hfill \iff n\phantom{\rule{0.333em}{0ex}}\mathrm{\text{even}}\hfill \\ \hfill 2+2\hfill & \hfill \iff n\phantom{\rule{0.333em}{0ex}}\mathrm{\text{odd}}\hfill \\ \hfill \hfill \end{array}\right)}}sen\left(nx\right)$$
$$\u22a2f\left(x\right)=\sum _{n=1}^{\infty}\{\begin{array}{cc}\hfill 0\hfill & \hfill \iff n\phantom{\rule{0.333em}{0ex}}\mathrm{\text{even}}\hfill \\ \hfill \frac{4}{n\pi}sen\left(nx\right)\hfill & \hfill \iff n\phantom{\rule{0.333em}{0ex}}\mathrm{\text{odd}}\hfill \\ \hfill \hfill \end{array}$$
$$\u22a2f\left(x\right)=\sum _{n=0}^{\infty}\frac{4}{\pi (2n+1)}sen\left(x(2n+1)\right)$$
$$=\frac{4}{\pi}\sum _{n=0}^{\infty}\frac{sen\left(x(2n+1)\right)}{2n+1}$$
$$\u22a2f\left(x\right)=\underset{{f}_{0}}{\underset{\u23df}{0}}+\frac{4}{\pi}sen\left(x\right)+0+\frac{4}{3\pi}sen\left(3x\right)+0+\frac{4}{5\pi}sen\left(5x\right)+...$$
f(x) could be just a discrete sample of a continuous signal, but because convolution with sinc(x) produces a continuous function (a sum of sine waves), the operation will fill in all the missing time points.
• NyquistShannon sampling theorem
continuous functions of bounded frequency bandwidth ("bandlimited") contain a bounded amount of information. they can be perfectly represented using a function of countable (i.e. discrete) sets and an interpolation operation.
the Nyquist frequency is the maximum sampling frequency at which the digitization of a continuous signal still losses fidelity. that is, aliasing occurs:
$${f}_{Nyquist}=2\phantom{\rule{0.278em}{0ex}}{f}_{max}$$
sampling frequency should be greater than twice the maximum frequency in the power spectrum of the original signal.
undersampling results in the unconsidered high frequencies aliasing (adding up) over ${f}_{Nyquist}(f{f}_{Nyquist})$ at the frequency domain. oversampling is harmless, beyond being a waste of disk space.
• testing for undersampling
what if the maximum frequency isn't known in advance? it is possible to test that the maximum frequency has been considered, because aliasing will make two power spectra of the same signal look dissimilar under two different sampling frequencies:
 register using two sampling frequencies, one greater than the one to test for the Nyquist criterion.
 compute both power spectra ( ${\leftF\left[x\left(t\right)\right]\right}^{2}$ ).
 if they are simply scaled versions of one another, aliasing didn't occur. i.e. the ratio between two features (e.g. maxima) should stay constant.
• other timeseries topics
• stochastic correlations
suppose the signal is generated by an stochastic process. if low frequencies dominate the power spectrum, then the process is nonstationary (timeseries mean value will be trending over long periods of time) and autocorrelation will be large.
corollary: even two independent sources of noise can display high statistical dependence.
averaging the time series would be no good. prediction should take autocovariance times into account, detrending methods are also available.
• stationarity

strict/particular stationarity: system's parameters are truly constant. too hard a requirement, since we must know the system's differential equations in advance.

weak/general stationarity: empirical transition probabilities in the F map stay constant. affected by lowband frequencies. alternatively, 1st moment and autocovariance stay constant and finite. this requires greater data availability, because short acquisition times may not be enough to establish statistical relevance.
• linear prediction
suppose there's a time series $\overrightarrow{s}=({s}_{1},{s}_{2},...,{s}_{N})$ whose value ${s}_{N+1}$ we want to predict. in a linear prediction algorithm we define ${\hat{s}}_{N+1}$ as a linear combination of the last m points:
$${\hat{s}}_{N+1}=\sum _{j=1}^{m}{a}_{j}{s}_{nm+j}$$
it's clear that there are as many linear combinations of the last m points as there are permutations (with substitution) of coefficients ${a}_{j}$ . in order to find the optimal coefficients we try to minimize the mean square error predicting the previous m  1 values:
$$\sum _{n=m}^{N1}{({\hat{s}}_{n+1}{s}_{n+1})}^{2}$$
• crossvalidation
 insample accuracy: estimated from predictions made for the same data segment used during training/optimisation phase. it doesn't reflect the algorithm's true predictive power (overestimate), since tests aren't predictions but "postdictions".
 outsample accuracy: error estimated from true predictions, using the values of data never "seen" before by our predictive model.
• phasespace methods
• dissipative systems and attractors
dissipative system: on average, the volume of the manifold describing the state trajectory contracts under dynamics, after certain initial conditions. alternatively, it can be said that #energy loss creates an attractor for that system, reducing the set of possible states that it can take with time. nonetheless, the system still operates far from thermodynamic equilibrium.
in order to verify the presence of an attractor, the L1 metric of the determinant of the system's Jacobian (i.e., of the state transition function's (F) partial derivatives) must be smaller than 1, on average:
$$\leftdet\left(\left(\begin{array}{ccc}\hfill \frac{\partial {f}_{1}}{\partial {x}_{1}}\hfill & \hfill \cdots \hfill & \hfill \frac{\partial {f}_{1}}{\partial {x}_{n}}\hfill \\ \hfill \vdots \hfill & \hfill \hfill & \hfill \vdots \hfill \\ \hfill \frac{\partial {f}_{n}}{\partial {x}_{1}}\hfill & \hfill \cdots \hfill & \hfill \frac{\partial {f}_{n}}{\partial {x}_{n}}\hfill \\ \hfill \hfill \end{array}\right)\right)\right<1$$
this means that the transformation's scaling factor produced by change on the different directions is a contracting one. visually, if phase space contains an attractor then the system is dissipative.
• visual inspection
 Poincaré section: allows to tell whether the trajectory behavior is classically deterministic, chaotic deterministic or random. it amounts to finding the intersection between trajectories and a secant plane.
 stroboscopic map: graphical representation of Poincaré's section technique for attractors. section is adjusted until the scatter plot is maximally decorrelated. also used for qualitative detection of strange/fractal attractors.
 "phase portrait": also known as delay representation. it is a bidimensional representation of any mdimensional phase space. each point is given by a pair of subsequent states in phase space (see also #TakensWhitney embedding theorem). it eases the visualisation of multidimensional systems and helps identify determinism, periodicity, linearity and chaos.
• TakensWhitney embedding theorem
it is possible to reconstruct a phase space manifold that is isomorphic to the original one, even when all information we have is a time series, through the method known as delay reconstruction.
requirements:
 stationary data
 lowdimensional system
 negligible noise
the basic idea is that for each subset of m equidistant points in the signal, there's an associated unique point in mdimensional phase space. each value in this subset of the time series becomes a component of the state vector:
note that the closer ${s}_{1}$ and ${s}_{2}$ are, the more correlated the reconstructed attractor; rendering it less useful:
the spacing parameter among points in the time series is called the time lag τ. to set an adequate value for τ, we want τ > L; with L being the autocovariance length. that is, we want the time between two measurements to be long enough so that they aren't correlated anymore.
finding the right number m of dimensions involves increasing the number of embedding components per point, until the manifold can't unfold anymore. we say there are false neighbors (and therefore that the embedding isn't highdimensional enough) whenever an increase in m separates seemingly close points considerably in reconstructed phase space.
• nonlinear prediction
unlike the #linear prediction algorithm for time series; a nonlinear algorithm doesn't use the last m points combined. rather, it builds upon the intuition that the most relevant states for making a prediction some time Δn after the present are the ones that most resembled the current state. it then averages those similar states after Δn:
$${\hat{s}}_{N+\Delta n}=\frac{1}{\left\mathcal{U}\left(\overrightarrow{{s}_{N}}\right)\right}\sum _{\overrightarrow{{s}_{n}}\in \mathcal{U}\left(\overrightarrow{{s}_{N}}\right)}{s}_{n+\Delta n}$$
where $\left\mathcal{U}\left(\overrightarrow{{s}_{N}}\right)\right$ denotes the cardinality of the current state vector's neighborhood (in phase space).
• as stationarity test

divide the series in intervals so short so as to have lineal segments.

predict and crossvalidate using all possible segment pairs.

if some training segments display unusually different performance, then they don't generalise to the whole series. therefore we can conclude that the attractor changed as we swapped training segments; and the time series isn't stationary.
• as a noise smoother
suppose the measured time series is the sum of its true value and noise:
$${s}_{n}={x}_{n}+{z}_{n}.$$
where ${z}_{n}$ is the random component. also suppose ${x}_{n}$ and ${z}_{n}$ are independent, and that temporal correlation is short.
we want to substitute ${s}_{n}$ with ${\hat{s}}_{n}$ . starting from the #nonlinear prediction algorithm:

in phase space, use trajectories which are close to the current one (both in the past and future portions), since the points inside that neighborhood belong to an ndimensional sphere with center at the vector state ${s}_{n}$ under consideration.

apply the nonlinear prediction algorithm afterwards: average the vectors in the neighborhood. thus the stochastic component will be largely cancelled out.
$${\hat{s}}_{{n}_{0}m/2}=\frac{1}{\left\mathcal{U}\left(\overrightarrow{{s}_{{n}_{0}}}\right)\right}\sum _{\overrightarrow{{s}_{n}}\in \mathcal{U}\left(\overrightarrow{{s}_{{n}_{0}}}\right)}{s}_{nm/2}$$
• chaos theory
• Lyapunov coefficient
quantifies chaos levels in a dynamical system. it is the exponent in the exponential divergence measurement of phase space. the system has as many Lyapunov exponents as dimensions, positive ones are chaotic dimensions:
Lyapunov coefficient  kind of attractor 

λ < 0  stable fixed point 
λ = 0  stable limit cycle 
λ > 0  chaos 
λ → ∞  noise 
• estimation from time series
it is advisable to perform a #phase portrait beforehand, so as to rule out that the system is nondeterministic.
$$\hat{\lambda}\left(\Delta t\right)=\frac{1}{N}\sum _{{n}_{0}=1}^{N}ln\left(\frac{1}{\left\mathcal{U}\left(\overrightarrow{{s}_{{n}_{0}}}\right)\right}\phantom{\rule{0.278em}{0ex}}\sum _{\overrightarrow{{s}_{n}}\in \mathcal{U}\left(\overrightarrow{{s}_{{n}_{0}}}\right)}{s}_{{n}_{0}+{\Delta}_{n}}{s}_{n+{\Delta}_{n}}\right)$$
caveats:
 embedding: if the #embedding dimension is too low divergence will be underestimated because of false neighbors.
 noise level: overestimated because of noise
 #stochastic correlations: if autocorrelation is big, the attractor will look compressed along a diagonal, underestimating true divergence
• fractal dimension
strange attractors are the flagship of chaos: trajectories are still deterministic (no crossroads) yet noncyclic (exact knowledge is required to tell accurately which trajectory the system is in). since some chaotic systems are also #dissipative, all of this implies that a subset of infinitelydense phase space is being relentlessly filled by a line of infinite detail.
• box counting
box counting is one of many methods to assess scaling property of fractals.
let ε be the side of each box (ndimensional squares), which are in turn used to segment the figure. then let N(ε) be the number of boxes. note that:
$$\underset{N\left(\epsilon \right)\to \infty}{lim}\phantom{\rule{0.278em}{0ex}}N\left(\epsilon \right)={\left(\frac{1}{\epsilon}\right)}^{D}$$
D being the boxcounting fractal dimension.
$$\u22a2\underset{N\left(\epsilon \right)\to \infty}{lim}\phantom{\rule{0.278em}{0ex}}ln\left(N\left(\epsilon \right)\right)=D\phantom{\rule{0.278em}{0ex}}ln\left(1/\epsilon \right)$$
$$\u22a2D=\underset{N\left(\epsilon \right)\to \infty}{lim}\phantom{\rule{0.278em}{0ex}}\frac{ln\left(N\left(\epsilon \right)\right)}{ln\left(1/\epsilon \right)}.$$
Example:
N(ε)  ε 

3^{0}  2^{0} 
3^{1}  2^{1} 
3^{2}  2^{2} 
...  ... 
3^{n}  2^{n} 
$$D=\underset{n\to \infty}{lim}\phantom{\rule{0.278em}{0ex}}\frac{ln\left({3}^{n}\right)}{ln\left({2}^{n}\right)}=\underset{n\to \infty}{lim}\phantom{\rule{0.278em}{0ex}}\frac{n\phantom{\rule{0.278em}{0ex}}ln\left(3\right)}{n\phantom{\rule{0.278em}{0ex}}ln\left(2\right)}=\frac{ln\left(3\right)}{ln\left(2\right)}=1.5849...$$
• correlation dimension
the correlation dimension can be used as an estimator of an attractor's dimension.(under certain circumstances), including fractal dimensions.
the idea now isn't to take the ratio of the number of boxes and their length ε in the limit to infinity. instead, we take the ratio of points (among all possible valid pairs) which fall within the neighborhood (defined by a radius of length ε) of some other point:
$$D=\underset{\epsilon \to 0}{lim}\phantom{\rule{0.278em}{0ex}}\underset{N\to \infty}{lim}\phantom{\rule{0.278em}{0ex}}\frac{\partial ln\phantom{\rule{0.278em}{0ex}}C(\epsilon ,N)}{\partial ln\epsilon};$$
where C(N, ϵ) is the correlation sum:
$$C(N,\u03f5)=\frac{2}{\left(N\right)(N1)}\sum _{i=1}^{N}\phantom{\rule{0.278em}{0ex}}\sum _{j=i+1+{n}_{min}}^{N}\mathit{\Theta}(\u03f5\left\right\overrightarrow{{s}_{i}}\overrightarrow{{s}_{j}}\left\right).$$
𝛩 is Heaviside's step function
• estimation from time series
given the lack of a phase space, it is necessary to #reconstruct it first. the right choice of delay τ during reconstruction of embedded vectors provides a suitable delay to estimate correlation dimension.
the aforementioned estimator of C(m, ϵ) is biased to underestimate the dimension, because data that are close in a trajectory of phase space tend to retain some temporal correlation.
the corrected estimator of C(m, ϵ) excludes sufficientlyclose vector pairs (in phase space); which usually correspond to subsequent vectors in the time series:
$$C(m,\u03f5)=\frac{2}{(N{n}_{min})(N{n}_{min}1)}\sum _{i=1}^{N}\phantom{\rule{0.278em}{0ex}}\sum _{j=i+1+{n}_{min}}^{N}\mathit{\Theta}(\u03f5\left\right\overrightarrow{{s}_{i}}\overrightarrow{{s}_{j}}\left\right)$$
with ${n}_{min}={t}_{min}/\Delta t$ . ${t}_{min}$ doesn't necessarily have to be the average correlation time.
the following plots from Kantz & Schreiber's book^{5} come from a maplike system. for a ${t}_{min}=500\phantom{\rule{0.278em}{0ex}}steps$ only 2.5% of all pairs is lost in the corrected correlation dimension estimator. this is statistically negligible.
possible error sources:
 the correlation is a welldefined construction (for finite data sets), whereas the correlation dimension is just an extrapolation that must be handled with care.
 it isn't enough to observe fractal scaling for a single choice of m. saturation should also occur for many values.
 if measurements are too noisy, a scaling regime may never be observed. the dimension estimate can be as big as m whenever ϵ is small enough, so that only noise is captured by it.
 noise affects scales 3 orders of magnitude as big as its own level. in order to observe a scaling region, noise should be at most 1/8 of the whole attractor.
 it is therefore advisable to use a #noise reduction algorithm in advance.
 certain stochastic systems may pass for lowdimension deterministic systems.
 lowresolution digitization will make the estimate look smaller
 selfsimilarity may never be observed, even for goodquality deterministic data, depending on the manifold's largescale structure.
• minimum delay
a spacetime separation plot shows the proportion of points inside the neighborhood as a function of Δt and ϵ. we hope to find some value of Δt after which increments in ϵ don't add extra points. such delay marks the ideal choice of Δt when computing the correlation sum.
this is more rigorous a technique than using the autocorrelation distance.
• as a noise meter
figures 19 and 20 introduced the value ε that marks the limit between noise and scaling regimes. we found a resolution limit in the data, when neighborhoods were so tiny that they only capture randomly distributed points in all directions.
in a nutshell, given that ε is a distance in phase space units, we can define the noise level as the ratio between ε and the attractor's "size".
• neuroscience applications
 how not to use it:
Lehnertz and Elger (1998)^{6} analysed localized intracranial EEG data from 16 epilepsy patients. they claimed that a loss of complexity in brain activity should result from the synchronization of pathological cells before and during seizures.
linear techniques can only predict seizures a few seconds in advance. here the intention is to use the correlation dimension estimate as a complexity detector, even though it cannot actually estimate the fractal dimension for these data.
a state called state 2 coincides with preictal synchronization some 10 to 30 minutes in advance to the seizure. nothing is mentioned with regard to confounding sources of complexity reduction, though.
quasiscaling regions were observed for state 2 (D_{2}^{eff} = 3.7) and the seizure state (D_{2}^{eff} = 5.3), but not the control state. in other words, complexity apparently increased, counter to theory, and there's no base level to compare it with.
plots don't really show a clear power law either that would justify the reliability of their estimates. nor was any statistical hypothesis test performed using surrogate data.
 a more credible one?
Martinerie et al (1998)^{7} attempted to predict seizures by looking at phase transitions between two subsections in reconstructed phase space, also built from intracranial EEG series. it is claimed that for most cases (17 out of 19) it is possible to foretell seizure onset 2 to 6 seconds in advance, and that all of them follow a similar trajectory in phase space.
the alternative hypothesis is actually explored with surrogate data, and authors made sure to smooth noise out previous to the embedding. the number of dimensions (16) may be too unwieldy, though.
• hypothesis testing
the maximal Lyapunov coefficient and the correlation dimension come handy in detecting nonlinear dynamics, however they cannot establish such a fact by themselves. it is necessary to observe a clear scaling region.
a stricter analysis would test the hypothesis that observed data are due to a linear process. this implies that certain parameter's conditional distribution (given H_{0}) has to be empirically estimated, so that it can be contrasted with the real data.
• surrogate data
surrogate data is a akin to resampling methods (permutation, Monte Carlo, bootstrapping) for time series. surrogate data share many similarities with true data for some respects, while consistent with H_{0} for others. when testing for nonlinearity, Fourier phases are generated randomly while the power spectrum is retained, since the imaginary part is where nonlinearity could emerge from.
the general procedure for generating surrogate data is as follows:

decompose the signal in amplitudes and phases using the (fast, discrete) Fourier transform:
$$F\left[{s}_{0}\left(t\right)\right]\phantom{\rule{0.278em}{0ex}}\to \phantom{\rule{0.278em}{0ex}}\{A\left(f\right),\phantom{\rule{0.278em}{0ex}}\phi \left(f\right)\}$$

substitute the alleged nonlinear properties with random numbers drawn from the uniform distribution ranging from 0 to 2π radians (trying to make ${\zeta}_{Nk}={\zeta}_{k}$ , so that the imaginary part being odd will result in a substitute series belonging to ℝ. see the #Fourier transform properties section):
$$\{A\left(f\right),\phantom{\rule{0.278em}{0ex}}\zeta \left(f\right)\}$$

wrap the surrogate series back with the inverse Fourier transform:
$${s}_{1}\left(t\right)\phantom{\rule{0.278em}{0ex}}\leftarrow \phantom{\rule{0.278em}{0ex}}{F}^{1}[\{A\left(f\right),\phantom{\rule{0.278em}{0ex}}\zeta \left(f\right)\}]$$

test the series out (dimension estimation, Lyapunov coefficient, autocorrelation, nonlinear prediction error, etc.).

rinse and repeat till the parameter's distribution given H_{0} is dense enough to yield good typeI error estimates.
• statistical significance
as in standard hypothesis testing, for a significance level of α (type I error: probability that H_{0} is erroneously rejected) we would like at most α% of the area under the surrogate data curve to overlap the original measurements, or be equal or more extreme than our point measurement. such a rankbased test is a nonparametric way of deriving a pvalue.
for instance, in order to reject H_{0} with 95% of success, it would be enough to perform $\frac{1}{.05}1=19$ tests (for a oneside test. $\frac{2}{.05}1=39$ for the twotail case).
• as a determinism detector
does passing the test also mean that the system is deterministic (since we tested for nonlinear determinism)?
partially yes, with some probability. recall that all statistical inference is based on limited evidence and inductive reasoning:
accept H_{0}  reject H_{0}  

H_{0} is true  1α (confidence)  α (type I error) 
H_{0} is false  β (type II error)  1β (power) 
more caveats:
 we have assumed that the available power spectrum captures at least one whole period in the time domain (i.e. no systemic sampling bias due to longterm process). otherwise, random events in a nonstationary process could be misinterpreted as deviations from H_{0}.
 the process may not follow the conjectured probability distribution (this problem is mitigated by nonparametric tests).
 multiple comparisons problem: performing many independent tests overblows type I error. for those cases, significance levels should be made more stringent according to some correction method (Bonferroni e.g.)
• as a phase transition detector
the same nonlinearity tests have been used to distinguish states of a system.
compute the same statistic for different time segments and compare with one another. the reasoning follows what we did in the #stationarity test.

complexity doesn't imply advanced math, for it can emerge from very simple descriptions (see rule 110 for instance). that said, this post relies on calculus (including some differential equations), inferential statistics (also the fundamentals of information theory), bits of graph theory and linear algebra; most (or all) of which should be familiar to STEM undergraduates. topics like Fourier analysis and chaos theory are introduced from there. ↩

not to be confused with "complicated" or "difficult". ↩

we could have arrived at the same harmonic oscillator equation using the mechanics of a pendulum, or an LC circuit. See system equivalence. ↩

$\forall \overrightarrow{z}\in \u2102\phantom{\rule{0.278em}{0ex}}(\left\overrightarrow{z}\right=\sqrt{\overrightarrow{z}\xb7\overline{\overrightarrow{z}}})$ , because $\sqrt{\overrightarrow{z}\xb7\overline{\overrightarrow{z}}}=\sqrt{{z}_{1}^{2}+i{z}_{1}{z}_{2}i{z}_{1}{z}_{2}+{z}_{2}^{2}}$ ↩

Holger Kantz, Thomas Schreiber (2003). Nonlinear Time Series Analysis. Cambridge University Press. ↩

Klaus Lehnertz, Christian E. Elger. Can Epileptic Seizures be Predicted? Evidence from Nonlinear Time Series Analysis of Brain Electrical Activity (1998). Phys. Rev. Lett. 80, 5019. https://doi.org/10.1103/PhysRevLett.80.5019 ↩

Martinerie, J., Adam, C., Le Van Quyen, M., Baulac, M., Clemenceau, S., Renault, B., & Varela, F. J. (1998). Epileptic seizures can be anticipated by nonlinear analysis. Nature medicine, 4(10), 1173. https://doi.org/10.1038/2667 ↩