Signal and Image Processing

2D images and formation

from matplotlib import pyplot as p
import numpy.random as r
I = r.rand(256,256)*0.5;
p.imshow(I ,cmap='gray', vmin=0.0,vmax=1.0);
p.show();

+

• rand(256,256) 生成一个 256x256 的随机矩阵, 值在 0-1 之间
• cmap = 'gray' 表示使用灰度图显示
• vmin 和 vmax 分别表示最小值和最大值
• matplotlib.pyplot.imshow() 函数用于显示图像, 一般可以自动数值normalize到 0-1 之间
• vmin 和 vmax 可以指定normalize的范围

Dimensionality of an Image

8 bit image: 256 gray levels

• e.g. 256x256 = 65,536 pixels
• 256 values ^ 65,536 pixels = 256^65,536

Image: 2D sampling of signal

• Signal: function depending on some variable with physical meaning.
• Image: sampling of that function.
  • 2 variables: xy coordinates
  • 3 variables: xy + time (video)
  • ‘Brightness’ is the value of the function for visible light
• Can be other physical values too: temperature, pressure, depth …
  • 信号不仅限于亮度，还可以表示其他物理量。例如，红外图像可以表示温度，海洋探测图像可以表示深度，气象图像可以表示压力等。

Sampling

Sampling in 1D

• Sampling in 1D takes a function and returns a vector whose elements are values of that function at the sample points.

等间隔采样

Sampling in 2D

• Sampling in 2D takes a function and returns a matrix.

Quantization

• Digital values – integer values
  • Quantization is lossy!! After quantization, the original signal cannot be reconstructed anymore
  • In contrast to sampling, as a sampled but not quantized signal can be reconstructed.
• Simple quantization uses equally spaced levels with k intervals 量化是一种将连续信号转换为离散值的过程。在简单量化中，整个信号的范围会被均匀地划分为 k 个区间（或称为级别）。这些区间之间的间隔是相同的，因此称为“等间隔”。每个信号的值都会被映射到其对应的区间，最终得到的是有限的离散值，而不是连续的信号。

Quantization Effects – Radiometric Resolution

Spatial Linear Filters

Four views of filtering

• Image filters in spatial domain
• Image filters in the frequency domain
• Image filters in the temporal domain
• Image pyramids

Image filters in spatial domain

$$h[m,n]=\sum _{k,l}f[k,l]\cdot I[m+k,n+l]$$

• Filter is a mathematical operation of a grid of numbers
• Smoothing, sharpening, measuring texture
  • 平滑、锐化、测量纹理

Box filter

$$\frac{1}{9}\left[\begin{array}{ccc}1& 1& 1\\ 1& 1& 1\\ 1& 1& 1\end{array}\right]$$

系数$\frac{1}{9}$ 用于保证整体亮度不发生变化

• Usages
  • Enhance images
    • Denoise, resize, increase contrast, etc.
  • Extract information from images
    • Texture, edges, distinctive points, etc.
  • Detect patterns
    • Template matching

Sobel filter

• 用于边缘检测

• Check horizontal edge 横边

$$\left[\begin{array}{ccc}1& 2& 1\\ 0& 0& 0\\ -1& -2& -1\end{array}\right]$$

• Check Vertical edge 竖边

$$\left[\begin{array}{ccc}1& 0& -1\\ 2& 0& -2\\ 1& 0& -1\end{array}\right]$$

相当于计算两侧的差值，差值越大，当前位置是边缘

Correlation and Convolution

Correlation

$$h[m,n]=\sum _{k,l}f[k,l]\cdot I[m+k,n+l]$$

h=scipy.signal.correlated2d(f,I)

Convolution

$$h[m,n]=\sum _{k,l}f[k,l]\cdot I[m-k,n-l]$$

h=scipy.signal.convolve2d(f,I)

• Convolution is the same as correlation with a 180° rotated filter kernel.
• Correlation and convolution are identical when the filter kernel is symmetric.

卷积与相关的关系：

• 卷积可以看作是相关的一种变体，唯一的区别是卷积中的滤波器会先进行180度的旋转。
• 当滤波器是对称的时，卷积和相关是相同的。这是因为旋转180度不会改变滤波器的形状。

Properties of linear filters

Linearity

$$imfilter(I,f_1+f_2)=imfilter(I,f_1)+imfilter(I,f_2)$$

线性滤波器的线性性意味着，对于输入图像 I 以及两个滤波器 f_1 和 f_2 ，对图像应用滤波器的结果等于先将滤波器相加，再应用一次滤波，或者分别应用每个滤波器，再将结果相加。 换句话说，滤波操作对滤波器的加法是可交换的，可以直接作用于滤波器的线性组合。

Shift invariance

$$imfilter(I,shift(f))=shift(imfilter(I,f))$$

平移不变性指的是滤波器的输出与像素位置无关，意思是无论图像中的像素在什么位置，滤波器对强度的操作结果都是相同的。也就是说，滤波器的应用不会因为图像的某个区域平移而改变输出。 在公式中，如果我们先将滤波器 f 平移，再对图像 I 进行滤波，这与先对图像 I 进行滤波，然后再平移滤波结果是等价的。

Any linear, shift-invariant operator can be represented as a convolution.

Convolution properties

1. Commutative

$$a\times b=b\times a$$

• Conceptually no difference between filter and signal
• But particular filtering implementations might break this equality

2. Associative

$$a\times (b\times c)=(a\times b)\times c$$

• Often apply several filters one after another: $(((a\times b1)\times b2)\times b3)$
• This is equivalent to applying one filter: $a\times (b1\times b2\times b3)$

3. Distributes over addition

$$a\times (b+c)=(a\times b)+(a\times c)$$

4. Scalars factor out

$$ka\times b=a\times kb=k(a\times b)$$

5. .Identity unit impulse

$$e=[0,0,1,0,0],a\times e=a$$

Gaussian filter

$$G_{\sigma }=\frac{1}{2\pi {\sigma }^2}{e}^{-\frac{x^2+y^2}{2{\sigma }^2}}$$

Property

• Remove high-frequency components from the image (a low-pass filter)
  • Images become more smooth
• Gaussian convolved with Gaussian is another Gaussian
• Separable kernel
  • Factors into product of two 1D Gaussians

Match Object

Subtract the median

f = D[ 57:117, 107:167 ]
f2 = f – np.mean(f)
D2 = D – np.mean(D)
I2 = correlate2d( D2, f2, 'same' )

Now zero centered.
Score is higher only when dark parts
match and when light parts match.

通过将图像和子区域的均值归零，相关得分在模板的暗区和亮区与图像相应区域匹配时才会较高。这避免了只匹配亮度（而不考虑结构）的高得分情况。

Non-linear filters

Median filter & Mean filter

• Operates over a window by selecting the median intensity in the window.
• Rank filter as based on ordering of gray levels
  • E.G., min, max, range filters

选择窗口内的中位数/平均数

中值滤波器：

• 能够保留图像的边缘和细节。
• 有效去除盐噪声和胡椒噪声。
• 对异常值有很好的鲁棒性。

均值滤波器：

• 会模糊图像边缘，平滑噪声和细节。
• 对极端噪声和异常值处理效果较差。