include(../../../m4-dip-html-macros.m4)

LESSON(Convolution Principles)


SUBLESSON(Concepts of Linear Systems and Convolution)

Discrete Convolution is a tool to build any linear and shift invariant
filter. The equation for the convolution, {\bf g(x)}, of the sequence
{\bf f(x)} with the convolution kernel {\bf h(x)} is

\[
g(x) = f(x) \ast h(x) = \sum_{all \; k} h(x-k) f(k)
\]

{\bf h(x)} is also called the impulse response, Point Spread Function
{\bf PSF}, window kernel, window mask, window, filter, or template.
This formula can be easily extended to the 2D case.

SUBLESSON(Region of Support)

We normally show the region of support of the convolution kernel. That is,
the region where the kernel values are valid.  The kernel values
are zero outside the region of support.

There are two main types of convolution, depending on how we define
the data: {\bf Linear Convolution} and {\bf Circular Convolution}.

BEGINLIST
ITEM For the 1D case, let f(x) and h(x) be infinite sequences, with region
of support of length N for f(x) and M for h(x), meaning that outside the
region of support all the data pointns are zero. Then the convolution
g(x) is said to {\bf Linear Convolution}. The resultant sequence g(x)
will also be infinite with region of support of length M+N-1.

ITEM Again for the 1D case, if we define f(x) and h(x) as periodic
sequences, with the same period N, then the convolution is said to be
{\bf Circular Convolution}.  The resultant sequence g(x) will also be
periodic with the same period N. In this case, outside the display of
f(x), the data is periodically repeated, and outside the region of support
of h(x), the data is zero extended up to the completion of one period with
dimension N. Remember that h(x) is also periodic with same period as f(x).
ENDLIST

Below are the illustrations on how the sequences are represented in
both cases:

HTMLTWOFIG(original,avg3x3,
           original image f, kernel h)

HTMLTWOFIG(original-padded,original-periodic,
           zero outside (for linear convolution),
           periodic (for circular convolution))

HTMLTWOFIG(3x3avg-kernel-linear,3x3avg-kernel-periodic,
           3x3 kernel for linear convolution,
           3x3 kernel for circular convolution 
            (periodic with same period as the image))

SUBLESSON(Origin of a Convolution Kernel)

When we define a convolution kernel, we do not always we have the origin of
its coordinates positioned at the top left corner of its region of support.
Normally, there is a shift in the resultant image if the origin of the
convolution kernel is not at the center of its region of support.
A simple name for the origin is "sweet spot".

SUBLESSON(Simple Examples of Convolution)

BEGINITEM
ITEM Shifted kernel: h(2,2)=1

HTMLTWOFIGWIDTHLINK(original,,, impulse-2-2,0.5,,
            Original, kernel)

HTMLTWOFIG(shift2-linear,shift2-periodic,
           `Linear Convolution (Note the output size: 
            2 wider, 2 higher)',
           `Circular Convolution (Note the output with same size)')

ITEM Averaging with 3x3 Kernel

HTMLTWOFIG(original,average3x3, 
           Original,average kernel)

HTMLTWOFIG(avg3x3-linear,avg3x3-periodic,
           `Linear convolution. 
            Note that the top left coordinates should be (-1,-1)
            since the kernel has negative coordinates',
           Circular Convolution)

ITEM Sobel X-direction

HTMLTWOFIG(original,sobel-x, 
           Original, Sobel kernel)

HTMLTWOFIG(sobel-linear,sobel-periodic,
           `Linear convolution. 
            Note that the top left coordinates should be (-1,-1)
            since the kernel has negative coordinates',
           Circular convolution)

ENDITEM

Observe that if there is an isolated point (impulse) in the input image
then the kernel appears reproduced in the filtered or output image.

BUILDINGWORKSPACE


LABPURPOSE(`The purpose of this lab exercise is to experiment with the
concepts of linear and circular con- volution, and to compare the two
methods.')

	At the time this lesson was prepared, both the linear and
circular convolution operators in Khoros had small problems. So to
fully reproduce linear and circular convolution with the Khoros
"Linear" operator, the original images have to be zero extended for
linear convolution or periodically extended for the circular
convolution. Also, in both cases, the correct portion of the image
has to be extracted.

	In this experiment the image IMAGEDIR(draw-small.viff) is
used. It is a 32 x 32 image with floating point pixels with values 0.0
and 1.0.

BEGINLIST %1

ITEM Create a "Zoom Display" procedure.

BEGINLIST %2

ITEM Since the images used in this experiment are small (32x32) and
(64x64), create a procedure to display them zoomed by a factor of 4,
using the operators EXPAND and DISPLAYIMAGE.  Rename this
procedure "Zoom Display". For instruction on how to create a procedure
in cantata, see Lab 2 of the "Getting Started with Khoros" section,
"Visual Programming". Read and display the image
IMAGEDIR(draw-small.viff) using the USERDEFINED operator and this
procedure.

BEGINLIST %3

ITEM USERDEFINEDMENU

ITEM EXPANDMENU

ITEM DISPLAYIMAGEMENU

ITEM Make a connection between the output of the USERDEFINED
operator and the input of the EXPAND operator before collapsing 
the EXPAND and DISPLAYIMAGE operators into a procedure. Currently,
I/O connections are not created for the glyph of a procedure unless
they have already been established prior to creating the procedure (In
other words, you won't get an input arrow on your procedure unless you
do this.).

ENDLIST %3

ENDLIST %2

ITEM Create a zero extended image for linear convolution. Read the 
image IMAGEDIR(drawsmall.viff) and zero extend it to size 64 x 64, so
that the input image appears in the center.  Use the PAD operator.

BEGINLIST %2

ITEM PADMENU

ITEM Select and duplicate the "Zoom Display" procedure, and display the
zero extended image.

ENDLIST %2

ITEM Create a periodic image for circular convolution and display
it. Read the image IMAGEDIR(draw-small.viff) and create a 4 x 4
mosaic using the operator "Inset" twice. First to put two images side
by side and the second to put this two-images top to bottom.

BEGINLIST %2

ITEM INSETMENU (2)

ITEM Select and duplicate the "Zoom Display" procedure, and display the
periodic image.

ENDLIST %2

ITEM Convolution kernels

BEGINLIST %2

ITEM Build a shifted h(2,2) = 1 kernel: This kernel is a 3x3 image,
where all values are zero, except for the position (2,2), which is

BEGINLIST %3

ITEM Use the operator IMPULSE to create the kernel, and verify it
using PRINTDATA and FILEVIEWER.

ITEM IMPULSEMENU

ITEM PRINTDATAMENU

ITEM FILEVIEWERMENU

ENDLIST %3

ITEM Build a 3x3 average kernel using the operator CONSTANT. The sum
of all pixels in the kernel should add up to 1. Use PRINTDATA 
and FILEVIEWER to verify the kernel.

BEGINLIST %3

ITEM CONSTANTMENU

ITEM PRINTDATAMENU

ITEM FILEVIEWERMENU

ENDLIST %3

ITEM Sobel X kernel

BEGINLIST %3

ITEM Read and print the contents of the ASCII file IMAGEDIR(sobel\_x.ascii) 
using the USERDEFINED and FILEVIEWER operators.

ITEM USERDEFINEDMENU

ITEM FILEVIEWERMENU

ENDLIST %3

ENDLIST %2

ITEM For all three kernels, perform linear convolution using the padded
image, and circular convolution using the periodic image. Use LINEAROPERATOR
to perform the convolution.

BEGINLIST %2

ITEM LINEAROPERATORMENU (6)

ENDLIST %2

ITEM Since the input images needed to be extended to perform the
convolutions, you now need to extract the correct portion of the
resultant images to get the "true" convolution results. Use the
operator EXTRACT.

BEGINLIST %2

ITEM EXTRACTMENU (6)

ITEM For linear convolution, you need to extract an image 2 pixels
wider and 2 pixels higher than the original, since the kernel
dimensions are 3x3. (i.e. there will be a 1 pixel border around the
entire image.)

ITEM For periodic convolution, you need to extract the bottom left
quadrant, since the kernel ori- gin is at (0,0)

ITEM Display and compare the results using the "Zoom Display"
Procedure. Also compare them against the original image.

ENDLIST %2

ENDLIST %1

EXERCISES

BEGINLIST %1

ITEM Explain how the position of the extracted image is dependent on the
kernel origin.

ITEM Create an encapsulated workspace operator to fully implement linear
and circular convolution for any kind of kernel.

ENDLIST %1

WORKSPACE(convolution.wk)

ENDLESSON
