圖像拼接技術(shù)就是針對(duì)同一場(chǎng)景的一系列圖片，根據(jù)圖片的特征，比如位置，重疊部分等，拼接成一張大幅的寬視角的圖像。

圖像拼接要求拼接后圖像最大程度的與原圖一致，失真盡可能的小，并且要盡量做到天衣無(wú)縫即沒(méi)有明顯的拼接線或其他拼接痕跡。

圖像拼接不能損失原始圖像信息

為達(dá)到以上目標(biāo)，圖像拼接要求具備以下條件：

1：圖像應(yīng)具有一定的特征性能，拼接正是通過(guò)這些特征來(lái)進(jìn)行的。

2：圖像需要具有重疊部分，一般情況下，這些重疊部分點(diǎn)圖像的1/4以上較為合理。

3、圖像的背景亮度差異不能太大，應(yīng)該低于10個(gè)灰度值，否則難以拼接成功。

4、圖像的方位差異不能太大，圖像應(yīng)該來(lái)源同一方位。

5、拼合邊界過(guò)渡應(yīng)平滑，以消除接拼痕跡

圖像拼接前，根據(jù)圖像情況，可以進(jìn)行圖像預(yù)處理，主要是對(duì)圖像進(jìn)行校正和噪聲濾波

1、校正

根據(jù)圖像失真原因，建立相應(yīng)的校正模型，從失真的圖像中提取所需要的信息。從圖像失真的逆過(guò)程來(lái)恢復(fù)圖像。這個(gè)過(guò)程也可以理解為設(shè)計(jì)一個(gè)濾波器，使用其能從失真圖像中計(jì)算得到真實(shí)圖像的估值，從而最大程度的恢復(fù)真實(shí)圖像。

2、噪聲濾波

噪聲在圖像上分布主要有兩種型式：

1、位置隨機(jī)，幅值基本相同，一般稱之為 椒鹽噪聲；

2、幅值隨機(jī)，但基本上每個(gè)點(diǎn)都存在，從幅值的分布統(tǒng)計(jì)來(lái)看，主要有高斯型，瑞利型，又有如頻譜均勻的噪聲。

對(duì)于這些噪聲，處理方法有如下幾種：

1、均值濾波

就是用均值替代原圖像中的各個(gè)像素。具體方法是：對(duì)將處理的像素，選擇一個(gè)模板，此模板為其鄰近的若干像素組成，用模板中的像素的均值去替代原來(lái)的像素值 。

2、中值濾波

中值濾波是一種非線性平滑技術(shù)，它將每一像素點(diǎn)的灰度值設(shè)置為該點(diǎn)某鄰域窗口內(nèi)的所有像素點(diǎn)灰度值的中值. 中值濾波是基于排序統(tǒng)計(jì)理論的一種能有效抑制噪聲的非線性信號(hào)處理技術(shù)，中值濾波的基本原理是把數(shù)字圖像或數(shù)字序列中一點(diǎn)的值用該點(diǎn)的一個(gè)鄰域中各點(diǎn)值的中值代替，讓周圍的像素值接近的真實(shí)值，從而消除孤立的噪聲點(diǎn)。方法是用某種結(jié)構(gòu)的二維滑動(dòng)模板，將板內(nèi)像素按照像素值的大小進(jìn)行排序，生成單調(diào)上升（或下降）的為二維數(shù)據(jù)序列。二維中值濾波輸出為g（x,y）=med{f(x-k,y-l),(k,l∈W)} ，其中，f(x,y)，g(x,y)分別為原始圖像和處理后圖像。W為二維模板，通常為3*3，5*5區(qū)域，也可以是不同的的形狀，如線狀，圓形，十字形，圓環(huán)形等。

圖像配比：

包含以下幾個(gè)要素

1、選定特征空間

特征空間是由不參與匹配的圖像特征構(gòu)成。特征可以為圖像的灰度特征，也可以是邊界，輪廓，顯著特征（如角點(diǎn)，線交叉點(diǎn)，高曲率點(diǎn)），統(tǒng)計(jì)特征（如矩不變量，中心），高層結(jié)構(gòu)描述與句法描述等；這里其實(shí)是定義了配準(zhǔn)的空間范圍。

2、相似度

評(píng)估待匹配特征之間的相似性，它通常定義為某種代價(jià)函數(shù)或者是距離函數(shù)。這里定義為需要選定的某種算法。

3、搜索空間

待估計(jì)參數(shù)組成的空間就稱為搜索空間，也就是說(shuō)，搜索空間是指所有可能的變換組成的空間。這其實(shí)就定義了搜索算法的空間復(fù)雜度

4、搜索策略

用合適的方法在搜索空間中找出平移，旋轉(zhuǎn)等變換參數(shù)的最佳估計(jì)，使得相似度達(dá)到最大值，這其實(shí)也就是定義了搜索算法的時(shí)間復(fù)雜度。

拼接方法：

1、基于區(qū)域的配準(zhǔn)方法

采用拼接圖像的灰度值檢測(cè)，對(duì)待配準(zhǔn)圖你中一塊區(qū)域與參考圖像中的相機(jī)尺寸的區(qū)域使用最小二乘法或者其它數(shù)學(xué)方法計(jì)算其灰度值的差異，對(duì)此差異比較后來(lái)判斷待拼接圖像重疊區(qū)域的相似度，由此得到待拼接圖像重疊區(qū)域的范圍和位置，從而實(shí)現(xiàn)圖像拼接。也可以通過(guò)FFT變換將圖像由時(shí)域變換到頻域，然后再進(jìn)行配準(zhǔn)。對(duì)位移量比較大的圖像，可以先校正圖像的旋轉(zhuǎn)，然后建立兩幅圖像之間的映射關(guān)系，總而言之，這種方法有很多不足，已經(jīng)不是主流了。

2、基于特征的配準(zhǔn)方法

基于特征的圖像配準(zhǔn)方法有很多形式及其改進(jìn)方式，其總體特點(diǎn)是：不直接利用圖像的像素，而是通過(guò)像素值導(dǎo)出圖像內(nèi)容最抽像的描述和符號(hào)特征，并用此特征為匹配模板，通過(guò)二維高斯模糊過(guò)濾，把幾幅待配準(zhǔn)圖像的灰度局總最大值，邊界邊緣輪廓，邊緣點(diǎn)，邊緣線段，組織（紋理）結(jié)構(gòu)，角、項(xiàng)點(diǎn)，拐點(diǎn)，交叉點(diǎn)，封閉曲線低級(jí)對(duì)應(yīng)特征點(diǎn)及利用特征圖像關(guān)系圖等高級(jí)特征，構(gòu)造方程組，通過(guò)數(shù)值計(jì)算得到變換數(shù)來(lái)進(jìn)行圖像對(duì)齊，進(jìn)而確定兩者的匹配位置，實(shí)現(xiàn)特征點(diǎn)，特征線等拼接，并且可以提高運(yùn)算速度。

基于特征的接拼方法，分為四個(gè)步驟

1、特征檢測(cè)：從圖像中檢測(cè)出顯著且獨(dú)特的圖像特征，諸如：閉合區(qū)域，直線段，邊緣，輪廓，點(diǎn)等。

2、特征匹配：從相似度確定圖像之間特征的對(duì)應(yīng)關(guān)系，又分為如下幾類：

2.1：使用空域關(guān)系的方法

2.2：使用不變描述符的方法

2.3:松弛方法

2.4:金字塔和小波方法

3、變換模型的估計(jì)：變換函數(shù)選擇和函數(shù)參數(shù)估計(jì)

4、圖像變換和重采樣：可以通過(guò)前向或后向的方式來(lái)實(shí)現(xiàn)，插值的方法有最近鄰插值、雙線性插值、雙三次函數(shù)插值、二次樣條插值、三次B樣條插值、高階B樣條插值。

基于特征的方法普遍適用于局部結(jié)構(gòu)信息更顯著

的情況，能夠處理圖像之間復(fù)雜變形的情況，不足之處是特征檢測(cè)困難且不穩(wěn)定，最關(guān)鍵的一點(diǎn)是需要有一種判斷力很強(qiáng)的、魯棒性能好的且對(duì)圖像之間變化保持不變的特征匹配算法。

下面是Halcon自帶例程，如何拼接圖像

**此例程講解了如何將幾張局部的PCB圖像拼接居一張大的馬賽克PCB圖像。
**此例程使用算子proj_match_points_ransac和算子 gen_projective_masaic完成上述工作。
**請(qǐng)注意：這個(gè)PCB圖像有一幾處看起來(lái)像拼接逢合線的破損點(diǎn)，為了更好的區(qū)分真正的縫合線，例程呈現(xiàn)逢合線。
dev_update_off ()
dev_close_window ()
dev_open_window (0, 0, 640, 480, 'white', WindowHandle)
dev_set_color ('green')
set_display_font (WindowHandle, 14, 'mono', 'true', 'false')
**一張一張的讀取圖像。
gen_empty_obj (Images)
for J := 1 to 6 by 1
    read_image (Image, 'mosaic/pcb_' + J$'02')
    concat_obj (Images, Image, Images)
    dev_display (Image)
    disp_message (WindowHandle, 'Image ' + J$'d', 'image', -1, -1, 'black', 'true')
    wait_seconds (1)
endfor
disp_continue_message (WindowHandle, 'black', 'true')
stop ()
* To show the point matches that are used to compute the projective
* transformation between the images, we will show all images in a large
* tiled image with some space between the images so that the extents
* of the images are easily visible.
dev_set_window_extents (-1, -1, 640 / 4, 2980 / 4)
tile_images_offset (Images, TiledImage, [0,500,1000,1500,2000,2500], [0,0,0,0,0,0], [-1,-1,-1,-1,-1,-1], [-1,-1,-1,-1,-1,-1], [-1,-1,-1,-1,-1,-1], [-1,-1,-1,-1,-1,-1], 640, 2980)
dev_clear_window ()
dev_display (TiledImage)
disp_message (WindowHandle, 'All 6 images', 'window', 12, 12, 'black', 'true')
disp_message (WindowHandle, 'Click 'Run'
to continue', 'window', 2980 / 4 - 50, 12, 'black', 'true')
stop ()
* Now we compute point matches between the five pairs of images and with this
* the projective transformation between the image pairs.  Note that the code
* below calls the point operator for each image pair.  Since the images form
* a strip, with a little book keeping we could make the process a little more
* efficient by saving the points from the last iteration (ImageT in pair J will
* be identical to ImageF in pair J+1).  This is not done here because such an
* optimization would be quite cumbersome in the general case where the images
* can lie in a general configuration that cannot be represented by a strip.
dev_clear_window ()
dev_display (TiledImage)
disp_message (WindowHandle, 'Point matches', 'window', 12, 3, 'black', 'true')
* We define the image pairs, i.e., which image should be mapped to which image.
From := [1,2,3,4,5]
To := [2,3,4,5,6]
Num := |From|
* We need a variable to accumulate the projective transformation matrices.
ProjMatrices := []
* Furthermore, since we want to create a rigid mosaic below we need to
* accumulate all the point correspondences and the number of matches per
* image pair.
Rows1 := []
Cols1 := []
Rows2 := []
Cols2 := []
NumMatches := []
* Now we can determine the transformations between the five image pairs.
for J := 0 to Num - 1 by 1
    F := From[J]
    T := To[J]
    select_obj (Images, ImageF, F)
    select_obj (Images, ImageT, T)
    * Extract the points in both images.
    points_foerstner (ImageF, 1, 2, 3, 200, 0.3, 'gauss', 'false', RowJunctionsF, ColJunctionsF, CoRRJunctionsF, CoRCJunctionsF, CoCCJunctionsF, RowAreaF, ColAreaF, CoRRAreaF, CoRCAreaF, CoCCAreaF)
    points_foerstner (ImageT, 1, 2, 3, 200, 0.3, 'gauss', 'false', RowJunctionsT, ColJunctionsT, CoRRJunctionsT, CoRCJunctionsT, CoCCJunctionsT, RowAreaT, ColAreaT, CoRRAreaT, CoRCAreaT, CoCCAreaT)
    * Determine the point matches and the transformation for the current
    * image pair.
    proj_match_points_ransac (ImageF, ImageT, RowJunctionsF, ColJunctionsF, RowJunctionsT, ColJunctionsT, 'ncc', 21, 0, 0, 480, 640, 0, 0.5, 'gold_standard', 1, 4364537, ProjMatrix, Points1, Points2)
    * Accumulate the transformation matrix.
    ProjMatrices := [ProjMatrices,ProjMatrix]
    * Accumulate the point matches and number of point matches.
    Rows1 := [Rows1,subset(RowJunctionsF,Points1)]
    Cols1 := [Cols1,subset(ColJunctionsF,Points1)]
    Rows2 := [Rows2,subset(RowJunctionsT,Points2)]
    Cols2 := [Cols2,subset(ColJunctionsT,Points2)]
    NumMatches := [NumMatches,|Points1|]
    * Generate crosses that represent the extracted points in the tiled image.
    * Note that we have to take the row offsets of the images in the tiled image
    * into account.
    gen_cross_contour_xld (PointsF, RowJunctionsF + (F - 1) * 500, ColJunctionsF, 6, rad(45))
    gen_cross_contour_xld (PointsT, RowJunctionsT + (T - 1) * 500, ColJunctionsT, 6, rad(45))
    * Generate a representation of the matched point pairs as lines.  We create
    * XLD contours from the lines so that we can zoom into the graphics window
    * to take a closer look at the matches.
    RowF := subset(RowJunctionsF,Points1) + (F - 1) * 500
    ColF := subset(ColJunctionsF,Points1)
    RowT := subset(RowJunctionsT,Points2) + (T - 1) * 500
    ColT := subset(ColJunctionsT,Points2)
    gen_empty_obj (Matches)
    for K := 0 to |RowF| - 1 by 1
        gen_contour_polygon_xld (Match, [RowF[K],RowT[K]], [ColF[K],ColT[K]])
        concat_obj (Matches, Match, Matches)
    endfor
    * Now display the extracted data.
    dev_set_color ('blue')
    dev_display (Matches)
    dev_set_color ('green')
    dev_display (PointsF)
    dev_display (PointsT)
endfor
disp_message (WindowHandle, 'Click 'Run'
to continue', 'window', 2980 / 4 - 50, 12, 'black', 'true')
stop ()
* Finally, we can generate the mosaic image from the projective transformations.
gen_projective_mosaic (Images, MosaicImage, 2, From, To, ProjMatrices, 'default', 'false', MosaicMatrices2D)
get_image_size (MosaicImage, Width, Height)
dev_set_window_extents (-1, -1, Width / 3, Height / 3)
dev_clear_window ()
dev_display (MosaicImage)
disp_message (WindowHandle, 'Projective mosaic', 'window', 12, 12, 'black', 'true')
disp_message (WindowHandle, 'Click 'Run'
to continue', 'window', Height / 3 - 50, 12, 'black', 'true')
stop ()
* To show more clearly that the folds visible in the image do not result from the
* mosaicking, we display the seams between the images in the mosaic image.
* This can be done most easily by creating an image that contains the border
* of the images, generating a mosaic from it, and segmenting the resulting
* mosaic image.
get_image_size (Image, Width, Height)
gen_image_const (ImageBlank, 'byte', Width, Height)
gen_rectangle1 (Rectangle, 0, 0, Height - 1, Width - 1)
paint_region (Rectangle, ImageBlank, ImageBorder, 255, 'margin')
gen_empty_obj (ImagesBorder)
for J := 1 to 6 by 1
    concat_obj (ImagesBorder, ImageBorder, ImagesBorder)
endfor
gen_projective_mosaic (ImagesBorder, MosaicImageBorder, 2, From, To, ProjMatrices, 'default', 'false', MosaicMatrices2D)
threshold (MosaicImageBorder, Seams, 128, 255)
dev_clear_window ()
dev_display (MosaicImage)
disp_message (WindowHandle, 'Seams between the
images', 'window', 12, 12, 'black', 'true')
dev_set_color ('yellow')
dev_display (Seams)
disp_message (WindowHandle, 'Click 'Run'
to continue', 'window', 550, 12, 'black', 'true')
stop ()
* If you look very closely at the projective mosaic above, you may note that
* there is a very slight projective distortion in the mosaic.  This happens
* because the transformations cannot be determined with perfect accuracy
* because of very small errors in the point coordinates due to noise.  Because
* of the strip configuration, essentially the overlapping area between the image
* pairs can act like a hinge around which the images may rotate out of the image
* plane.  In this example, we know that the mapping between the images must
* be a rigid transformation.  If we want to force the transformation to be rigid
* we can simply use bundle_adjust_mosaic.
bundle_adjust_mosaic (6, 1, From, To, ProjMatrices, Rows1, Cols1, Rows2, Cols2, NumMatches, 'rigid', MosaicMatrices2D, Rows, Cols, Error)
* Now, we can generate the mosaic image from the rigid transformations.
gen_bundle_adjusted_mosaic (Images, MosaicImageRigid, MosaicMatrices2D, 'default', 'false', TransMatrix2D)
get_image_size (MosaicImageRigid, Width, Height)
dev_set_window_extents (-1, -1, Width / 3, Height / 3)
dev_clear_window ()
dev_display (MosaicImageRigid)
disp_message (WindowHandle, 'Rigid mosaic', 'window', 12, 12, 'black', 'true')

下面我們看一下另一個(gè)例程：

這個(gè)例程使用proj_match_points_ransac_guided 和gen_projective_mosaic

主要介紹如何使用金字塔算法快速獲取兩個(gè)圖像的特征點(diǎn)進(jìn)行拼接。

* This example program shows how images can be combined
* into a mosaic image using proj_match_points_ransac_guided
* and gen_projective_mosaic.
* It is shown how the calculation of the projection between two
* images can be accelerated using an image pyramid.
* 
* Initializations
ImgPath := '3d_machine_vision/mosaic/'
ImgName := 'bga_r_'
Times := []
Colors := ['red','coral','yellow','lime green']
read_image (Images, ImgPath + ImgName + ['01','06'])
dev_update_off ()
dev_close_window ()
dev_open_window_fit_size (0, 0, 640, 980, 320, 490, WindowHandle)
dev_open_window_fit_size (0, 330, 490, 490, 1000, 490, WindowHandle1)
set_display_font (WindowHandle, 14, 'mono', 'true', 'false')
set_display_font (WindowHandle1, 14, 'mono', 'true', 'false')
* The internal camera parameters of the used camera
* (necessary to eliminate radial distortions)
CamParam := [0.0121693,-2675.63,7.40046e-006,7.4e-006,290.491,258.887,640,480]
change_radial_distortion_cam_par ('adaptive', CamParam, 0, CamParOut)
change_radial_distortion_image (Images, Images, Images, CamParam, CamParOut)
* To show the point matches that are used to compute the
* transformation between the images, we will show both images in a
* tiled image with some space between the images so that the extents
* of the images are easily visible.
tile_images_offset (Images, TiledImage, [0,500], [0,0], [-1,-1], [-1,-1], [-1,-1], [-1,-1], 640, 980)
* 
* Now we can determine the transformations between the image pairs.
From := 1
To := 2
select_obj (Images, ImageF, From)
select_obj (Images, ImageT, To)
* 
* Repeat the calculation 4 times with a different number of pyramid levels
for NumLevels := 1 to 4 by 1
    * 
    dev_clear_window ()
    dev_set_window (WindowHandle)
    dev_clear_window ()
    dev_display (TiledImage)
    disp_message (WindowHandle, ['Calculate point matches','with ' + NumLevels + ' pyramid levels','Please wait ...'], 'window', 20, 10, 'black', 'true')
    * 
    * Calculate the projection between the two images
    * Check the procedure's comments for details
    count_seconds (S1)
    proj_match_points_ransac_pyramid (ImageF, ImageT, NumLevels, RowFAll, ColFAll, RowTAll, ColTAll, ProjMatrix, Points1, Points2)
    count_seconds (S2)
    Times := [Times,S2 - S1]
    * 
    * Display point correspondences
    gen_cross_contour_xld (PointsF, RowFAll, ColFAll, 6, rad(45))
    gen_cross_contour_xld (PointsT, RowTAll + 500, ColTAll, 6, rad(45))
    RowF := subset(RowFAll,Points1)
    ColF := subset(ColFAll,Points1)
    RowT := subset(RowTAll,Points2) + 500
    ColT := subset(ColTAll,Points2)
    gen_empty_obj (Matches)
    for K := 0 to |RowF| - 1 by 1
        gen_contour_polygon_xld (Match, [RowF[K],RowT[K]], [ColF[K],ColT[K]])
        concat_obj (Matches, Match, Matches)
    endfor
    dev_display (TiledImage)
    dev_set_color ('blue')
    dev_display (Matches)
    dev_set_color ('green')
    dev_display (PointsF)
    dev_display (PointsT)
    disp_message (WindowHandle, [|RowF| + ' point matches','Time used: ' + (S2 - S1)$'.3' + ' s'], 'window', 20, 10, 'black', 'true')
    * 
    * Generate the mosaic image
    gen_projective_mosaic (Images, MosaicImage, 1, From, To, ProjMatrix, [2,1], 'false', MosaicMatrices2D)
    * 
    * Display mosaic image
    get_image_size (MosaicImage, Width, Height)
    dev_set_window (WindowHandle1)
    dev_resize_window_fit_image (MosaicImage, 0, 330, [400,700], 700)
    dev_clear_window ()
    dev_display (MosaicImage)
    disp_message (WindowHandle1, 'Projective mosaic (used ' + NumLevels + ' pyramid levels)', 'window', 20, 10, 'black', 'true')
    disp_continue_message (WindowHandle1, 'black', 'true')
    stop ()
endfor
* 
* Display execution times
dev_set_window (WindowHandle)
dev_close_window ()
MaxTime := max(Times)
BaseRow := 380
RectHeight := 300
disp_message (WindowHandle1, ['Time in s:','(#levels used)'], 'image', BaseRow + 20, 10, 'black', 'true')
for Index := 0 to |Times| - 1 by 1
    gen_rectangle1 (Rectangle, BaseRow - RectHeight * Times[Index] / MaxTime, 200 + Index * 100, BaseRow, 280 + Index * 100)
    disp_message (WindowHandle1, [Times[Index]$'.3','(' + (Index + 1) + ')'], 'image', BaseRow + 20, 200 + 100 * Index, 'black', 'true')
    dev_set_color (Colors[Index])
    dev_set_draw ('fill')
    dev_display (Rectangle)
endfor
disp_finished_message (WindowHandle1, 'black', 'true')

審核編輯：湯梓紅