計(jì)算機(jī)視覺(jué)ShapeStructureFromMotionI專題知識(shí)專家講座

期末作業(yè)通知作業(yè)形式:項(xiàng)目匯報(bào)(PPT,15分鐘),2-3人一組,自由組隊(duì).作業(yè)內(nèi)容:項(xiàng)目名稱,項(xiàng)目意義,相關(guān)工作,提出算法,試驗(yàn)結(jié)果,分析與結(jié)論.作業(yè)題目:計(jì)算機(jī)視覺(jué)范圍內(nèi)任意課題,提議與個(gè)人導(dǎo)師商議后決定.如有需要,可向任課老師申請(qǐng)題目.關(guān)鍵時(shí)間點(diǎn):

下周四之前,將項(xiàng)目名稱,團(tuán)體組員名單發(fā)至任課老師信箱.第15周課堂時(shí)間答辯.

計(jì)算機(jī)視覺(jué)ShapeStructureFromMotionI專題知識(shí)專家講座第1頁(yè)Shape(Structure)

From

X處理是從2D圖像到2.5D表面形狀(場(chǎng)景深度)問(wèn)題Shapefrommotion-------第七章,第八章Shapefromstereo------第九章Shapefrommonocularcues(shading,vanishingpoint,defocus,texture,….)------第十章計(jì)算機(jī)視覺(jué)ShapeStructureFromMotionI專題知識(shí)專家講座第2頁(yè)第七章基于運(yùn)動(dòng)視覺(jué)場(chǎng)景復(fù)原計(jì)算機(jī)視覺(jué)ShapeStructureFromMotionI專題知識(shí)專家講座第3頁(yè)三維運(yùn)動(dòng)預(yù)計(jì)三維運(yùn)動(dòng)預(yù)計(jì)是指從二維圖象序列來(lái)預(yù)計(jì)物體三維運(yùn)動(dòng)參數(shù)以及三維結(jié)構(gòu)。SFM(ShapeFromMotion)計(jì)算機(jī)視覺(jué)ShapeStructureFromMotionI專題知識(shí)專家講座第4頁(yè)Singularvaluedecomposition(SVD)計(jì)算機(jī)視覺(jué)ShapeStructureFromMotionI專題知識(shí)專家講座第5頁(yè)Everymatrixrepresentsatransformation/faculty/will/svd/index.html計(jì)算機(jī)視覺(jué)ShapeStructureFromMotionI專題知識(shí)專家講座第6頁(yè)Singularvaluedecomposition計(jì)算機(jī)視覺(jué)ShapeStructureFromMotionI專題知識(shí)專家講座第7頁(yè)Singularvaluedecomposition計(jì)算機(jī)視覺(jué)ShapeStructureFromMotionI專題知識(shí)專家講座第8頁(yè)P(yáng)seudo-inverse計(jì)算機(jī)視覺(jué)ShapeStructureFromMotionI專題知識(shí)專家講座第9頁(yè)三維剛體運(yùn)動(dòng)計(jì)算機(jī)視覺(jué)ShapeStructureFromMotionI專題知識(shí)專家講座第10頁(yè)小角度旋轉(zhuǎn)矩陣小角度旋轉(zhuǎn)計(jì)算機(jī)視覺(jué)ShapeStructureFromMotionI專題知識(shí)專家講座第11頁(yè)1.基于正交投影三維運(yùn)動(dòng)預(yù)計(jì)小角度旋轉(zhuǎn)矩陣6個(gè)未知數(shù)，3對(duì)點(diǎn)計(jì)算機(jī)視覺(jué)ShapeStructureFromMotionI專題知識(shí)專家講座第12頁(yè)基于正交投影三維運(yùn)動(dòng)預(yù)計(jì)

Aizawa,1989

1.依據(jù)對(duì)應(yīng)點(diǎn)和深度預(yù)計(jì)值，計(jì)算運(yùn)動(dòng)參數(shù)2.依據(jù)運(yùn)動(dòng)參數(shù)和對(duì)應(yīng)點(diǎn)，重新預(yù)計(jì)深度交替直到穩(wěn)定計(jì)算機(jī)視覺(jué)ShapeStructureFromMotionI專題知識(shí)專家講座第13頁(yè)基于正交投影三維運(yùn)動(dòng)預(yù)計(jì)

Bozdagi,1994

利用深度預(yù)計(jì)值隨機(jī)擾動(dòng)，跳出局部最優(yōu)

1.依據(jù)對(duì)應(yīng)點(diǎn)和深度預(yù)計(jì)值，計(jì)算運(yùn)動(dòng)參數(shù)

2.依據(jù)運(yùn)動(dòng)參數(shù)和深度預(yù)計(jì)值，預(yù)計(jì)對(duì)應(yīng)點(diǎn)坐標(biāo)

3.計(jì)算預(yù)計(jì)誤差計(jì)算機(jī)視覺(jué)ShapeStructureFromMotionI專題知識(shí)專家講座第14頁(yè)基于正交投影三維運(yùn)動(dòng)預(yù)計(jì)4.隨機(jī)擾動(dòng)深度預(yù)計(jì)值5.重復(fù)以上步驟試驗(yàn)證實(shí)，這種改進(jìn)迭代算法在初始深度值有50%誤差情況下，也能很好地收斂到正確運(yùn)動(dòng)參數(shù)值。計(jì)算機(jī)視覺(jué)ShapeStructureFromMotionI專題知識(shí)專家講座第15頁(yè)2基于透視投影模型三維運(yùn)動(dòng)預(yù)計(jì)規(guī)范化焦距F=1,分子分母同除以Zk計(jì)算機(jī)視覺(jué)ShapeStructureFromMotionI專題知識(shí)專家講座第16頁(yè)3基于外極線三維運(yùn)動(dòng)預(yù)計(jì)外極線方程幾何意義計(jì)算機(jī)視覺(jué)ShapeStructureFromMotionI專題知識(shí)專家講座第17頁(yè)基于外極線三維運(yùn)動(dòng)預(yù)計(jì)外極線方程三維剛體運(yùn)動(dòng)引進(jìn)一個(gè)反對(duì)稱矩陣：計(jì)算機(jī)視覺(jué)ShapeStructureFromMotionI專題知識(shí)專家講座第18頁(yè)基于外極線三維運(yùn)動(dòng)預(yù)計(jì)基本矩陣（essentialmatrix）平移矢量乘以不為零系數(shù)，不影響外極線方程成立所恢復(fù)運(yùn)動(dòng)參數(shù)是關(guān)于百分比系數(shù)解計(jì)算機(jī)視覺(jué)ShapeStructureFromMotionI專題知識(shí)專家講座第19頁(yè)本質(zhì)矩陣應(yīng)用可被用于

簡(jiǎn)化匹配問(wèn)題檢測(cè)錯(cuò)誤匹配計(jì)算機(jī)視覺(jué)ShapeStructureFromMotionI專題知識(shí)專家講座第20頁(yè)基于外極線三維運(yùn)動(dòng)預(yù)計(jì)外極線方程計(jì)算機(jī)視覺(jué)ShapeStructureFromMotionI專題知識(shí)專家講座第21頁(yè)基于外極線三維運(yùn)動(dòng)預(yù)計(jì)基本矩陣性質(zhì)

外極線方程待求參數(shù)

5個(gè)未知獨(dú)立參數(shù)，這也和運(yùn)動(dòng)參數(shù)自由度數(shù)量相一致，即三個(gè)旋轉(zhuǎn)自由度，二個(gè)平移自由度（或三個(gè)關(guān)于一個(gè)百分比系數(shù)平移自由度）.計(jì)算機(jī)視覺(jué)ShapeStructureFromMotionI專題知識(shí)專家講座第22頁(yè)(1)依據(jù)基本矩陣預(yù)計(jì)運(yùn)動(dòng)1.計(jì)算基本矩陣8對(duì)以上對(duì)應(yīng)點(diǎn)求穩(wěn)定解(實(shí)際經(jīng)常使用RANSAC算法)計(jì)算機(jī)視覺(jué)ShapeStructureFromMotionI專題知識(shí)專家講座第23頁(yè)(1)依據(jù)基本矩陣預(yù)計(jì)運(yùn)動(dòng)1.計(jì)算基本矩陣Inreality,insteadofsolving

,weseekE

to

minimize

,leasteigenvectorof.計(jì)算機(jī)視覺(jué)ShapeStructureFromMotionI專題知識(shí)專家講座第24頁(yè)8-pointalgorithmToenforcethatFisofrank2,FisreplacedbyF’thatminimizessubjectto.ItisachievedbySVD.Let,where,letthenisthesolution.計(jì)算機(jī)視覺(jué)ShapeStructureFromMotionI專題知識(shí)專家講座第25頁(yè)8-pointalgorithm%BuildtheconstraintmatrixA=[x2(1,:)‘.*x1(1,:)'x2(1,:)'.*x1(2,:)'x2(1,:)'...x2(2,:)'.*x1(1,:)'x2(2,:)'.*x1(2,:)'x2(2,:)'...x1(1,:)'x1(2,:)'ones(npts,1)];

[U,D,V]=svd(A);

%ExtractfundamentalmatrixfromthecolumnofV%correspondingtothesmallestsingularvalue.

E=reshape(V(:,9),3,3)';

%Enforcerank2constraint[U,D,V]=svd(E);

E=U*diag([D(1,1)D(2,2)0])*V';計(jì)算機(jī)視覺(jué)ShapeStructureFromMotionI專題知識(shí)專家講座第26頁(yè)8-pointalgorithmPros:itislinear,easytoimplementandfastCons:susceptibletonoise計(jì)算機(jī)視覺(jué)ShapeStructureFromMotionI專題知識(shí)專家講座第27頁(yè)P(yáng)roblemwith8-pointalgorithm~10000~10000~10000~10000~100~1001~100~100!Ordersofmagnitudedifferencebetweencolumnofdatamatrixleast-squaresyieldspoorresults計(jì)算機(jī)視覺(jué)ShapeStructureFromMotionI專題知識(shí)專家講座第28頁(yè)Normalized8-pointalgorithmTransforminputby

,Call8-pointon

toobtain

計(jì)算機(jī)視覺(jué)ShapeStructureFromMotionI專題知識(shí)專家講座第29頁(yè)Normalized8-pointalgorithm(0,0)(700,500)(700,0)(0,500)(1,-1)(0,0)(1,1)(-1,1)(-1,-1)normalizedleastsquaresyieldsgoodresultsTransformimageto~[-1,1]x[-1,1]計(jì)算機(jī)視覺(jué)ShapeStructureFromMotionI專題知識(shí)專家講座第30頁(yè)Normalized8-pointalgorithmA=[x2(1,:)‘.*x1(1,:)'x2(1,:)'.*x1(2,:)'x2(1,:)'...x2(2,:)'.*x1(1,:)'x2(2,:)'.*x1(2,:)'x2(2,:)'...x1(1,:)'x1(2,:)'ones(npts,1)];

[U,D,V]=svd(A);

E=reshape(V(:,9),3,3)';

[U,D,V]=svd(E);

E=U*diag([D(1,1)D(2,2)0])*V';%Denormalise

E=T2'*E*T1;[x1,T1]=normalise2dpts(x1);[x2,T2]=normalise2dpts(x2);計(jì)算機(jī)視覺(jué)ShapeStructureFromMotionI專題知識(shí)專家講座第31頁(yè)Normalizationfunction[newpts,T]=normalise2dpts(pts)c=mean(pts(1:2,:)')';%Centroidnewp(1,:)=pts(1,:)-c(1);%Shiftorigintocentroid.newp(2,:)=pts(2,:)-c(2);

meandist=mean(sqrt(newp(1,:).^2+newp(2,:).^2));scale=sqrt(2)/meandist;

T=[scale0-scale*c(1)0scale-scale*c(2)001];newpts=T*pts;計(jì)算機(jī)視覺(jué)ShapeStructureFromMotionI專題知識(shí)專家講座第32頁(yè)RANSACrepeatselectminimalsample(8matches)computesolution(s)forFdetermineinliersuntil(#inliers,#samples)<95%||toomanytimes

computeEbasedonallinliers計(jì)算機(jī)視覺(jué)ShapeStructureFromMotionI專題知識(shí)專家講座第33頁(yè)依據(jù)基本矩陣預(yù)計(jì)運(yùn)動(dòng)2.預(yù)計(jì)運(yùn)動(dòng)參數(shù)T:根據(jù)基本矩陣性質(zhì)R:根據(jù)計(jì)算機(jī)視覺(jué)ShapeStructureFromMotionI專題知識(shí)專家講座第34頁(yè)(2)直接依據(jù)外極線方程預(yù)計(jì)運(yùn)動(dòng)理想情況下：因?yàn)檎`差，改求：計(jì)算機(jī)視覺(jué)ShapeStructureFromMotionI專題知識(shí)專家講座第35頁(yè)Structurefrommotion計(jì)算機(jī)視覺(jué)ShapeStructureFromMotionI專題知識(shí)專家講座第36頁(yè)Structurefrommotionstructureformotion:automaticrecoveryofcameramotionandscenestructurefromtwoormoreimages.Itisaselfcalibrationtechniqueandcalledautomaticcameratrackingormatchmoving.Unknowncameraviewpoints計(jì)算機(jī)視覺(jué)ShapeStructureFromMotionI專題知識(shí)專家講座第37頁(yè)Structurefrommotion2Dfeaturetracking3Destimationoptimization(bundleadjust)geometryfittingSFMpipeline計(jì)算機(jī)視覺(jué)ShapeStructureFromMotionI專題知識(shí)專家講座第38頁(yè)StructurefrommotionStep1:TrackFeaturesDetectgoodfeatures,Shi&Tomasi,SIFTFindcorrespondencesbetweenframeswindow-basedcorrelationSIFTmatching計(jì)算機(jī)視覺(jué)ShapeStructureFromMotionI專題知識(shí)專家講座第39頁(yè)KLTtracking/~stb/klt/計(jì)算機(jī)視覺(jué)ShapeStruc