Lib4U

‎"Behind every stack of books there is a flood of knowledge."

3D Realtime Face Reconstruction with Matlab and Kinect

facerec-489x250

This code written in Matlab demonstrates how to reconstruct a face from information captured by the Kinect. What it does, is that it makes a pointgrid, and fills that grid...

This code written in Matlab demonstrates how to reconstruct a face from information captured by the Kinect. What it does, is that it makes a pointgrid, and fills that grid in with the appropriate color.

I managed to map the texture on the pointgrid aquired from the Kinect 3D image.

This is what my program does:

  1. Gets 3D + live image
  2. Converts 3D coordinates to real-world coordinates
  3. Makes a trigonial mesh
  4. Fills in the entire trigonial mesh with the correct colors (from live image)
  5. Starts from 1

Specs:

  • ~2 seconds per frame (0.5 FPS)
  • Accurate to ~5mm
  • Works from 50cm to at least 700cm
  • 0.3 MP Resolution
  • If not set to realtime (=postprocessing) framerate would be 30FPS

See below this youtube of my face, which consists of around 40.000 datapoints:

And then here follows the code. Looks basic? Yeah it’s fairly basic.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
 
clc
clear all
close all

% Create context and read xml file at first
context = mxNiCreateContext('SamplesConfig.xml');

%% Initialise FIGURE
width = 640;
height = 480;
% depth image
% figure(1), h1 = imagesc(zeros(height,width,'uint16'));
%            set(gcf,'MenuBar','none'); set(gca,'Position',[0 0 1 1]);
% rgb image
% figure(2), h2 = imagesc(zeros(height,width,3,'uint8'));
%            set(gcf,'MenuBar','none'); set(gca,'Position',[0 0 1 1]);
% rgb+depth image
%figure(3), h3 = imagesc(zeros(height,width,3,'uint8')); hold on;
%           set(gcf,'MenuBar','none'); set(gca,'Position',[0 0 1 1]);
%           h4 = imagesc(zeros(height,width,'uint16')); hold off;
% converted XYZ plot 
%figure(4); h5 = mesh(zeros(height,width,'double'),zeros(height,width,'double'),zeros(height,width,'double'));
%           axis([-800 800 -600 600 -10000 -1 ]); view(180,90);
%           xlabel('X [mm]'); ylabel('Y [mm]'); zlabel('Z [mm]');
%           title('XYZ ?iThe world coordinate system?j');
figure(5);  %h6=trisurf(t,p(:,1),p(:,2),p(:,3),1:size(p,1),'edgecolor','none')
            %axis equal
            %shading interp %colours are interpolated inside triangles

%% acquire images
for k=1:2
option.adjust_view_point = true;
% Acquire rgb and depth image
tic
[rgb, depth] = mxNiImage(context, option);

real_XYZ = mxNiConvertProjectiveToRealWorld(context, depth); % return XYZ[mm, mm, mm] on the world coordinate system

[M]=-mn32m3(real_XYZ); %dit duurt erg lang
[Mrgb]=mnrgb2mrgb(rgb); %dit duurt erg lang

itmp=sum(M~=0); %locate existing info
stmp=itmp~=0; %positive if there is info
M=M(:,stmp);
Mrgb=Mrgb(:,stmp);

itmp=M(3,:)>-800; 
stmp=itmp==1;
M=M(:,stmp);
Mrgb=Mrgb(:,stmp);

p=M';
C=double(Mrgb')/255;
t=delaunay(p(:,1),p(:,2));
colormap(C);

trisurf(t,p(:,1),p(:,2),p(:,3),1:size(p,1),'edgecolor','none')
axis equal;view(200,66)
shading interp

%real_X = real_XYZ(:,:,1);
%real_Y = real_XYZ(:,:,2);
%real_Z = real_XYZ(:,:,3);

% update FIGURE
% set(h1,'CData',depth); 
% set(h2,'CData',rgb); 
%set(h3,'CData',rgb); 
%set(h4,'CData',depth);
%set(h4,'AlphaData',double(depth/50));
%
%tmp = real_XYZ(:);
%tmp(find(real_XYZ==0)) = NaN;
%tmp = reshape(tmp,[height width 3]);

%set(h5,'XData',-tmp(:,:,1));
%set(h5,'YData',-tmp(:,:,2));
%set(h5,'ZData',-tmp(:,:,3));

drawnow;
disp(['itr=' sprintf('%d',k) , ' : FPS=' sprintf('%f',1/toc)]);
end
% Delete the context object 
mxNiDeleteContext(context);

MN322M3.m function

1
2
3
4
5
6
7
8
function [M]=mn32m3v2(MN)
X=MN(:,:,1)';
Y=MN(:,:,2)';
Z=MN(:,:,3)';
X=X(:);
Y=Y(:);
Z=Z(:);
M=[X Y Z];

MNRGB2MRGB.m function

1
2
3
4
5
6
7
8
function [M]=mnrgb2mrgbv2(MN)
X=MN(:,:,1)';
Y=MN(:,:,2)';
Z=MN(:,:,3)';
X=X(:);
Y=Y(:);
Z=Z(:);
M=uint8([X Y Z]);

 

Source:

http://www.timzaman.com/?p=495&lang=en

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Virtual Fashion Technology

Virtual Fashion Education

toitocuaanhem

"chúng tôi chỉ là tôi tớ của anh em, vì Đức Kitô" (2Cr 4,5b)

VentureBeat

News About Tech, Money and Innovation

digitalerr0r

Modern art using the GPU

Theme Showcase

Find the perfect theme for your blog.

lsuvietnam

Learn to Learn

Gocomay's Blog

Con tằm đến thác vẫn còn vương tơ

Toán cho Vật lý

Khoa Vật lý, Đại học Sư phạm Tp.HCM - ĐT :(08)-38352020 - 109

Maths 4 Physics & more...

Blog Toán Cao Cấp (M4Ps)

Bucket List Publications

Indulge- Travel, Adventure, & New Experiences

Lib4U

‎"Behind every stack of books there is a flood of knowledge."

The WordPress.com Blog

The latest news on WordPress.com and the WordPress community.

%d bloggers like this: