CODEISPOETRY » math

Face / Mimik tracking in AS3

admin — Mon, 20 Jul 2009 09:03:10 +0000

http://play.blog2t.net/terminator-salvation-realtime-machine-vision-as3/

Once again Tomek alias Og2t wrote a great flash demo. This time its about machine vision and just awesome. It’s motion based head / eyetracking with a lot of cool “hacks” like blob detection – histogram analysis – motion detection and more. Furthermore Tomek wants to provide a full framework.

This is a part of the whole video filter framework I am developing just now, the inspiration came from Joa Ebert’s Image Processing library (as far as I know, he’s cooking a complete rewrite). The full source code (including Pixel Bender kernels and examples) will be soon released on Google Code and will feature face/eye tracking/gestures and few other things (surprise!) A lot of people are very sceptic about the whole eye tracking idea, they don’t believe it’s precise enough to make any use of it – I will prove that it is, and it works! (just watch closely how it tracks my eyeballs on the video!)

I tried it, but I still prefer the camshift algorithm for face detection. Anyway: GREAT WORK!

Must see -> Erik Natzke

admin — Fri, 10 Jul 2009 15:47:35 +0000

As far as i got it Erik Natzke creates art using tech (Flash for sure, maybe processing too). You can also buy his prints. Here are some images. :

SURF the next big thing

admin — Wed, 08 Jul 2009 22:12:36 +0000

SURF is a computer vision technique which helps doing crazy things ;) Here is some footage showing whats possible. NOTE: no flash implementation yet ;) – I think it wouldn’t work -> performance

openFrameworks & OpenCV SURF from postspectacular on Vimeo.

Augmenting a Postcard from Julian Oliver on Vimeo.

Optimized edge detection algorithm.

admin — Wed, 01 Jul 2009 13:35:43 +0000

I optimized the edge detection (sobel) algorithm found here a little. Still there is a lot potential for optimizing things. You can find tipps in joa’s wiki

Here is my code

private function detectEdges(bmd : BitmapData) : void
{
 
 var myGausianFilter : ConvolutionFilter =
 new ConvolutionFilter(5, 5, 
 [2,4,5,4,2,4,9,12,9,4,5,12,15,12,5,4,9,12,9,4,2,4,5,4,2],
 115);
 
 // Edge Data
 findEdges();
 
 function findEdges() : void
 {	
  //Apply Smoothing Filter
  bmd.lock();
  bmd.applyFilter(bmd, bmd.rect, new Point(0, 0), myGausianFilter);
 
  //Create New Bitmap to hold edge data
  ourEdgeData = new BitmapData(bmd.width, bmd.height, false);
 
  var gx : int;;
  var gy : int;
  var gray : uint;
 
  //Loop through original data and calculate edges
  for(var w : int = 0;w &lt; bmd.width; w++)
  {
   for(var h : int = 0;h &lt; bmd.height; h++)
   {
    pV0 = int(bmd.getPixel(w, h - 1)/ uint(0xffffff)*255);
    pV45 = int(bmd.getPixel(w + 1, h - 1)/ uint(0xffffff)*255);
    pV90 = int(bmd.getPixel(w + 1, h)/ uint(0xffffff)*255);
    pV35 = int(bmd.getPixel(w + 1, h + 1)/ uint(0xffffff)*255);
    pV80 = int(bmd.getPixel(w, h + 1)/ uint(0xffffff)*255);
    pV225 = int(bmd.getPixel(w - 1, h + 1)/ uint(0xffffff)*255);
    pV270 = int(bmd.getPixel(w - 1, h)/ uint(0xffffff)*255);			
    pV315 = int(bmd.getPixel(w - 1, h - 1)/ uint(0xffffff)*255);	 
 
    gx = (pV45+(pV90*2)+pV35)-(pV315+(pV270*2)+pV225);
    gy = (pV315+(pV0*2)+pV45)-(pV225+(pV80*2)+pV35);
    gray = Math.abs(gx) + Math.abs(gy);
 
    // Check to see if values aren't our of bounds
    if(gray &gt; 255)
     gray = 255;
 
    if(gray &lt; 0)
	  gray = 0;
 
    // Build New Pixel
    newPixelValue = (gray &lt;&lt; 16)+(gray &lt;&lt; 8)+(gray);
 
    // Copy New Pixel Into Edge Data Bitmap
    ourEdgeData.setPixel(w, h, newPixelValue);	
   }	
  }
 
  bmd.unlock();
 }
 
 function getGray(pixelValue : uint) : uint
 {
  var red : uint = (pixelValue &gt;&gt; 16 &amp; 0xFF) * 0.30;
  var green : uint = (pixelValue &gt;&gt; 8 &amp; 0xFF) * 0.59;
  var blue : uint = (pixelValue &amp; 0xFF) * 0.11;
 
  return (red + green + blue);
 }
 
 _bmp3.bitmapData = ourEdgeData;
 
}

Edge detection in Flash

admin — Tue, 30 Jun 2009 08:41:09 +0000

here is a good post about edge detection with the sobel-operator

sobel-edge-detection-in-flash

Works really well, but does not fit my needs.

Face tracking!

admin — Fri, 26 Jun 2009 14:27:03 +0000

I found http://www.mukimuki.fr/flashblog/2009/05/28/camshift-smile-you-are-tracking/

The Camshift algorithm is propably the best tracking algorithm for the flash platform. It’s stable, fast and good coded. Love it!

Flashtip : Zoom in a clip to a specific point

admin — Tue, 27 Jan 2009 13:31:26 +0000

Imagine you have to zoom in a mc on a specific point.
This math snippet works fine for me (I used TweenLite in my Code):

var tmpEndX : Number = ((spr.width)-(width*scaleFact))/2;
var tmpEndY : Number = ((spr.height)-(spr.height*scaleFact))/2;
 
var tmpXShift : Number = (mouseX-(width/2))*scaleFact;
var tmpYShift : Number = (mouseY-(spr.height/2))*scaleFact;
 
TweenMax.to(this, 0.5,{scaleX : scaleFact});
TweenMax.to(this, 0.5,{scaleY : scaleFact});
TweenMax.to(this, 0.5,{x : tmpEndX - tmpXShift});
TweenMax.to(this, 0.5,{y : tmpEndY - tmpYShift});

Demo and full Class here

vertex detection in quad

admin — Sat, 01 Nov 2008 16:09:05 +0000

Problem:

You have a quad outline as point data, and want to detect the four vertices

I googled that problem a lot but couldn’t find any solution that fitted my needs. Moreover I didn’t want to copy paste something I do not really understand, so I solved it my way

In this case I know the points D and B (detecting theese points is easy, but in some special cases you track D and A but that’s an other problem). Now I wanted to store the outline in a point array. So I started at D doing this:

_currentPt = findStartPoint(f_bmd, f_rect);
while(findNextBlackPixel(f_bmd, _currentPt, 1 , 0 , 0, 1));
while(findNextBlackPixel(f_bmd, _currentPt, 0 , 1, -1, 0));
while(findNextBlackPixel(f_bmd, _currentPt, -1 ,0, 0, -1));
while(findNextBlackPixel(f_bmd, _currentPt, 0 ,-1, 1, 0));
 
...
...
 
private function findNextBlackPixel(f_bmd : BitmapData,  pt : Point, shiftA : int, shiftB : int, shiftC : int, shiftD : int) : Boolean
{
	if(f_bmd.getPixel32(pt.x + shiftA, pt.y + shiftB) == 0xff000000)
	{
		_currentPt.x = pt.x + shiftA;
		_currentPt.y = pt.y + shiftB;
		_ptsAr.push(new Point(_currentPt.x, _currentPt.y));
		return true;
	}
	else if(f_bmd.getPixel32(pt.x + shiftC, pt.y + shiftD) == 0xff000000)
	{
		_currentPt.x = pt.x + shiftC;
		_currentPt.y = pt.y + shiftD;
		_ptsAr.push(new Point(_currentPt.x, _currentPt.y));
		return true;
	}
	_ptsAr.push(new Point(_currentPt.x, _currentPt.y));
	return false;
}

This is how I get the outline Array (Blue Line)

Now here’s the problem:

Between D and B has to be a point with MaxHeight. So how can I calculate that height value.

My solution:

Looping over triangles with differenc C values. Then you have a triangle with known Points DBC DBC’ DBC” and so on.

Fancy math:

Calculating the length from DB BC DC via sinus (easy). Now we have a trianlge DBC without any right angle. BUT we know the length of each side. After some researching I remembered this cosine math.

So we know the angle BDC now. With simple sinus we can calulate the MaxHeight. Comparing the heights -> you’re done :D

Have fun.