It's a tiny bit late maybe, but here are two image processing techniques that deal with automated thresholding and edge detection that I showed in my "2d or not 2d" talk in 2007 and in "The Pixel Whisperer" in 2008.

I've had a look at my presentation demos and repackaged their code into a single class called ThresholdBitmap. This is a BitmapData class with a few extras that help you if you plan to extract blobs or edges from a camera stream or some other bitmap.

The general problem when you do camera based experiments is that you have unnown lighting conditions. Using a simple fixed threshold might work at your development machine but might fail completely on a user's computer that has a different camera model or a different backdrop.

In image processing there are several automated thresholding methods, that are supposed to help with this problem, by looking at an image's histogram and adjusting the threshold to a level that separates the foreground from the background in an optimum way: Moment-Preservation, Maximum Entropy, Discriminant and Otsu. Those are quite capable in finding the optimum threshold level if you have relatively uniform lighting over the whole scene.

As an (better) alternative I included an adaptive threshold method which is implemented in PixelBender and which works very well also with uneven lighting, especially if you try to detect QR codes or other markers.

Furthermore the class includes a very fast edge detection (based on the thresholded image) which gives you nice 1 pixel wide edges in most cases.

Here is a demo that allows you to play around (you'll need a webcam)

Sourceview and code can be found here

Credits: the demo uses the super useful Minimal Components by Keith Peters and the Hi.RES! Stats by Mr. Doob.

Whilst playing around with some geometry that involved gradient fills I had to realize that the native implementation of Flash's createGradientBox() is unfortunately entirely useless as soon as you try to introduce rotation and scaling to a gradient. The reason seems to be that internally the order of matrix operations is unsuited for that task.

Maybe I've searched not thoroughly enough, but I didn't find any solution for this out there. So I've written a tiny class that fixes it. Here's a comparison of the difference between matrix.createGradientBox() and my new GradientMatrix.getGradientBox()

To use it you can use the same values that you would use in the native function, the only difference is that you don't have to create a new Matrix first. The syntax is var myGradientMatrix:Matrix = GradientMatrix.getGradientBox( width, height, rotation, tx, ty);

Download GradientMatrix.as here

I've started to have a bit of fun with the optimized Marilena face-tracking class and build a webcam based magic mirror. The specialty here is that it tries to displace your face only and leave the rest of the scenery untouched and additionaly adjusts the displacement scale based on how close you are to the camera:

Check out Manic Mirror at the Incubator

Yesterday Seb-Lee Delisle pointed us via Twitter to this great example of a Flash based real-time face detection. It turns out that already more than half a year ago Ohtsuka Masakazu had been porting the face detection part of OpenCV to AS3 and added the source code to the Spark project - which is like a Actionscript candy box full of surprises.

So I had a look at the source code and found one feature which I wanted to change: face detection is based on so-called Haar Cascades, simplified one could say that this is a very long list of zones that get checked for certain features in an image. In the orginal version this is an xml file of almost 1 MB size. Because this is so big the xml file gets compressed into a zip file where it fortunately becomes just 100K. So in the orginal version the zip file has first to be loaded and then unpacked and parsed. My consideration was that swfs get zipped anyway, so why not turn the whole xml file directly into a class?

So that's what I did. My adapted version does not need to load any external files anymore (which was my goal) and the whole swf becomes even about 50K smaller than the original version. I also did a few other optimizations like replacing Arrays with linked lists and reading a from ByteArray instead of using getPixel(). The downside is that since the Haar data is being hardcoded into the class now you will not be able to use this class to track anything else but faces (which in theory you could). But I expect that most people will not even know how to prepare a different Haar cascade recognition set - for example I'm one of those.

What's interesting is that this code will even run in Flash Player 9, since there is no PixelBender or Alchemy and neither the new Vector data type being used. Obviously by targeting Flash Player 10 one could even add a few more optimizations, but that's something for another weekend.

Example 1: Ohtsuka Masakazu's modified sample file

Example 2: Real-time webcam face tracking

Here's the modified sourcecode which includes two example files: Marilena_mod10.zip

Here are a few more of my entries for #tweetcoding:

"Moskitos":
g.clear();ls(1);o[i++]=[mouseX,mouseY,2*r(),2*r()]; for each(p in o){mt(p[0],p[1]); lt(p[0]+=2*s(p[1]*p[2]),p[1]+=2*s(p[0]*p[3])); };i%=2000

"Love in 3D":
if(i<250&&(x=y++)) with(addChild(new TextField)) text="<3",x=r()*500-250,y=r()*500-250,z=i*2-250, rotationX=rotationZ=r()*360;rotationY=i++

"Hyperdrive":
if (!o.b)addChild(o.c=new Bitmap(o.b=new BitmapData(500,500))),o.b.noise(2); z=s(i-=0.01)*15;o.c.z=-z;o.b.draw(this,o.o,o.o,"hardlight");

"Fibonacci Sequence":
if(!i)i=1,o.b=0,o.c=addChild(new TextField),o.c.autoSize="left",o.c.wordWrap=o.c.multiline=1; o.c.appendText(" "+i),o.a=o.b,o.b=i,i+=o.a

"All Invaders":
if(!i) g.beginFill(0),a=b=i=64,d=511; g.drawRect(a&=d,b&=d,i,i); g.drawRect(d-a+1-i,b,i,i); a+=r()<0.4?i:0;b+=r()<0.4?i:0;

"The Golden Days":
g.clear();g.beginFill(0); x=y=20;for(i=81;i--;g.drawCircle(40*a,40*b,d*2)) a=(i%9),b=int(i/9), d=m.sqrt((d=mouseX/40-a)*d+(d=mouseY/40-b)*d)

"Superstar":
g.beginFill(r()<0.4?0:1e8);q=r()*640;c=r()*480;a=2;u=r()*10; for(d=12;--d;(d<11?lt:mt)(q+u*a*s(i+=0.63), c+u*a*s(i+1.57)),a=7-a){}

Another two entries for #tweetcoding.

"1967":
i+=0.06;g.clear();g.beginFill(0);for(x=80;x--;){g.drawCircle(25+70*(x%9),25+70*(y=int(x/9)),70+(s(x%9*0.7+i+s(y*0.7+i)))*35)}

Grant Skinner has started a litte competition on Twitter called tweetcoding where the goal is to create something nice in 140 chars of Actionscript 3. And the fine folks at Adobe actually will give a free copy of Flash CS4 to the winner, who will be picked on March 1st.

As a starting point there is a small wrapper which predefines a few common functions and objects with short variable names. Above you see my first entry - which requires Flash Player 10 btw: which looks like this as a tweet

I just came across Sakri Rosenstrom's posts (1 2) about his methods of finding the first non-transparent pixel in a bitmap. Looks like he got inspired by my talk at MAX Europe - unfortunately he seems to have misunderstood my explanation back then. My bad - I should have posted my method a long time ago - so here we go. [Notice: it turns out that Sakri's method is faster than mine - please check the update at the bottom of this post]

Here's a little demo.

First you might ask what the hell this method is needed for at all. There are three applications that come to my mind right away: blob tracking for multi-touch interfaces (using my flood fill method), QR-code recognition and bitmap vectorization. All of them have in common that after having detected a blob of uniform color in a bitmap you want to trace the outline of it. In order to do that you need to have a starting point of which you know that it lies on the edge. That's what the method I will show you here does.

If we were programming in C the approach would be to start at the top left corner of the bitmap and do a getPixel(x,y) from left to right and top to bottom until you find a pixel that is not 0x00000000. But since we are using Actionscript there are a few methods in the BitmapData class that will be quite faster than a loop.

The workhorse in this case is the getColorBoundsRect() method. In case you forgot what it does: this method will return you a Rectangle which encloses all pixels of a certain color inside a BitmapData object.

The starting situation: we have a transparent bitmap which contains non-transparent pixels somewhere. In step one we use getColorBoundsRect to narrow down on the minimum area that still contains any non transparent pixels:

var r1:Rectangle = bitmapData.getColorBoundsRect( 0xff000000, 0, false );

If the height of r1 equals 0 we know that the image is completely empty and can stop rightaway. If not we can continue. And here's the trick. We've got a rectangle now and we know for sure that somewhere in the topmost row of that rectangle there is a non-transparent pixel. What we cannot count on is that it is the one at the very left, aka r1.topLeft - it's absolutely possible that there is just a single pixel set in that row and it can be anywhere between r1.left and r1.right.

But here's the trick: we simply use getColorBoundsRect again - unfortunately we cannot do this using the same bitmapData. We need to extract that first row of pixels into a separate temporary bitmap:

var temp:BitmapData = new BitmapData( r1.width, 1, true, 0 );
temp.copyPixels( bitmapData, r1, new Point());

Now that we've got a 1 pixel high bitmapdata which contains at least one non-transparent pixel we can continue:

var r2:Rectangle = temp.getColorBoundsRect( 0xff000000, 0, false );

The last step is to add the offset we have found here to the previously found top left corner of first colorbounds:

var startPoint:Point = r1.topLeft.add( r2.topLeft );

All together it looks like this:

public static function getFirstNonTransparentPixel( bmd:BitmapData ):Point
{
var r1:Rectangle = bmd.getColorBoundsRect( 0xff000000, 0, false );
if ( r1.width > 0 )
{
var temp:BitmapData = new BitmapData( r1.width, 1, true, 0 );
temp.copyPixels( bmd, r1, new Point());
var r2:Rectangle = temp.getColorBoundsRect( 0xff000000, 0, false );
return r1.topLeft.add( r2.topLeft );
}
return null;
}

Or you can download the EdgeFinder.as class here

[Important Notice] I should have made some speed test before recommending this method. It turns out that getColorBoundsRect() is not as fast as I thought it was. Sakri Rosenstrom's HitTest method is twice as fast as mine. So I've taken another look at his implementation and optimized it a bit more, making a kind of hybrid between both of our methods. My test tell me that this optimzed version is about twice as fast - I have updated the code in EdgeFinder.as already

public static function getFirstNonTransparentPixel( bmd:BitmapData ):Point
{
var hit_rect:Rectangle=new Rectangle(0,0,bmd.width,1);
var p:Point = new Point();
for( hit_rect.y = 0; hit_rect.y < bmd.height; hit_rect.y++ )
{
if( bmd.hitTest( p, 0x01, hit_rect) )
{
var hit_bmd:BitmapData=new BitmapData( bmd.width, 1, true, 0 );
hit_bmd.copyPixels( bmd, hit_rect, p );
return hit_rect.topLeft.add( hit_bmd.getColorBoundsRect(0xFF000000, 0, false).topLeft );
}
}
return null;
}

In an attack of serious procrastination during the last days I've added a few new pieces to my Incubator:

"Twittoscope" is a little Twitter mashup that analyzes the lengths of a Twitter user's tweets and generates a histogram from that - which can serve as a guide as to what kind of messages you can expect from that user in the long run.

"Feedback" is a webcam experiment which started with a little Pixelbender feedback filter but grew into a kind of hypnotic visual toy that wants to be explored.

In "Passing By" I have revisited my Anavision engine and have used it to create a meditative piece which uses copyright-free images from the flickr commons project to generate a never repeating illusion which is somewhere between staring outside a train window with the landscpe passing by and watching the reflection of nature in a water surface.

"Aurora Binaris" is best descibed as "binary nothern lights". It is also based on flickr commons images, but they are only used to create some organic looking randomness.

In my "Here be Pixels" talk one of the things I showed was how to use a combination of noise and the displacementMap filter to encrypt and decrypt images or data with Flash. In case you didn't see it, here's the principle behind it as a Peacock composition:
The following post is about a related issue which uses a very similar technique, but I will also explain the encryption decryption method along the way.

Imagine you want to visit every pixel in an image in random order but you do not want to visit any pixel more than once. Obviously what you cannot to is to simply pick a series of random x/y coordinates and hope that you will hit non-visited pixels as often as possible. The problem is that even for a very small image of 100x100 pixels (which contains 10000 pixels) the probability that you hit a non-visited pixel will change from 100% (when you set the first pixel) to 0.01% (when you try to hit that last non-visited pixel) whilst you proceed through the image.

So another approach you might be tempted to try is to first store a list of all possible coordinates in an array then shuffle that list and finally visit the coordinates in the order they get served from that array. This will work and for small images it might even be okay to create and shuffle a list of 10000 elements. But when you get into areas of typically sized images this approach is definitely not optimum. Having to shuffle 480000 items in realtime doesn't sound like fun.

So what's the alternative? The one way to do it is actually being used internally by the Flash player - inside the notorious BitmapData.pixelDissolve() function. It uses a so called linear feedback shift register which is a really nifty numerical apparatus that produces a unique random-looking order of all integers between 1 and x. The only issue is that it's pretty hard to implement a random seed for this technique which is why you don't find that as a property in the Flash implementation. This means that for each image of the same dimension the order in which the pixels are visited will always be identical.

But there's obviously another option. Imagine an image like a gigantic 2-dimensional Rubic's Cube. Now shift each vertical pixel column by a different random amount up or down. Those pixels that end outside the image will wrap around and be reinserted on the other side. Repeat that process for all pixel rows. Then once more for the columns and for the rows again. This is the equivalent of randomly rotating a magic cube. Now the interesting part is that you will not loose a single pixel in this process. No pixel will be accidentaly overwritten by another one moving over it. This means each original pixel will still be there only on a different location. What's even nicer is that if you knew the exact order in which the columns and rows had been offset you could revert that process and unscramble the image when you just apply it backwards. That is what actually happens in the Peacock example above. BTW - this technique is not limited to bitmaps, you can use it to scramble and unscramble any kind of data.

But what I actually wanted to show you here is another class which employs no bitmapData, noise or displacementMap filters at all but just uses plain math to achieve a similar result: the CoordinateShuffler class will return you a list of uniquely shuffled coordinates for images of arbitrary sizes. Since it only needs to create two small lookup tables it is very lightweight and fast and should come handy whenver you want to check a part of an image's pixels randomly, spread elements over an image or create a random transition but want to make sure that in the end all pixels are covered.

In the demo you can set the random seed, the amount of horizontal + vertical shuffle rounds and the size of the lookup table which controls the amount of each column's and row's offset. The smaller the table size is the less random the shuffling will look, nevertheless this might be still interesting for certain graphical effects. The long slider controls the index, meaning "give me the next 500 random coordinates starting from index 10012". When you select "accumulate" it means "give me all random coordinates from index 0 up to index 53100".

In the end this might look like just a different kind of pixelDiffusion filter but believe me - it has many more interesting uses. For example the AutoPainter and the Stippling hubs inside of Peacock are using it, too.

CoordinateShuffler Demo | source