October 04, 2005
Flash BitmapExporter: Compress and Save Images

Here is a demo for something that I hope may prove useful for some people. I have written a BitmapExporter class which allows to save PNGs or JPEGs out of swf files. There have been some Flash 8 demos already that show this possibility, so the novelty factors of my class are that the images get compressed before they are sent to the server and that Flash remoting is used to transfer the data. On the server side I'm running this with AMFPHP (I have exchanged that for a regular PHP script which works with loadVars and POST requests) in conjunction GD (which comes usually pre-installed with PHP) to recreate the images.

The demo allows you to doodle some drawings and to save them to your hard drive as a JPEG. Because of its security sandbox Flash cannot access your harddrive directly. It first has to send the image data to the server and then commence a download from there. We are so used to the small filesizes of JPEGs or PNG that it's quite surprising to see what size an uncompressed image really has. The canvas in this demo for example has a dimension of 500x425 pixels. That's 212500 pixel x 4 bytes per pixel = 850000 Bytes or about 830K.

The challenge is to create a compression algorithm in ActionScript which will process the image data with reasonable speed without overexerting Flash. I didn't want to start studying the JPEG or PNG format specifications, so I rolled my own method. It uses an indexed color palette (but without reducing the amount of colors) and a run-length encoded table of the offsets of the colorindices of the consecutive pixels (wow, that sounds complicated). In reality that means that it will compress noisy images not so good as graphics. Therefore I have built in an optional method to reduce the bit-depths of images which does improve the compression ratio for photos and webcam shots.

The image is not necessarily processed in one block - the blocksize can be varied based on package size as well as processing time. Doing this the Flash plugin remains responsive and on top of that it allows for a bit "parallel processing": whilst the server is receiving data and creating the first part of the image (which takes the longest time in the process), the swf file is already compressing the next data block. Now isn't that clever?

So here comes the demo. You can vary the brush's start and end size by using the arrow keys on your keyboard. Clicking in the palette will change the brush color. And one more thing: please be kind with my traffic bill - this is a functional demo, please don't use it to save hundreds of images. Even though the images do get compressed it still uses up quite some bandwidth if thousands of people do this.

[Update:] Here's the link to the source code: http://www.quasimondo.com/scrapyard/BitmapExporter_v22.zip
Please read the license information in the BitmapExporter.as file, especially if you are planning to use this class in a commercial project.

Posted at October 04, 2005 10:39 PM | Further reading
Comments

Hey that's cool. I started doing this last night but it got late. I wasn't happy with RLE so I'm looking into LZ77 either as an alternative, or on top. LZ77 runs pretty quick in ActionScript and there's plenty of Java versions out there that just need re-data-typing. What compression ratio's are you getting from your RLE+color tables?

Posted by: Richard Leggett on October 4, 2005 11:14 PM

The ratio for the empty white canvas of the demo is 0.03% - For a noisy image it can be as worse as 90%. BTW - the ratio and the amount of transfered data will be displayed after you have saved the image. It really depends on the content and on the amount of packages you split the image into. But it is definitely worth to check out further compression methods.

Posted by: Mario Klingemann on October 4, 2005 11:30 PM

Nice demo, but two issues:

- Can't seem to change color in drawing app
- Once an image is "saved" where did it go? It does not appear.

Also, do you intend to post source code, both AS and PHP?

Thanks a lot. :)

Matt

Posted by: Matt on October 4, 2005 11:37 PM

If I draw more than just one or two lines, it hangs partway through the upload process.

Posted by: David R on October 4, 2005 11:39 PM

@Matt: Well actually a "Save Image" system-box should appear and you then decide where to save the jpeg on your harddrive. And to change the color you should only have to click into the color gradient field. Hmmm that's strange. But you are sure that you have the Flash 8 plugin installed? Are you maybe on a Mac?

Is anybody else having this problem, too?

Posted by: Mario Klingemann on October 4, 2005 11:42 PM

If it hangs at the moment it's maybe because my server is pretty busy right now. The GD part seems to suck quite a lot of CPU cycles.

Posted by: Mario Klingemann on October 4, 2005 11:44 PM

Mario - sorry for the alarm. I was using a, um, "funny" player. ;)

I'm sure no one else will have this problem.

Got it to work magically.

... which brings me to my original question - you gonna give us source code? :)

Posted by: Matt on October 4, 2005 11:54 PM

@Richard:: anything that involves taking substrings in AS is suicide. I had implemented a gzip algorithm in AS (basically a PNG implementation) and it took ridiculously long to process. LZ77 should suffer from the same issues as well. 'Relatively fast in Actionscript' can still be dog slow if you have 800k of data to take care of. Don't forget the additional overhead as anything out of the base64 chars gets munched (strings are not binary safe). My own implementation uses either GD or ImageMagick as doing setPixels in GD for images of this size is dog-slow. On the client-side I believe I'm using roughly the same algorithm as Mario. However I do all of the encoding on the client-side, then send to PHP. With the ImageMagick method, reconstructing 600k of data takes about 3 seconds on a non accelerated PHP installation. On the client-side it takes about 2 seconds to encode. I implemented different versions of it, some using diff-encoding (Left filter in PNG), some using regular encoding; I didn't see any difference in output size though. I'd be interested in seeing the palette code; I've personally been able to reconstruct only a graycale palette using thresholds... My own compression for 100% white is about... 2 bytes per 100 pixels, compared to 400 bytes using strict base64 encoding, so 2:400 ~ 0.5%. Full-color should yield same results as you, depending on the palette algo you're using. It should be on IFBIN soon.

Posted by: Patrick Mineault on October 4, 2005 11:57 PM

For the ImageMagick part... Once decoded on the server-side, I save as a .rgb file (it's simply a large rbg string really), then ask ImageMagick to convert to PNG24. Cuts CPU usage ~90-95%.

Posted by: Patrick Mineault on October 5, 2005 12:03 AM

@Patrick: ImageMagick sounds good. I went for GD right now because I think its availability is higher on the typical PHP servers. But of course the option to choose between GD and ImageMagick would be a useful addition. I'm looking forward to see your implementation.

Posted by: Mario Klingemann on October 5, 2005 12:09 AM

OK, thanks Mario, though I'd be super jazzed if I even had a sample of what you have working now, to study.

I will wait, though, if I must. :)

Best,

Matt

Posted by: Matt on October 5, 2005 12:13 AM

@patrick Thanks for the warning, I might just abandon that idea for now and opt for something a little less CPU intensive :)

@mario Cool thanks for the info, I'm eager to see this code also. ;)

Posted by: Richard Leggett on October 5, 2005 12:19 AM

I've actually contacted the guys at Boutell to see how I could write a string in GD or GD2 format. The issue is that we have a raw string which would be ridiculously simple to batch read for GD. I've implemented a BMP writer in my class that writes the 54 bytes of BMP header, and then it dumps the whole BMP string in the file; I can open the file in Paint or Fireworks, it's fine, only it's horizontally reversed because of byte-order in the BMP format. So I can write a valid image _without_ using GD at all, the issue is that GD doesn't support BMP or BMP-like formats, except through proprietary, undocumented GD and GD2 formats. I tried to read the C code and it seems it should be easy enough to write a GD string but I haven't figured out the details of the format. So you've got this wonderful BMP file but it's useless to GD and you have to use imagesetpixel in a for loop on which PHP chokes. ImageMagick will read BMP and .rgb files so it saves that step, but the better solution would be to construct a GD string I believe.

Posted by: Patrick Mineault on October 5, 2005 12:41 AM

Here's the BMP writing method if you're interested:

http://www.5etdemi.com/experiments/writebmp.phps

Posted by: Patrick Mineault on October 5, 2005 12:45 AM

This is a great example. Thanks for putting the time in to make this sample. I look forward to checking out the source.

Posted by: Tim Brady on October 5, 2005 02:36 AM

Hi,

Well if you release the code, I'll convert the PHP part to Coldfusion if you find that ok...

Well next week friday I'm in a plane for 11 hours(going to MAX), so if you have some code available for me to convert to CF, that would just be great.

Looks very usefull this image creator... already have some ideas for it :-)

Posted by: Michael on October 5, 2005 05:10 AM

Ive been messing with saving images on the server recently as well.

great example and great discussion. I'll be having a look into some of this compression stuff as soon as i get a chance.

Posted by: Tink on October 5, 2005 03:00 PM

Wicked.....simply wicked!!

Posted by: Campbell on October 6, 2005 04:44 AM

can you send me the code how to do it please it is very urgent
thanks
Rami

Posted by: Rami cohen on October 9, 2005 12:46 PM

Hey Mario, this is directly to you but I figured others reading might want to pick up the task too.

Basically there is really smart guy called Peter Bourke, I've been readign alot about his fractals and geometry code math recently and done some cool experiments in flash, but anyway, he has this artcile on image compression which I don't fully understand but you might. He unfortunately hasn't included the c source (which converts to AS2 so damn easily!), but it may give you some ideas on how to better compress images.

http://astronomy.swin.edu.au/~pbourke/colour/ycc/

Posted by: Jon B on October 10, 2005 02:33 AM

Thank you Jon, that is a very interesting article and the C code is actually not necessary because the concept is easy to grasp. It sounds definitely like something I should try.

Posted by: Mario Klingemann on October 10, 2005 10:20 AM

Well here's hoping you create some great compression classes, that would be awesome. Also I'm hoping someone creates an exe 'wrapper' (maybe a new screenweaver version) that will allow saving of images direct to the desktop.

Posted by: Jon B on October 10, 2005 10:42 PM

ooh, by the way, you must have heard about the new binary sockets in the flash player 8.5, surely they would allow you to get away from amfphp and provide a fast way to transfer the images to the server?

Posted by: Jon B on October 10, 2005 10:47 PM

I have put together a flash version of Paul Bourke's YCrCb compression as discussed in his paper that Jon B pointed me to above. This uses Flash 8's filters and is fast and the example swf lets you play with block size, color space/blur/etc and gives size reports and lets you see the effects.
As you can see even large block sizes can look good, but 4 is looks good. The link is
http://timmaffett.com/source/bourkeYCrCbCompressionExperiment.html
and source is available as well at
http://timmaffett.com/source/bourkeYCrCb.zip

I have a lzw compression engine running that I am going to use this (for non small palette bitmaps).
I plan on making a PHP script to write gd2 bitmaps (As Patrick Mineault points out above) to convert quicky on server. I look forward to your solution as well Mario!

-tim

Posted by: tim maffett on October 13, 2005 09:14 AM

That looks really great Tim - thank you for posting! I will see if I can integrate your script into my class.

Posted by: Mario Klingemann on October 13, 2005 10:33 AM

That is awesome, I don't quite get everything that is going on tho but I'm glad to see it works and does it well. I feel that soon there will be a brilliant solution for compressing data and sending it the server. What would be cool is if there were one compression class that automatically analysed the image and chose the best compression algorithm based on few parameters (like size vs quality, lossless or lossy, maybe even a target size) - this would mean it would be insanely simple to transfer any type of images to the server.

I'm looking forward to what comes from all this, I only wish I was smarter and could attempt to help :(

Posted by: Jon B on October 13, 2005 11:27 AM

Yes, a script that automatically chooses a different compression method depending on the content of the image would be great. The only downside I see is that in the time it takes to analyse the image some compression algorithm could have already done the work and started sending the data. But it is definitely something to explore further.

Posted by: Mario Klingemann on October 13, 2005 11:38 AM

I've been messing around with this for the last few days, using YCC (although not nearly as neat as Tim's ColorMatrixFilter) and LZW compression. Unfortunately the largest issue is being able to get this data to the server (I'm also using AMFPHP).

As far as I can see the only "packed" data type to send is a string, and as Patrick pointed out it appears that only Base64 strings will safely get through. I haven't looked into the AMFPHP source to see how the unserializing works and whether this can be remedied, but in the meantime that means a (small) 33% increase from the YCC or RGB Base256.

However, using LZW compression gives a multibyte string, and with small noisy images the LZW indices can get into the many thousands. Even with some dodgy corner cutting the best I have reliably converted this to Base64 is with a 133% increase from the LZW string (7 Base64 bytes per 3 multibyte LZW chars). In optimal cases this can be cut down to 5 Base64 bytes, but only with either tiny image chunks or flat/gradient colours. In the majority of cases I found that this Base64'd LZW string was actually larger than without the LZW at all...

Perhaps if we could transfer Base256 strings to PHP it would be worthwhile, but in the short-term I am going leave out any "compression", and just use YCC.

Posted by: Grant Cox on October 14, 2005 02:26 AM

has anybody looked a GhostWire's PHPObject as an alternative to AMFPHP in this instance? I'm not sure whether it would help in the transfer of compressed data, but it might insightful.

Also, although it's a while off yet I wouldn't mind knowing if I was right in thinking that the forthcoming binary sockets in Flash Player 8.5 would be of help in this area.

Posted by: Jon B on October 14, 2005 04:13 AM

FYI - I send my data as Numbers to the server and not as Strings. In my theory this is the most compact format you can send if the protocol is binary. The problem with strings is that you cannot use the chr(0) character in Flash. One workaround if you still want to use strings is to replace all occurences of chr(0) with chr(1), but this will probably only work on the pixel level and not for compressed data.

Posted by: Mario Klingemann on October 14, 2005 10:32 AM

If \x00 is the only byte value that can't be stored in a string in flash, why not invent some sort of simple encoding/escaping (that you apply to bytes emitted from the compression algorithm) eg. \x00 becomes \x01\x02 and \x01 becomes \x01\x01 ...all other values are left unchanged... Then you reverse this on the server before uncompressing. (when you encounter a \x01, check the next byte to determine if it's a \x00 or a \x01)

Posted by: ? on October 14, 2005 02:53 PM

The problem is that you are always one number short because you get values between 0..255 but can only cram them into the range 1...255. For images this doesn't matter, because the eye will not really notice the difference between 0x000000 and 0x010101, but if you are using a compression algorithm there is a huge difference between repeating a pattern 0x0100 times or 0x0101 times.

Posted by: Mario Klingemann on October 14, 2005 02:57 PM

This is all getting far too technical for me to even follow, is there anywhere that explains colour math and hex numbers and stuff for a relative n00b in the field?

Completely unrelated but I've been trying to find away to 'mix' argb hex values for instance, imagine a pixel coloured 0xCCFF0066 and then imagine another pixel coloured 0x33EE55FF 'laid' over the top - what would the new colour be? I have tried many ways to do it that I think are logical but it doesn't seem to work so I think I'm very dumb :(

Sorry for digressing tho.

Posted by: Jon B on October 15, 2005 01:53 AM

One easy way to do that is to use Flash to make such a calculation:

var a:BitmapData = new BitmapData(1,1,true,0xCCFF0066);
var b:BitmapData = new BitmapData(1,1,true,0x33EE55FF);
a.draw(b);
trace( a.getPixel32(0,0));
a.dispose();
b.dispose();

To calculate it manually involves some Math because of the alpha in both channels. I'd have to look that up.


Posted by: Mario Klingemann on October 15, 2005 12:17 PM

But if you want to do it the hard way, here is a little bit of information about alpha compositing: http://en.wikipedia.org/wiki/Alpha_compositing

Posted by: Mario Klingemann on October 15, 2005 12:28 PM

Hey Mario, I don't want to turn this into a Q/A or forum type situation so I wont keep going on about it, thanks for responding kindly tho. The new bitmap method isn't viable really since I'm altering around 4000+ pixels per second (dependent on computer speed), I might give it a go but I think it will prove a little inaccurate and may use up too much memory. I am surprised how hard it is to 'mix' colours tho, the few n00b-logical approaches I tried never seemed to work propertly and always ended up going 'out of range' so to speak and disappearing giving the effect that my progressive image generation was 'eating itself' - not nearly as cool as it sounds tho. I thought maybe the colorTransform class could be used to accurately simulate placing one colour ontop of another but I can't figure out how it works - maybe some more colour tutorials or demos would be nice, your matrix demo was nice although I am still confused by matrixes.

Thanks for your help, and once again sorry for digressing.

Posted by: Jon B on October 15, 2005 06:35 PM

Jon - I don't mind to discuss this here - it might interest others, too. As for the method shown - of course you wouldn't do it one pixel at a time, but with a complete bitmap containing 100000+ pixels at a time. It would be nice if you could provide a link to an example on what you are trying to achieve. But of course you can also mail me if you prefer that.

Posted by: Mario Klingemann on October 15, 2005 09:22 PM

Ok well a link is hard, but basically I'm recreating processing experiments in flash using BitmapData, so I needed a fast way to lay down individual pixels, so far I have created many 'Strange Attractors' with reasonable success, some of which are based on example processing code from Jared Tarbell. I can do it with single colours using a very simple 'incremental' type of colour addition stopping at 255, but this is rather dull especially when you see some of the nice colours in JT's examples. Generally the process involves laying down single pixels to a formula very rapidly and watching them 'build up', the semi-transparent nature is important in this process, however come to think of it, it should be possible without using alpha at all, just more simple colour 'blending', well maybe at least, I'm not sure if the colours would be the same tho. But that is what I have been doing, it's not as fast as processing code obviously, but it's ok and come flash player 8.5 I'm sure it can be optimised further and be pretty fast.

Essentially the method I'm trying to emulate in flash is the processing 'point', but whereas processing allows multiple points at the same x,y in Flash this obviously means creating layering objects and this slows everything down, hence the required 'blending' technique, preferably one that can blend any number of hex colours in one go rather than just two at a time since sometimes multiple colours get laid down in the same place at the same time. If there is anyway to avoid converting the hex values to individual argb values and then converting back to hex to use setPixel32 then i think it would be better and faster and speed is all important.

Colour is a confusing subject, especially when it is defined using hex numbers, a class to simplify some colour processing techniques would be useful, I'm still not absolutely sure that what I want can't be achieved using the built in classes, but I don't fully understand 'multipliers' and 'offsets'.

Posted by: Jon B on October 16, 2005 12:45 AM

I understand where you are heading and I'd say that for this task the draw() command is your friend. Especially if you use it with the "add" or "subtract" blend mode and colors in a very low range like 0x010101. This will accumulate the values of your pixel brush in the bitmap - and it is very fast, too. And you don't even have to use single pixels - I'd try with circles or event gradient circles.

Here is a very basic example on how to accumulate the pixel values. But remember that you can use complete movieclips instead of the "brush" bitmap:

import flash.display.BitmapData;
import flash.geom.Matrix;
var canvas:BitmapData = new BitmapData(400, 400, false, 0);
attachBitmap(canvas, 0);
var brush:BitmapData = new BitmapData(1, 1, false, 0x030811);

function onEnterFrame(){
for (var i=0;i<1000;i++){
var m:Matrix = new Matrix(1, 0, 0, 1, Math.random()*400, Math.random()*400);
canvas.draw(brush, m, null, "add");
}
}

Posted by: Mario Klingemann on October 16, 2005 01:04 AM

Thanks Mario, you have given me some more ideas too, the idea of bitmap brushes has some brilliant possibilities.

This solution is cool, flexible, and pretty fast considering, I still long for more speed in flash, but don't we all, maybe 8.5 with it's supposed 10x speed increase will be of help. If it does what they say it'll be awesome, I can't what for next spring now, I always feel bad about being impatient tho, shouldn't go wishing time away.

Posted by: Jon B on October 16, 2005 03:13 AM

Hi Mario,

I recommend you give ServiceCapture a go (http://kevinlangdon.com/serviceCapture/), to see how much data you are really transferring. In my tests using an array of Numbers is much larger than a string - 10,000 Numbers in an array is about 90KB (10KB for an empty array of that size, 8bytes per Number), whereas a 10,000 character string (admittedly only Base256, or even Base64) is only 10KB.

Flash Numbers have an enormous range (64bits), whereas PHP can only have 32bit ints. So at best you are wasting half of the data, and if you are working with 24bit / 16bit values (as I expect most LZW indices would be) you will be wasting even more, plus the added overhead of the array...

I just tried, and you are correct that Flash does not transmit 0x00 in a string, but then PHP doesn't seem to accept a whole bunch outside of the Base64 range (I still haven't had a chance to look into this further). As long as the pixel data (RGB / YCC) is kept as arrays up until the serializing there will be no issue.

Posted by: Grant Cox on October 16, 2005 03:29 AM

In case anybody is interested this seems to be some color equations that simulate the blendmodes.

http://blog.andre-michelle.com/2005/blendmodes-math/

Posted by: Jon B on October 16, 2005 03:59 AM

Grant, thank you for this excellent tip! If that's true that would be pretty bad and I'd better use strings instead of numbers. But are you sure about those 64 bits? I will have to study the swf format specifications again, because I always thought it was 32 bits.

Posted by: Mario Klingemann on October 16, 2005 12:23 PM

I must admit I hadn't checked the Flash docs, just looked at the request size in ServiceCapture. I just checked the docs then, and it lists Number.MAX_VALUE as 1.79e+308 (http://livedocs.macromedia.com/flash/8/main/wwhelp/wwhimpl/common/html/wwhelp.htm?context=LiveDocs_Parts&file=00002570.html)

Now, this is actually 2^1024, which would be 128bytes! Which astounded me, until I realised that of course this is a floating point number, not an integer... I had been so caught up in the binary logic for integers I hadn't even considered floats. And after having a look, PHP supports 64bit floats - so I have changed my mind and now think an array of Numbers is the way to go, as long as you pack into all 64bits.

There will still be the overhead of 1byte / Number for the Array index (the penalty Actionscript has for non typed arrays), but this should be notably less than the Base64 overhead.

Posted by: Grant Cox on October 17, 2005 09:42 AM

I just tested it and it looks like even though the internal datatype for numbers may be Double you cannot calculate with more than 4 bytes in actionscript. I've tried this:

var test:Number = 1;
test <<= 8
test |= 2
test <<= 8
test |= 3
test <<= 8
test |= 4
test <<= 8
test |= 5
test <<= 8
test |= 6
test <<= 8
test |= 7
test <<= 8
test |= 8

// in theory test should be now 0x0102030405060708

trace( test & 0xff ); // traces 8
test >>= 8;
trace( test & 0xff ); // traces 7
test >>= 8;
trace( test & 0xff ); // traces 6
test >>= 8;
trace( test & 0xff ); // traces 5
test >>= 8;
trace( test & 0xff ); // traces 0, not 4

test >>= 8;
trace( test & 0xff ); // traces 0, not 3
test >>= 8;
trace( test & 0xff ); // traces 0, not 2
test >>= 8;
trace( test & 0xff ); // traces 0, not 1

This will only work with AS3 where we hopefully get typed numbers.

Posted by: Mario Klingemann on October 17, 2005 11:40 AM

AS 3.0 will solve all your problems :)
ByteArray + Socket (...but I see no way to monitor the writing of data to the socket, so that might be a problem :P)

Btw Mario: my suggestion to "encode" \x00 as \x01\x02 and \x01 as \x01\x01 would allow you to store any value 0-255 (but you'd double the size of your data if all bytes were \x00 or \x01, of course)

Posted by: ? on October 17, 2005 02:56 PM

Yes, now I see how this can work. I will try that. Thanks for the idea! Because 0x00 normally will occur more often than other numbers im images(that's only a guess though) it might improve the file size if I XOR the numbers first with something like 0xff or even better - check first which number has the least occurence in the data and XOR with that.

Posted by: Mario Klingemann on October 17, 2005 03:07 PM

AMFPHP allows to log all remoting messages and I have just looked at the datablocks that are sent by the BitmapExporter - whoa there's really lot's of air in form of 0x00 in there - definitely 64bit Numbers. It looks like I will have to change the format to strings then.

Posted by: Mario Klingemann on October 17, 2005 06:26 PM

Well, there is no need to wait for AS3 (although, it looks like you can download the Flex2 alpha now). You are right, you cannot use the bit shifting as they are integer operations, and while you cannot specify whether you want an int or a float (you just get Number), they are still separate internally...

But, I have been working on a class to pack and unpack bytes into a 64bit float. It is working quite well, just the occasional sequence gets garbled (most likely due to overflow / underflow / denormalizing - I haven't put much into checking those yet). Then to hopefully port to PHP :)

I will release the source soon, once it is a bit more solid (and I might put in pack/unpack for multibyte chars too).

Posted by: Grant Cox on October 17, 2005 07:47 PM

I'm currently converting my class to use Strings instead of numbers and I made another annoying discovery: Flash Remoting somewhere along the way converts all strings to UTF-8 before they are sent to the server. Unfortunately that's pretty bad for this task because all chars > 0x7f need two bytes then. I'm now trying to find a way to get rid of that encoding, but that doesn't look that easy. I'm already wondering if it's time to dump the remoting and use a simple http POST.

Posted by: Mario Klingemann on October 17, 2005 10:28 PM

Ok, I'm npt very smart with such things so this may be a really naive and dumb suggestion, but firstly is it possible to send strings to php using an XML Socket? If not then would the overhead of using multiple sendAndLoad calls to send packets be prohibitive?

Just my n00b-esque logic so don't hate me if I've missed the point.

Posted by: Jon B on October 17, 2005 11:47 PM

Using the XML-Socket is definitely an option and the overhead should actually be very small. It's only not my personal first choice as I want to go for a solution that is easy to deploy also on servers with limited capabilities. Installing an XML-socket server is not something for everybody.

Posted by: Mario Klingemann on October 17, 2005 11:55 PM

I thought PHP could be used to simulate a basic socket server tho, although my knowledge of PHP is lousy I'm sure I read about that.

Posted by: Jon B on October 18, 2005 01:46 AM

Here's a challenge: send a string containing all the chars from 0x00 to 0xff to the server and try to do that with as little overhead as possible. Well, the overhead is secondary - rather just try to send them so all of them arrive at least. I currently need 285 chars to have all 256 arrive at the server. Flash Remoting needs 386 chars for that.

Posted by: Mario Klingemann on October 18, 2005 02:28 AM

Here is a good approach!

http://www.kaourantin.net/2005/10/png-encoder-in-as3.html

Posted by: dmendels on October 18, 2005 09:28 AM

from http://www.flashguru.co.uk/actionscript-3-new-capabilities/

"BitmapData.getPixels()

No, longer do you have to loop through every pixel in a bitmap, one at a time with getPixel to send a bitmap to the server. This method returns a ByteArray containing the hexadecimal color value of each of the pixels in the specified rectangular region of a bitmap. Use this method in conjunction with the new ZLib compression method; ByteArray.compress() to compress and send a bitmap over the wire to a server so it can be converted into a file ready for downloading."

It kinda sucks when something you've been working so hard on suddenly becomes so simple - but at least your work has a several months of being the only realistically feasible way (until the 8.5 player comes out of testing)

Posted by: Jon B on October 18, 2005 01:05 PM

Well, that's life. It happened before with my pixeldata2swf script, my drop shadow and my justified text component. It's called progress.

I will still finish the class, because it will take a while until everybody adopts coding in AS3. And also some people might still prefer to use the Flash IDE to create their swf.

Posted by: Mario Klingemann on October 18, 2005 01:11 PM

Well, I've largely completed the 64bit float packing class and the LZW class (modified from Ash's http://www.razorberry.com/blog/archives/2004/08/22/lzw-compression-methods-in-as2/ ).

The packing works well (it's tighter than a Base64 string), and it is fast enough (the unpacking on PHP is a bit slow), and, as it preserves the bytes correctly it is worthwhile to use LZW, particularly on images with gradients/flat colours.

I have provided both Actionscript and PHP classes, for packing/unpacking both bytes and multibytes into a 64bit float, and for compressing/uncompressing to LZW.

You can download the Pack64Float classes from www.ensogroup.com.au/grant/pack64float.zip

and the LZW classes from www.ensogroup.com.au/grant/LZW.zip

Posted by: Grant Cox on October 19, 2005 08:35 AM

Oops, those didn't hyperlink without a protocol (I guess that's what the preview is for :P).

Pack64Float classes - http://www.ensogroup.com.au/grant/pack64float.zip

LZW classes - http://www.ensogroup.com.au/grant/LZW.zip

And I agree with Mario - while this stuff will be easier in FP8.5, I need to have this working in a project in a couple of weeks, and I won't tell our clients to download an alpha player. Also, while the pixel reading is a FP8 thing, these two classes could easily be used back in FP6 (though I'm not sure what for :P)

Posted by: Grant Cox on October 19, 2005 08:39 AM

My comment wasn't meant to undermine your work, it's clear you have both worked hard and I know it's for reason too since it'll be a while before FP8.5 hits. I just wish they had managed to to get the 8.5 features into the 8 player as intended, although it's good they aren't doing a rush job I guess.

Plus I think a LZW class is a great thing, people always need compression and since flashLite 2 will suppport AS2 and can see it having a healthy life in reducing bandwidth usage.

As an interesting thought, does it mean that Tiffs can be be imported into flash and converted into a bitmap?

Posted by: Jon B on October 19, 2005 11:32 AM

I am nearly done with my compression class as well, it now auto detects low (word packing and 256palette/encode/decode, and F8 makes this fast.
I look forward to looking at Grant's code here in a second (i should go to bed, its 5am now ;) (as well as mario's). I am doing simple base64 encode on lzh bytes for transmission, i am interested in both of your 'safe ascii' encoding methods, as they sound like better ratios.
Jon, on a note related to your processing in f8 experiments, I am nearly ready to release a almost entirely feature complete F8 implementation of the Processing library (except 3d) It makes it trivial to port processings example and the performance it not to bad (depending on the example.) I have ported many (around 50 examples). Here is one of my favorites of j.tarbell's
http://timmaffett.com/source/fp5/bubblechamber_processing.html
(click the mouse)
or here is one of Patrick Mineault'shttp://timmaffett.com/source/fp5/bit101particle_processing.html

I will post the code library and all examples in the next couple days. (I started on the image compression diversion to implement saveImage())
I thought you guys might be interested ?
(I think i first read of processing on this blog ;)
I created this library last month as a way to use the new flash 8 stuff. (it is not a port of the processing engine, but a cleanroom implementation from processing docs) I can't wait to get it running in as3/flash 8.5, which should be trivial and give us speeds that would make even the more math/cpu intensive processing apps match the java versions.

-tim

Posted by: tim maffett on October 19, 2005 02:39 PM

I have recently converted some of Jared's Processing work to AS3 and the speed is brilliant. Still not as fast as Processing, but it's good enough to make it not matter (my conversions in AS2 worked well but were a little on the slow and tedious side to watch).

I myself have been working on an AS3 library to assist in the porting of point based processing experiments - it doesn't use the method names of processing, I'm not trying to emulate everything, I just want a tool set that will help me develop stuff faster. I'm very keen to see your processing library tho, sounds extremely interesting.

Posted by: Jon B on October 20, 2005 02:46 PM

Oh yeah, I forgot to mention that stuff about optimal coding in AMFPHP ;) Indeed, everything outside of ASCII-128 is sent as Unicode through Remoting. If you attempt to send in numbers through an array, know that there is overhead involved: Remoting's only numeric type is an actionscript Number, specified as an 8-byte double. How the actionscript player truncates actionscript numbers is beyond my comprehension, all I know is I get it as an 8-byte little endian number on AMFPHP's side. To each 8-byte number you have to add 1 byte for the type, 0x02. Thus you get an overhead of 12.5%, and a lot of headaches about figuring out how to reach the full range of the number type in AMF. Sending a single string in AMF gives 6 bytes of data per 256 bytes, that is, an inefficiency of 33%. That might sound like a lot (12.5% vs. 33%) but I think the issue at hand is not bandwidth but speed. In that case in fact, strings will probably make up more than enough for the difference in speed because AMFPHP's array unserializer is really, really slow. In fact I wouldn't be surprised if that were the cause of your timeouts. This is what is done for each array:

function readArray() {
$ret = array(); // init the array object
$length = $this->inputStream->readLong(); // get the length of the array
for ($i = 0; $i inputStream->readByte(); // grab the type for each element
$ret[] = $this->readData($type); // grab each element
}
return $ret; // return the data

}

And:

function readDouble() {
$bytes = substr($this->raw_data, $this->current_byte, 8);
$this->current_byte += 8;
if ($this->isBigEndian) {
$bytes = strrev($bytes);
}
$zz = unpack("dflt", $bytes); // unpack the bytes
return $zz['flt']; // return the number from the associative array
}

That's a lot of processing. Compare this with:

function readString() {
return $this->inputStream->readUTF();
}

And:

function readUTF() {
$length = $this->readInt(); // get the length of the string (1st 2 bytes)
//BUg fix:: if string is empty skip ahead
if($length == 0)
{
return "";
}
else
{
$val = substr($this->raw_data, $this->current_byte, $length); // grab the string
$this->current_byte += $length; // move the seek head to the end of the string
return $this->charsetHandler->transliterate($val); // return the string
}
}

So we're talking about roughly 10 function calls versus imageWidth*imageSize/2*~10 function calls on the AMFPHP side. Couple that with the fact that 24-bit pixels split neatly into 4 base64 characters and you have a clear winner. Thus let it be spoken: base64 is the best solution. I thought about it for a really long time, trust me ;)

Posted by: Patrick Mineault on October 26, 2005 03:52 AM

Hi Patrick,

I take it that the issue with strings outside of ASCII-128 is just PHP's lack of multibyte character support, coupled with Remoting sending these single byte chars as Unicode?

I have also found the AMFPHP unserializing to be the bottleneck in other projects, so I specifically benchmarked it in my testing. However, I still found that providing reasonably small chunks of data are sent (approx 100x100 pixels) the time spent in PHP with a large array was offset by the time spend in Actionscript with a large string. Of course there are the pros and cons of moving processing to the client or server, but as I recall the difference was in the order of 1.0 seconds to unserialize, decompress and create an image from a string, vs 1.5 seconds with a float array. I will look into my actual benchmarks when I get home, but the difference was certainly comparable.

Plus, the problem is that while a 24bit pixels split very nicely into Base64, LZW compression gives a series of ints, with smaller images these remain in 16bits. And these do not fit into Base64 so well, certainly the overhead usually offsets any benefit the compression gained.

So certainly, Base64 is the much simpler and generally faster option but has more data overhead, whereas Float packing is more complex, generally slower but will use less bandwidth.

My test system currently supports both, so I will run some tests sometime soon to see how they both hold up under a heavy load.

Posted by: Grant Cox on October 26, 2005 08:37 AM

Thank you for this enlightening post Patrick. But are you sure that 6 Bytes overhead for each 256 bytes is 33%? I'd say that is 2%, isn't it?

I have already dropped remoting from my class and switched to loadVars. Currently I've got one bug somewhere with the unserialization of my data, but after that it should work fineI hope.

Posted by: Mario Klingemann on October 26, 2005 11:29 AM

Hi Mario,

Patrick would mean 6bytes per 256 bits (as in Base64 you are utilising 6/8 bits per byte, meaning a 33% increase).

It will be interesting to see how your LoadVars compares. What kind of overhead is involved in a request like that, are there any size limits, or is it possible to set the LoadVars HTTP headers to simulate a file upload of your data?

Posted by: Grant Cox on October 26, 2005 11:33 PM

The issue does not lie with PHP's lack of Unicode support. in fact, one can get back the original ASCII string using an appropriate charset handler, set in gateway.php. I believe that using LoadVars won't get you anywhere either, as everything is also probably sent using Unicode (You'd have to test it obviously), unless using System.useCodePage, but that comes with it's own series of problems.

Of course, if you send only chunks of 10000 pixels, that may come up to only about 0.5 seconds difference on the server side. However in my current tests using my custom algo, encoding takes approximately a second or two on the client side, while decoding takes about 3 seconds on the server side, including 2 full seconds for ImageMagick processing. This is on Windows with a non-accelerated version of PHP. The image itself is about 100,000 pixels. Meanwhile the compression gives about 130k to upload, that is, about 5-10 seconds to upload on a decent line.

As far as the LZW indices I think the naive palette approach is not the best in this case. Mario mentioned that he was creating a grayscale palette, plus unindexed colors. That's the approach I'm using. Since base64 leaves about 31 printable characters in ASCII < 128 set, I use these plus a meta character to indicate a paletted pixel, and another meta character for repetition, used either for paletted or non-paletted pictures. As for creating the palette itself, I'm using threshold on grayscale values, so the palette consists or the 30 or so most used gray values. This is under the presumption that grays are the most used colors in the image. If that is not the case, we can apply a left convolution filter (ie, the modified pixel value is the value of the current pixel - the pixel on the left mod 256). This is equivalent to the 'Left' scanline filter in the PNG spec. After that, for most gif-like images, the colors will be mostly isolated pixel followed by large white areas, and gray where gradients used to lie. Of course this is a good algorithm only for gif-like images.

As for JPEG-like images, I would recommend waiting for 8.5 and then Flash should be fast enough for realistically doing discrete cosine transforms in a relatively short amount of time.

Posted by: Patrick Mineault on October 27, 2005 11:23 AM

FYI - I am using System.useCodepage in order to switch off Unicode encoding whilst sending the loadVars. And there are some really strange issues, but I think I have overcome them - I don't know if that is a bug in Flash or it's system dependent, but from the ASCII codes between 1 and 255, in the range between 0x80 and 0xa0 there are several characters simply replaced by 0x3f - which means I have to encode 29 codes with two byte versions - which is what I was referring to earlier.

Posted by: Mario Klingemann on October 27, 2005 12:21 PM

@Jed - it's difficult to say what's the problem without having a look at your code. In which format are you sending the pixels? Are you using toString(16)? A common error is to always expect a 6 letter string like AABBCC, which toString(16) does not guarantee.

Posted by: Mario Klingemann on November 5, 2005 11:00 AM

I have been working on a Flash application were a user modifies a clip using buttons that do a setTransform on the clip. We would love to then be able to save this clip to png, jpg. This stuff is way over my head. I see what you are doing but I do not know how to apply it to what I am doing. I could not post the URL due to the restriction on P0K3R.

Posted by: Mike (X-Files) on November 8, 2005 03:46 PM

Mario--
Yes. I've been using toString(16). I suspect my problems also have something to do with alpha channels?

Here is my code in flash that sends to the php script:

function outputClip(nr){
var pixels:Array = new Array()
//Create a new BitmapData
var snap = new BitmapData(this["snapshot"+nr].width, this["snapshot"+nr].height,false);
//Copy image
snap.draw(this["snapshot"+nr]);
snap.rectangle.topLeft, myFilters[nr])
var w:Number = snap.width, tmp
var h:Number = snap.height
//Build pixels array
for(var a=0; a<=w; a++){
for(var b=0; b<=h; b++){
tmp = snap.getPixel32(a, b).toString(16);
pixels.push(tmp.substr(1));
}
}
System.useCodepage = true;
output.img = pixels.toString(16);
output.height = h;
output.width = w;
output.imageName = randFileName;
// send the output object to the php script for rendering.
output.send("show.php", "output", "POST");
}

Posted by: Jed on November 28, 2005 05:44 PM

Jed - I think there is on toString(16) to much in your code. This line:
output.img = pixels.toString(16);
should better be called
output.img = pixels.toString();

Then I don't understand why you are saying:
pixels.push(tmp.substr(1));
This will cut off the first letter of the hex number - I don't think you want to do that. Just say

pixels.push(snap.getPixel32(a, b).toString(16));


If you are ignoring the alpha channel on the serverside, you could save some bandwidth when you say:

pixels.push((snap.getPixel32(a, b) & 0xffffff ).toString(16));

Posted by: Mario Klingemann on November 28, 2005 05:57 PM

Yes! Finally. That fixed my troubles. Not sure why the first character was being trimmed. Maybe left over from some procedure i was concocting to remove the alpha info? Regardless. No more mysterious blue where there shouldve been black or white. Thank you very much in helping me understand this.

Posted by: Jed on November 28, 2005 08:46 PM

Hello Mario,
Vary cool work :)
I want to ask you Is there any limit for the width and height of the saving bitmap image?

Posted by: Stiakooo on January 25, 2006 09:21 AM

I tried to save image 1650x2350. But when I press "Save" button I can only see the progressbar showing "Initialising(0%)" for a second and then it dissapears and "Save" button is visible again.
I think that "idle" case of "status" function is triggered almost immediately after pressing "Save" button.
My objective is to save big images suitable for printing.
bwt I have no problems saving small images. This problem only occurs when I set large width and height values for the BitmapData object:
var snapshot:BitmapData = new BitmapData(1650,2350,false);

Posted by: Stiakooo on January 25, 2006 03:14 PM

Good work Mario, by the way great speech at Spark Europe i learned a lot from it.

This works great after a few hours of reading and experimenting. I got the hang of it.
I think i found a bug, at least it doesn't work on my server.
I have a movieclip with the following properties width=1010 height=670. If i try to export this clip to a jpg and save the new image to my desktop. Then image will not be saved, but when i scale the clip to 60% it works.
Maybe i`m doing something wrong. Could you help me.

Thnx Ronald

Posted by: Ronald on February 1, 2006 05:03 PM

I have made a simple swf with only the FileReference.download. And i get a onIOError in the second setup. So i think that the problem lies there. How, why i don't know, because there is no authentication needed for that server.

this is the link:
http://www.fiertest.nl/test/testdownloadsize.html

the image is 63 kb

Posted by: Ronald on February 2, 2006 10:23 AM

pls give the idea about painting and saving in flash
(in details... script)

Posted by: syam on February 2, 2006 11:53 AM

strange i have tested just 1 minute ago and it still doesn't work.Yes the html page reside in the same domain as the swf and getImage.php file. Only in a different folder of course :)

Posted by: Ronald on February 2, 2006 12:24 PM

When i tried it at home, it also works fine. Think that problem lies with my computer at work or oure network. Thank you for effort with helping me with this problem.

Posted by: Ronald on February 2, 2006 08:20 PM

Hi,
Im working on a piece that exporting would be great. I have the script working and sending the way i want it. BUT.....

I try to 'snap' a photo of a webcam and all that i get is a off white file. ANY THOUGHTS..

Thanks..

T

Posted by: Todd Vanderlin on March 22, 2006 07:13 PM

oh damn - I just realized that I have deleted a whole bunch of valid comments on my blog whilst removing spam. Sorry if your's was among them.

Posted by: Mario Klingemann on May 18, 2007 03:33 PM

hi folks,
I am, trying to attain the same image compression and creating an image using a .net script.Could any one suggest where I can have the code for the same.. I tried interpretting the code that i downloaded for this page.. but had some difficulties in making it possible.. ols help

Posted by: dan on May 18, 2007 11:24 PM

Do you have the original source code/fla and php with the drawing app? I wanted to duplicate the exact way you have it here because I cannot get the above sample to work? I am trying to understand how to manipulate location of image one processed. I also want to see the return to my desktop. Please and help would be great!

Posted by: jay on May 21, 2007 09:18 PM

Since I have further plans with the drawing algorithms the sourcecode for the drawing app is currently not available.

Posted by: Mario Klingemann on May 21, 2007 10:15 PM

Is there a way to insert a link in the image so that when the image is showed we can click on it?

best regards and thanks for this great script.
Jonas

Posted by: jonas on May 22, 2007 03:17 PM

The BitmapExporter class has two options what happens after the bitmap has been created on the server: the default behavior is what happens in the demo above - a system window will pop up and ask you where to download the image to. The alternative behavior is set by setting the dontRetrieve argument to false. Then there will be no automatic popup and you will have to listen to the "onSaved" event in your program. This will be fired as soon as the image is available on the server and the eventObject will contain the url of the image. Then you can do whatever you want with this url - e.g. load it into your flash file, display a text link to it or trigger another script on your server which will mail that bitmap to someone.

Posted by: Mario Klingemann on May 22, 2007 03:43 PM

Hi, my comment was deleted,but I'm not a spammer :).
I don't know if there was an answer to my question, so I'll try again. It seems it's a great stuff, very useful tool, but I'm always getting an error occurence in ex-ample on this page: [onAddPixelBlock] Unkown error type: [8] Undefined offset number. The only compression mode, which is going well, is fastscan, default is going well too, but not on every machine. Other methods throw an error (similar to error on this page). Is there someone who has the same problem as me, or is it me the problem ;) ?
Thank you for any reply.

Posted by: kloff on May 23, 2007 11:21 AM

I'm currently investigating an issue that seems to be related to the default codepage of the user's OS. As I probably mentioned somewhere above one of the issues I had when posting strings to a server side script from flash is that in its default setting Flash automatically converts all chars to Unicode. This effectively doubles the amount of sent data but is actually a total waste of resources. So what I do is to temporarily set System.useCodepage = true. This forces Flash to only send Ascii chars. But it seems that there are some internal issues with different local OS. When using fastscan data is encoded base16 so there are no unicode chars anyway.

Posted by: Mario Klingemann on May 23, 2007 12:14 PM

Oh! As I see here, now I know how to send the files to a mail. Just with the onSaved event in the AS. Ok. I understand that, but I'm a beginner, I'm starting and I don't know yet how to call a PHP or PHP function from AS. I'm sorry I'm so new. I have a mail PHP to support attachments that works fine, but how do I call it from that event in AS?

If you don't have time to tell me that, can you or somebody reading this tell me where can I get started with calling PHP from AS? Thank you very much, it's so important to me

Posted by: Salvador on May 27, 2007 06:13 PM

Since I have published a revised version of BitmapExporter here: http://www.quasimondo.com/archives/000645.php I am closing the comments on this page. Please use the new page for questions or feedback.

Posted by: Mario Klingemann on May 28, 2007 03:06 PM
Post a comment
Name:


Email Address:


URL:


Comments:


Remember info?



Thank you!

Most Visited Entries
Sketches, Works & Source Code
Lectures
Contact
Backlog
In Love with
Powered by
Movable Type 2.661

© Copyright Mario Klingemann

Syndicate this site:
RSS 1.0 - RSS 2.0

Quasimondo @ flickr
Quasimondo @ LinkedIn
Quasimondo @ Twitter
Quasimondo @ Facebook
Quasimondo @ MySpace
Quasimondo is a Bright
Citizen of the TRansnational Republic
My other blog in german
Impressum


My family name is written Klingemann,
not Klingelmann, Klingeman, Klingaman, Kingemann,
Kindermann, Killingaman, Klingman, Klingmann, Klingonman
Klingemman, Cleangerman, Klingerman or Kleangerman

profile for Quasimondo at Stack Overflow, Q&A for professional and enthusiast programmers