Updated: 17/11/2011
First up here is the demo:
click here
Controls:
Mouse to move camera
Mouse wheel to zoom
Space bar to toggle mouse movement
check box to turn the post processing on or off
number 0-9 on keyboard to try some preloaded filters (emboss, blur, edge detection etc…)
slider to control the sample offset
you can enter your own values into the matrix if you like, but be nice as there isn’t any error checking
let me know if it works for you!
will need flash player 11 to view
get flash player
video of it running if the demo fails:
So what is a convolution filter and how does it work?
Basically a convolution filter (with regards to images) is where for each pixel to be sampled, a set of surrounding pixels is also sampled. The influence of the surrounding pixels is controlled by a matrix.
i.e.
[0,0,0,
0,1,0,
0,0,0]
The current pixel is represented by the middle value.
The 1st value in the matrix is a sample taken from the pixel North-West of the current pixel
2nd is North
3rd is North-East
4th is East
…
i.e.
[NW, N, NE,
W , x, E
SW , S, SE]
x = the current pixel
so the positions in the matrix determine where the samples come from and the values in the matrix determine their influence.
in the matrix
[0,0,0,
0,1,0,
0,0,0]
all the surrounding pixels will have no (0) influence on the output
with the following matrix
[1,1,1,
1,1,1,
1,1,1]
they all have equal weighting effectively blurring the image
various effects can be achieved by using difference weight combinations such as edge detection and sharpening.
How to do it with the AGAL helper:
simply call:
AGAL.convolve(output, sample, texture, uv, offset, matrix)+
where output is the destination register
where sample is a temporary register for storing the modified uvs
where texture is the texture to sample from
where uv is the current uv coord of the sample (usually passed from the vertex shader)
where offset* is the value to offset the samples by (you will need to calculate this with something like 1/viewportWidth)
where matrix** is the convolution matrix
*at the moment the same offset is used for x and y when in reality there should be two (one for each axis) will add this in at a later date as it hasn’t caused an issue so far.
**
sample code to upload a matrix of 9 values from a vector of numbers called “_filter” into fragment constant 0 – 2:
context3D.setProgramConstantsFromVector(Context3DProgramType.FRAGMENT, 0, Vector.<Number>([_filter[0],_filter[1],_filter[2], 1])); context3D.setProgramConstantsFromVector(Context3DProgramType.FRAGMENT, 1, Vector.<Number>([_filter[3],_filter[4],_filter[5], 1])); context3D.setProgramConstantsFromVector(Context3DProgramType.FRAGMENT, 2, Vector.<Number>([_filter[6],_filter[7],_filter[8], 1])); |
for this to work properly you should normalise you matrix so that the sum of the values in your matrix == 1.
here is some code to do that:
var sum:Number = 0; for(var i:int = 0 ; i < _filter.length; i++) { sum += _filter[i]; } for(i = 0 ; i < _filter.length; i++) { if(_filter[i] != 0 && sum != 0)_filter[i] /= sum; } |
this would turn the matrix
[1,1,1,
1,1,1,
1,1,1]
into
[1/9,1/9,1/9,
1/9,1/9,1/9,
1/9,1/9,1/9]
good luck 🙂
6 replies on “Molehill Convolution”
Hey. trying to add the blur convolution filter to Starling.
Filter:
var _filter:Vector. = new [1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0];
var sum:Number = 0;
for(var i:int = 0 ; i < _filter.length; i++){
sum += _filter[i];
}
for(i = 0 ; i < _filter.length; i++){
if(_filter[i] != 0 && sum != 0)_filter[i] /= sum;
}
Constants:
context.setProgramConstantsFromVector(Context3DProgramType.FRAGMENT, 1, Vector.([_filter[0],_filter[1],_filter[2], 1]));
context.setProgramConstantsFromVector(Context3DProgramType.FRAGMENT, 2, Vector.([_filter[3],_filter[4],_filter[5], 1]));
context.setProgramConstantsFromVector(Context3DProgramType.FRAGMENT, 3, Vector.([_filter[6],_filter[7],_filter[8], 1]));
context.setProgramConstantsFromVector(Context3DProgramType.FRAGMENT, 4, Vector.([0.1,0.1,0.1, 1])); //offset constant
Vertex Program:
var vertexProgramCode:String =
“m44 op, va0, vc0 \n” + // 4×4 matrix transform to output clipspace
“mov v0, va1 \n” + // pass color to fragment program
“mov v1, va2 \n”; // pass texture coordinates to fragment program
Fragment program:
var fragmentProgramCode:String = AGAL.convolve(“ft3”, “ft4”, “fs1”, “v1”, “fc4”, “fc1”);
fragmentProgramCode += “mov oc, ft3”;
Final fragment program output:
mov ft4 v1
add ft4.y ft4.y fc4
sub ft4.x ft4.x fc4
tex ft4 ft4 fs1
mul ft4 ft4 fc1.x
add ft3 ft3 ft4
mov ft4 v1
add ft4.y ft4.y fc4
tex ft4 ft4 fs1
mul ft4 ft4 fc1.y
add ft3 ft3 ft4
mov ft4 v1
add ft4.y ft4.y fc4
add ft4.x ft4.x fc4
tex ft4 ft4 fs1
mul ft4 ft4 fc1.z
add ft3 ft3 ft4
mov ft4 v1
sub ft4.x ft4.x fc4
tex ft4 ft4 fs1
mul ft4 ft4 fc11.x
add ft3 ft3 ft4
mov ft4 v1
tex ft4 ft4 fs1
mul ft4 ft4 fc11.y
add ft3 ft3 ft4
mov ft4 v1
add ft4.x ft4.x fc4
tex ft4 ft4 fs1
mul ft4 ft4 fc11.z
add ft3 ft3 ft4
mov ft4 v1
sub ft4.y ft4.y fc4
sub ft4.x ft4.x fc4
tex ft4 ft4 fs1
mul ft4 ft4 fc12.x
add ft3 ft3 ft4
mov ft4 v1
sub ft4.y ft4.y fc4
tex ft4 ft4 fs1
mul ft4 ft4 fc12.y
add ft3 ft3 ft4
mov ft4 v1
sub ft4.y ft4.y fc4
add ft4.x ft4.x fc4
tex ft4 ft4 fs1
mul ft4 ft4 fc12.z
add ft3 ft3 ft4
mov oc, ft3
So I register fc1,fc2,fc3 as the convolution matrix but I’m only sending fc1 as the matrix to the agal helper. How can it read the other values if fc1 is the only one used?
Compiler sends this error:
AGAL validation failed: Temporary register read without being written to for source operand 1 at token 6 of fragment program.
ok, the function looks at the first index and then works out the next two so they must be sequential
i.e. if you pass in fc1, it will know to look for fc2 and fc3
with regards to the offset it only wants one value
so change
context.setProgramConstantsFromVector(Context3DProgramType.FRAGMENT, 4, Vector.([0.1,0.1,0.1, 1])); //offset constant
to
context.setProgramConstantsFromVector(Context3DProgramType.FRAGMENT, 4, Vector.([0.1,0,0, 0])); //offset constant
and pass in
“fc4.x″
ah and I can see the error too, you need to set the value of the output to zero 1st
(I will fix this in next release)
so before:
var fragmentProgramCode:String = AGAL.convolve(“ft3″, “ft4″, “fs1″, “v1″, “fc4″, “fc1″);
do
var fragmentProgramCode:String = AGAL.mov(“ft3″, “fc4.y”);//zero ft3
fragmentProgramCode += AGAL.convolve(“ft3″, “ft4″, “fs1″, “v1″, “fc4.x″, “fc1″);
let me know how you get on
Thanks! Works like a charm! I will try to release the convolution filter as a Starling extension and give credit where it’s due 🙂
go for it, remember to try out other filters too! can get some nice effects like in the video ^^
Found a problem. Where it is sopposed to be fc2 it is fc11. And fc3 is fc12. So maybe some kind of parsing error?
Anyway, here is a stress test with 100 .png files:
http://www.emilj.se/starling/
And the final code:
fragmentProgramCode =
“mov ft3, fc4.y \n” +
“mov ft4 v1 \n” +
“add ft4.y ft4.y fc4.x \n” +
“sub ft4.x ft4.x fc4.x \n” +
“tex ft4 ft4 fs1 \n” +
“mul ft4 ft4 fc1.x \n” +
“add ft3 ft3 ft4 \n” +
“mov ft4 v1 \n” +
“add ft4.y ft4.y fc4.x \n” +
“tex ft4 ft4 fs1 \n” +
“mul ft4 ft4 fc1.y \n” +
“add ft3 ft3 ft4 \n” +
“mov ft4 v1 \n” +
“add ft4.y ft4.y fc4.x \n” +
“add ft4.x ft4.x fc4.x \n” +
“tex ft4 ft4 fs1 \n” +
“mul ft4 ft4 fc1.z \n” +
“add ft3 ft3 ft4 \n” +
“mov ft4 v1 \n” +
“sub ft4.x ft4.x fc4.x \n” +
“tex ft4 ft4 fs1 \n” +
“mul ft4 ft4 fc2.x \n” +
“add ft3 ft3 ft4 \n” +
“mov ft4 v1 \n” +
“tex ft4 ft4 fs1 \n” +
“mul ft4 ft4 fc2.y \n” +
“add ft3 ft3 ft4 \n” +
“mov ft4 v1 \n” +
“add ft4.x ft4.x fc4.x \n” +
“tex ft4 ft4 fs1 \n” +
“mul ft4 ft4 fc2.z \n” +
“add ft3 ft3 ft4 \n” +
“mov ft4 v1 \n” +
“sub ft4.y ft4.y fc4.x \n” +
“sub ft4.x ft4.x fc4.x \n” +
“tex ft4 ft4 fs1 \n” +
“mul ft4 ft4 fc3.x \n” +
“add ft3 ft3 ft4 \n” +
“mov ft4 v1 \n” +
“sub ft4.y ft4.y fc4.x \n” +
“tex ft4 ft4 fs1 \n” +
“mul ft4 ft4 fc3.y \n” +
“add ft3 ft3 ft4 \n” +
“mov ft4 v1 \n” +
“sub ft4.y ft4.y fc4.x \n” +
“add ft4.x ft4.x fc4.x \n” +
“tex ft4 ft4 fs1 \n” +
“mul ft4 ft4 fc3.z \n” +
“add ft3 ft3 ft4 \n” +
“mul ft3, ft3, fc0 \n” +
“mov oc, ft3”;
Thanks for all your help! 🙂