In 2007, Adobe launched Pixel Bender for the Flash Runtime. This allowed Flash Developers parallel processing for the first time. This presentation was the first introduction to the new capabilities in the Flash Runtime.
Crypto Cloud Review - How To Earn Up To $500 Per DAY Of Bitcoin 100% On AutoP...
Image and Video Processing Using Adobe Image Foundation's Toolkit For Flash - MAX 2007
1. 2007 Adobe Systems Incorporated. All Rights Reserved.
1
Kevin Goldsmith
Engineering Manager - AIF
Adobe Systems Inc.
Bob Archer
Senior Computer Scientist - AIF
Adobe Systems Inc.
Saikat Kanjilal
Computer Scientist - Emerging Solutions
Adobe Systems Inc.
MAX 2007
CONNECT. DISCOVER. INSPIRE.
17. 2007 Adobe Systems Incorporated. All Rights Reserved.
17
This Talk
Adobe Image Foundation Toolkit
Technology Preview
18. 2007 Adobe Systems Incorporated. All Rights Reserved.
18
This Talk
AIF Toolkit
19. 2007 Adobe Systems Incorporated. All Rights Reserved.
19
This Talk
AIF Toolkit
Available on Adobe Labs RIGHT NOW
http://labs.adobe.com/wiki/index.php/AIF_Toolkit
20. 2007 Adobe Systems Incorporated. All Rights Reserved.
20
This Talk
(Codename) Hydra Language
21. 2007 Adobe Systems Incorporated. All Rights Reserved.
21
This Talk
(Codename) Hydra Language
(Adobe Legal makes us say Codename)
22. 2007 Adobe Systems Incorporated. All Rights Reserved.
22
This Talk
(Codename) Hydra Language
(Adobe Legal makes us say Codename)
We’ll have a name just as cool soon
23. 2007 Adobe Systems Incorporated. All Rights Reserved.
23
This Talk
AIF
AIF
24. 2007 Adobe Systems Incorporated. All Rights Reserved.
24
This Talk
Ability for Flash Authors to create Filters a highly requested feature
25. 2007 Adobe Systems Incorporated. All Rights Reserved.
25
This Talk
Ability for Flash Authors to create Filters a highly requested feature
Iterating on pixels in the bitmap object is slow and difficult
26. 2007 Adobe Systems Incorporated. All Rights Reserved.
26
This Talk
Ability for Flash Authors to create Filters a highly requested feature
Iterating on pixels in the bitmap object is slow and difficult
Hydra for Flash can make doing animated filters easier with high performance
27. 2007 Adobe Systems Incorporated. All Rights Reserved.
27
This Talk
Ability for Flash Authors to create Filters a highly requested feature
Iterating on pixels in the bitmap object is slow and difficult
Hydra for Flash can make doing animated filters easier with high performance
For certain classes of filters
28. 2007 Adobe Systems Incorporated. All Rights Reserved.
28
Flash Bitmap API vs Hydra
29. 2007 Adobe Systems Incorporated. All Rights Reserved.
29
Flash Bitmap API vs Hydra
// loop through the pixels:
for (var x:Number = xMin; x < xMax; x++) {
for (var y:Number = yMin; y < yMax; y++) {
// get the pixel's RGB value:
var rgba:Number = bmp.getPixel32(x,y);
// isolate channels:
var red:Number = (rgba & 0xFF000000) >>
24;
var green:Number = (rgba & 0x00FF0000)
>> 16;
var blue:Number = (rgba & 0x0000FF00) >>
8;
var alpha:Number = (rgba & 0x000000FF);
red = red * .5;
green = green * .8;
var output:Number = (red << 24) | (blue <<
16) | (green << 8) | alpha;
outbmp.setPixel32(x,y,output);
}
}
kernel NewFilter
{
void evaluatePixel(in image4 src, out pixel4
dst)
{
pixel4 temp =
sampleNearest(src,outCoord());
dst = pixel4( temp.r * .5, temp.b, temp.g *
.8, temp.a );
}
}
30. 2007 Adobe Systems Incorporated. All Rights Reserved.
30
This Talk
An Introduction to the Hydra Language and the AIF Toolkit
A Sneak Peak of the AIF Server
31. 2007 Adobe Systems Incorporated. All Rights Reserved.
31
ƒ Introduction
ƒ Demonstration – some sample filters
ƒ Code walkthrough – identity filter
ƒ Hydra programming model
ƒ Code walkthrough – polka dot filter
ƒ Hydra in detail
ƒ Hydra for Flash
ƒ Efficiency tips
ƒ Demonstration – AIF toolkit
ƒ Bibliography
ƒ Summary
ƒ AIF server
Demonstration – some sample filters
32. 2007 Adobe Systems Incorporated. All Rights Reserved.
32
ƒ Introduction
ƒ Demonstration – some sample filters
ƒ Code walkthrough – identity filter
ƒ Hydra programming model
ƒ Code walkthrough – polka dot filter
ƒ Hydra in detail
ƒ Hydra for Flash
ƒ Efficiency tips
ƒ Demonstration – AIF toolkit
ƒ Bibliography
ƒ Summary
ƒ AIF server
Code walkthrough – identity filter
33. 2007 Adobe Systems Incorporated. All Rights Reserved.
33
kernel Identity
{
void evaluatePixel( in image4 src, out pixel4 dest )
{
dest = sampleNearest( src, outCoord() );
}
}
Code walkthrough – identity filter
34. 2007 Adobe Systems Incorporated. All Rights Reserved.
34
kernel Identity
{
void evaluatePixel( in image4 src, out pixel4 dest )
{
dest = sampleNearest( src, outCoord() );
}
}
Code walkthrough – identity filter
35. 2007 Adobe Systems Incorporated. All Rights Reserved.
35
kernel Identity
{
void evaluatePixel( in image4 src, out pixel4 dest )
{
dest = sampleNearest( src, outCoord() );
}
}
Code walkthrough – identity filter
36. 2007 Adobe Systems Incorporated. All Rights Reserved.
36
kernel Identity
{
void evaluatePixel( in image4 src, out pixel4 dest )
{
dest = sampleNearest( src, outCoord() );
}
}
Code walkthrough – identity filter
37. 2007 Adobe Systems Incorporated. All Rights Reserved.
37
kernel Identity
{
void evaluatePixel( in image4 src, out pixel4 dest )
{
dest = sampleNearest( src, outCoord() );
}
}
Code walkthrough – identity filter
38. 2007 Adobe Systems Incorporated. All Rights Reserved.
38
kernel Identity
{
void evaluatePixel( in image4 src, out pixel4 dest )
{
dest = sampleNearest( src, outCoord() );
}
}
Code walkthrough – identity filter
39. 2007 Adobe Systems Incorporated. All Rights Reserved.
39
Digression – what is an image?
40. 2007 Adobe Systems Incorporated. All Rights Reserved.
40
Digression – what is an image?
41. 2007 Adobe Systems Incorporated. All Rights Reserved.
41
Digression – what is an image?
42. 2007 Adobe Systems Incorporated. All Rights Reserved.
42
Digression – what is an image?
43. 2007 Adobe Systems Incorporated. All Rights Reserved.
43
Digression – what is an image?
44. 2007 Adobe Systems Incorporated. All Rights Reserved.
44
Digression – what is an image?
45. 2007 Adobe Systems Incorporated. All Rights Reserved.
45
Digression – what is an image?
46. 2007 Adobe Systems Incorporated. All Rights Reserved.
46
Digression – what is an image?
47. 2007 Adobe Systems Incorporated. All Rights Reserved.
47
Digression – what is an image?
48. 2007 Adobe Systems Incorporated. All Rights Reserved.
48
kernel Identity
{
void evaluatePixel( in image4 src, out pixel4 dest )
{
dest = sampleNearest( src, outCoord() );
}
}
Code walkthrough – identity filter
49. 2007 Adobe Systems Incorporated. All Rights Reserved.
49
kernel Identity
{
void evaluatePixel( in image4 src, out pixel4 dest )
{
dest = sampleNearest( src, outCoord() );
}
}
Code walkthrough – identity filter
50. 2007 Adobe Systems Incorporated. All Rights Reserved.
50
kernel Identity
{
void evaluatePixel( in image4 src, out pixel4 dest )
{
dest = sampleNearest( src, outCoord() );
}
}
Code walkthrough – identity filter
51. 2007 Adobe Systems Incorporated. All Rights Reserved.
51
kernel Identity
{
void evaluatePixel( in image4 src, out pixel4 dest )
{
dest = sampleNearest( src, outCoord() );
}
}
Code walkthrough – identity filter
52. 2007 Adobe Systems Incorporated. All Rights Reserved.
52
ƒ Introduction
ƒ Demonstration – some sample filters
ƒ Code walkthrough – identity filter
ƒ Hydra programming model
ƒ Code walkthrough – polka dot filter
ƒ Hydra in detail
ƒ Hydra for Flash
ƒ Efficiency tips
ƒ Demonstration – AIF toolkit
ƒ Bibliography
ƒ Summary
ƒ AIF server
Hydra programming model
53. 2007 Adobe Systems Incorporated. All Rights Reserved.
53
Write a function that produces a single
output pixel
Hydra programming model
54. 2007 Adobe Systems Incorporated. All Rights Reserved.
54
Hydra programming model
55. 2007 Adobe Systems Incorporated. All Rights Reserved.
55
Hydra programming model
56. 2007 Adobe Systems Incorporated. All Rights Reserved.
56
Hydra programming model
57. 2007 Adobe Systems Incorporated. All Rights Reserved.
57
Hydra programming model
58. 2007 Adobe Systems Incorporated. All Rights Reserved.
58
ƒ Introduction
ƒ Demonstration – some sample filters
ƒ Code walkthrough – identity filter
ƒ Hydra programming model
ƒ Code walkthrough – polka dot filter
ƒ Hydra in detail
ƒ Hydra for Flash
ƒ Efficiency tips
ƒ Demonstration – AIF toolkit
ƒ Bibliography
ƒ Summary
ƒ AIF server
Code walkthrough – polka dot filter
59. 2007 Adobe Systems Incorporated. All Rights Reserved.
59
Code walkthrough – polka dot filter
60. 2007 Adobe Systems Incorporated. All Rights Reserved.
60
Code walkthrough – polka dot filter
61. 2007 Adobe Systems Incorporated. All Rights Reserved.
61
Code walkthrough – polka dot filter
62. 2007 Adobe Systems Incorporated. All Rights Reserved.
62
Code walkthrough – polka dot filter
63. 2007 Adobe Systems Incorporated. All Rights Reserved.
63
Code walkthrough – polka dot filter
64. 2007 Adobe Systems Incorporated. All Rights Reserved.
64
Code walkthrough – polka dot filter
65. 2007 Adobe Systems Incorporated. All Rights Reserved.
65
Code walkthrough – polka dot filter
66. 2007 Adobe Systems Incorporated. All Rights Reserved.
66
Code walkthrough – polka dot filter
67. 2007 Adobe Systems Incorporated. All Rights Reserved.
67
Code walkthrough – polka dot filter
68. 2007 Adobe Systems Incorporated. All Rights Reserved.
68
Code walkthrough – polka dot filter
69. 2007 Adobe Systems Incorporated. All Rights Reserved.
69
Code walkthrough – polka dot filter
70. 2007 Adobe Systems Incorporated. All Rights Reserved.
70
kernel PinkPolkaDots
{
parameter float radius;
parameter float spacing;
const pixel4 pink = pixel4( 1.0, 0.75 ,0.8, 1.0 );
void evaluatePixel( in image4 src, out pixel4 dest )
{
float2 center = floor(outCoord() / spacing)*spacing + (spacing / 2.0);
float dist = distance( center, outCoord() );
pixel4 originalColor = sampleNearest( src, outCoord() );
dest = dist < radius ? pink : originalColor;
}
}
Code walkthrough – polka dot filter
71. 2007 Adobe Systems Incorporated. All Rights Reserved.
71
kernel PinkPolkaDots
{
parameter float radius;
parameter float spacing;
const pixel4 pink = pixel4( 1.0, 0.75 ,0.8, 1.0 );
void evaluatePixel( in image4 src, out pixel4 dest )
{
float2 center = floor(outCoord() / spacing)*spacing + (spacing / 2.0);
float dist = distance( center, outCoord() );
pixel4 originalColor = sampleNearest( src, outCoord() );
dest = dist < radius ? pink : originalColor;
}
}
Code walkthrough – polka dot filter
72. 2007 Adobe Systems Incorporated. All Rights Reserved.
72
kernel PinkPolkaDots
{
parameter float radius;
parameter float spacing;
const pixel4 pink = pixel4( 1.0, 0.75 ,0.8, 1.0 );
void evaluatePixel( in image4 src, out pixel4 dest )
{
float2 center = floor(outCoord() / spacing)*spacing + (spacing / 2.0);
float dist = distance( center, outCoord() );
pixel4 originalColor = sampleNearest( src, outCoord() );
dest = dist < radius ? pink : originalColor;
}
}
Code walkthrough – polka dot filter
73. 2007 Adobe Systems Incorporated. All Rights Reserved.
73
kernel PinkPolkaDots
{
parameter float radius;
parameter float spacing;
const pixel4 pink = pixel4( 1.0, 0.75 ,0.8, 1.0 );
void evaluatePixel( in image4 src, out pixel4 dest )
{
float2 center = floor(outCoord() / spacing)*spacing + (spacing / 2.0);
float dist = distance( center, outCoord() );
pixel4 originalColor = sampleNearest( src, outCoord() );
dest = dist < radius ? pink : originalColor;
}
}
Code walkthrough – polka dot filter
74. 2007 Adobe Systems Incorporated. All Rights Reserved.
74
ƒ Introduction
ƒ Demonstration – some sample filters
ƒ Code walkthrough – identity filter
ƒ Hydra programming model
ƒ Code walkthrough – polka dot filter
ƒ Hydra in detail
ƒ Hydra for Flash
ƒ Efficiency tips
ƒ Demonstration – AIF toolkit
ƒ Bibliography
ƒ Summary
ƒ AIF server
Hydra in detail
75. 2007 Adobe Systems Incorporated. All Rights Reserved.
75
kernel PinkPolkaDots
{
parameter float radius;
parameter float spacing;
const pixel4 pink = pixel4( 1.0, 0.75 ,0.8, 1.0 );
void evaluatePixel( in image4 src, out pixel4 dest )
{
float2 center = floor(outCoord() / spacing)*spacing + (spacing / 2.0);
float dist = distance( center, outCoord() );
pixel4 originalColor = sampleNearest( src, outCoord() );
dest = dist < radius ? pink : originalColor;
}
}
Hydra in detail – Syntax and Semantics
76. 2007 Adobe Systems Incorporated. All Rights Reserved.
76
kernel PinkPolkaDots
{
parameter float radius;
parameter float spacing;
const pixel4 pink = pixel4( 1.0, 0.75 ,0.8, 1.0 );
void evaluatePixel( in image4 src, out pixel4 dest )
{
float2 center = floor(outCoord() / spacing)*spacing + (spacing / 2.0);
float dist = distance( center, outCoord() );
pixel4 originalColor = sampleNearest( src, outCoord() );
dest = dist < radius ? pink : originalColor;
}
}
Hydra in detail – Syntax and Semantics
77. 2007 Adobe Systems Incorporated. All Rights Reserved.
77
kernel PinkPolkaDots
{
parameter float radius;
parameter float spacing;
const pixel4 pink = pixel4( 1.0, 0.75 ,0.8, 1.0 );
void evaluatePixel( in image4 src, out pixel4 dest )
{
float2 center = floor(outCoord() / spacing)*spacing + (spacing / 2.0);
float dist = distance( center, outCoord() );
pixel4 originalColor = sampleNearest( src, outCoord() );
dest = dist < radius ? pink : originalColor;
}
}
Hydra in detail – Syntax and Semantics
78. 2007 Adobe Systems Incorporated. All Rights Reserved.
78
float
int
bool
pixel1
int2 float2 bool2 pixel2
int3 float3 bool3 pixel3
int4 float4 bool4 pixel4
float2x2
float3x3
float4x4
Hydra in detail – Types
79. 2007 Adobe Systems Incorporated. All Rights Reserved.
79
float
int
bool
pixel1
int2 float2 bool2 pixel2
int3 float3 bool3 pixel3
int4 float4 bool4 pixel4
float2x2
float3x3
float4x4
Hydra in detail – Types
80. 2007 Adobe Systems Incorporated. All Rights Reserved.
80
float
int
bool
pixel1
int2 float2 bool2 pixel2
int3 float3 bool3 pixel3
int4 float4 bool4 pixel4
float2x2
float3x3
float4x4
Hydra in detail – Types
81. 2007 Adobe Systems Incorporated. All Rights Reserved.
81
kernel Twirl
{
parameter float radius;
parameter float2 center;
parameter float twirlAngle;
void evaluatePixel( in image4 src, out float4 dest )
{
float2 relativePos = outCoord() - center;
float distFromCenter = length( relativePos ) / radius;
float cosAngle = cos(twirlAngle);
float sinAngle = sin(twirlAngle);
float2x2 rotationMat = float2x2(
cosAngle, sinAngle,
-sinAngle, cosAngle );
relativePos = rotationMat * relativePos;
dest = sampleLinear( src, relativePos + center );
}
}
Hydra in detail – Operators
82. 2007 Adobe Systems Incorporated. All Rights Reserved.
82
kernel Twirl
{
parameter float radius;
parameter float2 center;
parameter float twirlAngle;
void evaluatePixel( in image4 src, out float4 dest )
{
float2 relativePos = outCoord() - center;
float distFromCenter = length( relativePos ) / radius;
float cosAngle = cos(twirlAngle);
float sinAngle = sin(twirlAngle);
float2x2 rotationMat = float2x2(
cosAngle, sinAngle,
-sinAngle, cosAngle );
relativePos = rotationMat * relativePos;
dest = sampleLinear( src, relativePos + center );
}
}
Hydra in detail – Operators
83. 2007 Adobe Systems Incorporated. All Rights Reserved.
83
kernel Twirl
{
parameter float radius;
parameter float2 center;
parameter float twirlAngle;
void evaluatePixel( in image4 src, out float4 dest )
{
float2 relativePos = outCoord() - center;
float distFromCenter = length( relativePos ) / radius;
float cosAngle = cos(twirlAngle);
float sinAngle = sin(twirlAngle);
float2x2 rotationMat = float2x2(
cosAngle, sinAngle,
-sinAngle, cosAngle );
relativePos = rotationMat * relativePos;
dest = sampleLinear( src, relativePos + center );
}
}
Hydra in detail – Operators
84. 2007 Adobe Systems Incorporated. All Rights Reserved.
84
Hydra in detail – Functions
sin
cos
tan
asin
acos
atan
atan
radians
degrees
pow
exp
exp2
log
log2
sqrt
abs
sign
floor
ceil
fract
mod
min
max
step
clamp
mix
smoothStep
matrixCompMult
inverseSqrt
length
distance
dot
cross
any
all
not
nowhere
everywhere
transform
union
intersect
outset
inset
bounds
isEmpty
sample
sampleLinear
sampleNearest
lessThan
lessThanEqual
greaterThan
greaterThanEqual
equal
notEqual
85. 2007 Adobe Systems Incorporated. All Rights Reserved.
85
Hydra in detail – Functions
sin
cos
tan
asin
acos
atan
atan
radians
degrees
pow
exp
exp2
log
log2
sqrt
abs
sign
floor
ceil
fract
mod
min
max
step
clamp
mix
smoothStep
matrixCompMult
inverseSqrt
length
distance
dot
cross
any
all
not
nowhere
everywhere
transform
union
intersect
outset
inset
bounds
isEmpty
sample
sampleLinear
sampleNearest
lessThan
lessThanEqual
greaterThan
greaterThanEqual
equal
notEqual
86. 2007 Adobe Systems Incorporated. All Rights Reserved.
86
Hydra in detail – Functions
sin
cos
tan
asin
acos
atan
atan
radians
degrees
pow
exp
exp2
log
log2
sqrt
abs
sign
floor
ceil
fract
mod
min
max
step
clamp
mix
smoothStep
matrixCompMult
inverseSqrt
length
distance
dot
cross
any
all
not
nowhere
everywhere
transform
union
intersect
outset
inset
bounds
isEmpty
sample
sampleLinear
sampleNearest
lessThan
lessThanEqual
greaterThan
greaterThanEqual
equal
notEqual
87. 2007 Adobe Systems Incorporated. All Rights Reserved.
87
Hydra in detail – Functions
sin
cos
tan
asin
acos
atan
atan
radians
degrees
pow
exp
exp2
log
log2
sqrt
abs
sign
floor
ceil
fract
mod
min
max
step
clamp
mix
smoothStep
matrixCompMult
inverseSqrt
length
distance
dot
cross
any
all
not
nowhere
everywhere
transform
union
intersect
outset
inset
bounds
isEmpty
sample
sampleLinear
sampleNearest
lessThan
lessThanEqual
greaterThan
greaterThanEqual
equal
notEqual
88. 2007 Adobe Systems Incorporated. All Rights Reserved.
88
ƒ Introduction
ƒ Demonstration – some sample filters
ƒ Code walkthrough – identity filter
ƒ Hydra programming model
ƒ Code walkthrough – polka dot filter
ƒ Hydra in detail
ƒ Hydra for Flash
ƒ Efficiency tips
ƒ Demonstration – AIF toolkit
ƒ Bibliography
ƒ Summary
ƒ AIF server
Hydra for Flash
89. 2007 Adobe Systems Incorporated. All Rights Reserved.
89
Hydra for Flash
90. 2007 Adobe Systems Incorporated. All Rights Reserved.
90
ƒ Introduction
ƒ Demonstration – some sample filters
ƒ Code walkthrough – identity filter
ƒ Hydra programming model
ƒ Code walkthrough – polka dot filter
ƒ Hydra in detail
ƒ Hydra for Flash
ƒ Efficiency tips
ƒ Demonstration – AIF toolkit
ƒ Bibliography
ƒ Summary
ƒ AIF server
Efficiency tips
91. 2007 Adobe Systems Incorporated. All Rights Reserved.
91
Write a function that produces a single
output pixel
Efficiency tips
92. 2007 Adobe Systems Incorporated. All Rights Reserved.
92
Efficiency tips
93. 2007 Adobe Systems Incorporated. All Rights Reserved.
93
Efficiency tips
94. 2007 Adobe Systems Incorporated. All Rights Reserved.
94
Efficiency tips
95. 2007 Adobe Systems Incorporated. All Rights Reserved.
95
Efficiency tips
96. 2007 Adobe Systems Incorporated. All Rights Reserved.
96
ƒ Introduction
ƒ Demonstration – some sample filters
ƒ Code walkthrough – identity filter
ƒ Hydra programming model
ƒ Code walkthrough – polka dot filter
ƒ Hydra in detail
ƒ Hydra for Flash
ƒ Efficiency tips
ƒ Demonstration – AIF toolkit
ƒ Bibliography
ƒ Summary
ƒ AIF server
Demonstration – AIF toolkit
97. 2007 Adobe Systems Incorporated. All Rights Reserved.
97
Writing a kernel in the AIF Toolkit
98. 2007 Adobe Systems Incorporated. All Rights Reserved.
98
ƒ Andrew S. Glassner – Principles of Digital Image Synthesis
ƒ Randi Rost – OpenGL® Shading Language
ƒ George Wolberg - Digital Image Warping
ƒ Randima Fernando – GPU Gems
ƒ Matt Pharr & Randima Fernando – GPU Gems 2
ƒ Hurbert Nguyen – GPU Gems 3
ƒ Shader examples: http://shady.goyman.com/
Bibliography
99. 2007 Adobe Systems Incorporated. All Rights Reserved.
99
ƒ Hydra language
ƒ AIF Toolkit
ƒ http://labs.adobe.com/wiki/index.php/AIF_Toolkit
Summary
100. 2007 Adobe Systems Incorporated. All Rights Reserved.
10
0
ƒ Introduction
ƒ Demonstration – some sample filters
ƒ Code walkthrough – identity filter
ƒ Hydra programming model
ƒ Code walkthrough – polka dot filter
ƒ Hydra in detail
ƒ Hydra for Flash
ƒ Efficiency tips
ƒ Demonstration – AIF toolkit
ƒ Bibliography
ƒ Summary
ƒ AIF server
AIF server