Dr Lindsay MacDonald, 3DIMPact Research Group, Faculty of Engineering, UCL: Image Sets under Directional Lighting: A Richer Representation of Cultural Heritage Objects

1. Image Sets under Directional Lighting UCL Centre for Digital Humanities
Seminar – 11th March 2015
Dr Lindsay MacDonald, 3DIMPact Research Group,
Department of Civil, Environmental and Geomatic Engineering, UCL 1
Image Sets under Directional Lighting
A Richer Representation of Cultural Heritage Objects
Lindsay MacDonald
Department of Civil, Environmental and Geomatic Engineering
University College London
Digital Humanities Seminar, March 2015
Engelbach, R. (1934)
A foundation scene of the second dynasty.
The Journal of Egyptian Archaeology,
Vol. 20, pp.183‐184.

2. Image Sets under Directional Lighting UCL Centre for Digital Humanities
Seminar – 11th March 2015
Dr Lindsay MacDonald, 3DIMPact Research Group,
Department of Civil, Environmental and Geomatic Engineering, UCL 2
The power of raking light
to reveal surface detail…
http://www.factumfoundation.org/pag/208/Making‐the‐Facsimile
The UCL Dome
1 metre hemisphere
64 flash lights
Nikon D200 camera
In use since 2006

3. Image Sets under Directional Lighting UCL Centre for Digital Humanities
Seminar – 11th March 2015
Dr Lindsay MacDonald, 3DIMPact Research Group,
Department of Civil, Environmental and Geomatic Engineering, UCL 3
Layout of lamps
• 64 lights in total
• 5 horizontal tiers
• Aim to distribute
lights approximately
uniformly over the
hemisphere
A
E
D
C
B
16
15
14
13
1211
10
9
8
7
6
5
4
3
2
1
Plan
Elevation
Nikon D200
Fixed mounting

4. Image Sets under Directional Lighting UCL Centre for Digital Humanities
Seminar – 11th March 2015
Dr Lindsay MacDonald, 3DIMPact Research Group,
Department of Civil, Environmental and Geomatic Engineering, UCL 4
Flash board
Flash board
Top view (outside) Bottom view (inside)

5. Image Sets under Directional Lighting UCL Centre for Digital Humanities
Seminar – 11th March 2015
Dr Lindsay MacDonald, 3DIMPact Research Group,
Department of Civil, Environmental and Geomatic Engineering, UCL 5
Hinged
superstructure
Object placed
on baseboard

6. Image Sets under Directional Lighting UCL Centre for Digital Humanities
Seminar – 11th March 2015
Dr Lindsay MacDonald, 3DIMPact Research Group,
Department of Civil, Environmental and Geomatic Engineering, UCL 6
Imaging cultural artefacts
Petrie Museum, UCL
Egyptian funerary cone
c.1200 BC
Refers to a priest Nefer‐Iah
and the moon‐god Lah,
and a priestess Hemet‐
Netjer and the god Amun.

7. Image Sets under Directional Lighting UCL Centre for Digital Humanities
Seminar – 11th March 2015
Dr Lindsay MacDonald, 3DIMPact Research Group,
Department of Civil, Environmental and Geomatic Engineering, UCL 7
Column 2
To the ka‐spirit of the
high priest of Iah, Nefer‐
Column 1
Iah true of voice,
revered, at peace.
Column 3
Lady of the house, chantress
of Amun, singer of Mut,
Column 4
Hemet‐netjer, true of voice,
at peace, beloved.
Translated by Prof Stephen Quirke,
UCL Institute of Archaeology
80°

8. Image Sets under Directional Lighting UCL Centre for Digital Humanities
Seminar – 11th March 2015
Dr Lindsay MacDonald, 3DIMPact Research Group,
Department of Civil, Environmental and Geomatic Engineering, UCL 8
60°
40°

9. Image Sets under Directional Lighting UCL Centre for Digital Humanities
Seminar – 11th March 2015
Dr Lindsay MacDonald, 3DIMPact Research Group,
Department of Civil, Environmental and Geomatic Engineering, UCL 9
20°
5°

10. Image Sets under Directional Lighting UCL Centre for Digital Humanities
Seminar – 11th March 2015
Dr Lindsay MacDonald, 3DIMPact Research Group,
Department of Civil, Environmental and Geomatic Engineering, UCL 10

11. Image Sets under Directional Lighting UCL Centre for Digital Humanities
Seminar – 11th March 2015
Dr Lindsay MacDonald, 3DIMPact Research Group,
Department of Civil, Environmental and Geomatic Engineering, UCL 11

12. Image Sets under Directional Lighting UCL Centre for Digital Humanities
Seminar – 11th March 2015
Dr Lindsay MacDonald, 3DIMPact Research Group,
Department of Civil, Environmental and Geomatic Engineering, UCL 12

13. Image Sets under Directional Lighting UCL Centre for Digital Humanities
Seminar – 11th March 2015
Dr Lindsay MacDonald, 3DIMPact Research Group,
Department of Civil, Environmental and Geomatic Engineering, UCL 13

14. Image Sets under Directional Lighting UCL Centre for Digital Humanities
Seminar – 11th March 2015
Dr Lindsay MacDonald, 3DIMPact Research Group,
Department of Civil, Environmental and Geomatic Engineering, UCL 14

15. Image Sets under Directional Lighting UCL Centre for Digital Humanities
Seminar – 11th March 2015
Dr Lindsay MacDonald, 3DIMPact Research Group,
Department of Civil, Environmental and Geomatic Engineering, UCL 15

16. Image Sets under Directional Lighting UCL Centre for Digital Humanities
Seminar – 11th March 2015
Dr Lindsay MacDonald, 3DIMPact Research Group,
Department of Civil, Environmental and Geomatic Engineering, UCL 16

17. Image Sets under Directional Lighting UCL Centre for Digital Humanities
Seminar – 11th March 2015
Dr Lindsay MacDonald, 3DIMPact Research Group,
Department of Civil, Environmental and Geomatic Engineering, UCL 17

18. Image Sets under Directional Lighting UCL Centre for Digital Humanities
Seminar – 11th March 2015
Dr Lindsay MacDonald, 3DIMPact Research Group,
Department of Civil, Environmental and Geomatic Engineering, UCL 18
Increasing
angle of
elevation
Set of 64 images
100x100 pixel detail
Image sets from the dome
1. Visualisation by interactive movement of a virtual light source
2. 3D reconstruction of the object surface
3. Modelling of specular highlights from the surface
Each image is illuminated from a different direction.
All images are in pixel register.
A richer representation than normal photography!
Three ways of using dome image sets

19. Image Sets under Directional Lighting UCL Centre for Digital Humanities
Seminar – 11th March 2015
Dr Lindsay MacDonald, 3DIMPact Research Group,
Department of Civil, Environmental and Geomatic Engineering, UCL 19
Part 1 – Interactive visualisation
Imagine moving a candle around to different positions over a surface
Appearance of surface relief changes as light is moved: the illusion of 3D
Intensity distribution at one pixel
• Vector of 64 values
• Low values similar to
cosine (Lambertian)
• Few high values near
specular direction

20. Image Sets under Directional Lighting UCL Centre for Digital Humanities
Seminar – 11th March 2015
Dr Lindsay MacDonald, 3DIMPact Research Group,
Department of Civil, Environmental and Geomatic Engineering, UCL 20
Azimuthal equidistant projection
• Projection of
hemisphere onto plane
• Useful properties:
All points on map are at
proportionately correct
distances from centre
All points on map are at
correct azimuth (direction)
from the centre point
Polar plot of distribution
• Plot intensities of pixel
for 64 lamps.
• Position in plane
corresponds to direction
vector of lamp.
• Specular values stand
out above others.

21. Image Sets under Directional Lighting UCL Centre for Digital Humanities
Seminar – 11th March 2015
Dr Lindsay MacDonald, 3DIMPact Research Group,
Department of Civil, Environmental and Geomatic Engineering, UCL 21
Polynomial texture mapping
• A form of the bidirectional texture function (BTF)
model, simplified by holding exitant direction constant
with reflected angle always toward fixed camera:
• Assuming a Lambertian surface, the reconstruction
function is separable, with a constant colour per pixel
modulated by an angle‐dependent luminance factor L:
Malzbender, T., Gelb, D. and Wolters, H. (2001)
‘Polynomial Texture Maps’, Proc. ACM SIGGRAPH, 28, 519‐528.
, , Θ , Φ , ,
Θ , Φ , , , , ,
u, v ‐ texture coordinates
a0 - a5 ‐ fitted coefficients stored in texture map
lu, lv ‐ projection of light direction into texture plane
5v4u3vu2
2
v1
2
u0vu alalallalala),lL(u,v;l 
lu
lv
Light direction parametrisation

22. Image Sets under Directional Lighting UCL Centre for Digital Humanities
Seminar – 11th March 2015
Dr Lindsay MacDonald, 3DIMPact Research Group,
Department of Civil, Environmental and Geomatic Engineering, UCL 22







































 1
1
0
5
1
0
111111
111111
000000
1
1
1
NvNuNvNuN
2
vN
2
uN
vuvu
2
v
2
u
vuvu
2
v
2
u
L
L
L
a
a
a
llllll
llllll
llllll

For each pixel, given N light sources and observed
intensities L0 … LN-1 (reflected from the object),
compute best fit for the six parameters (a0‐a5)
using Singular Value Decomposition (SVD).
Fitting PTM to image data
Inverse matrix calculated only once
Polar plot of PTM distribution
• PTM values for 64
lamps calculated by
biquadratic function.
• Models cosine
distribution well
• Cannot model the
specular peak

23. Image Sets under Directional Lighting UCL Centre for Digital Humanities
Seminar – 11th March 2015
Dr Lindsay MacDonald, 3DIMPact Research Group,
Department of Civil, Environmental and Geomatic Engineering, UCL 23
Demonstration of
PTM viewer
Software
developed
by Tom
Malzbender
at HP Labs
in 2001
Circular area
represents
hemisphere
for the
movement
of virtual
light source
Hemispherical harmonics
1/2
6/ cos cos cos
3/2 2cos 1
6/ sin cos cos
30/ cos 2 cos cos
30/ cos 1 2cos cos cos
5/2 1 6 cos cos
30/ sin 1 2cos cos cos
30/ sin 2 cos cos
140/ cos 3 cos cos
210/ cos 2 1 2cos cos cos
84/ cos cos cos 1 5 cos cos
7/2 12cos 1 30cos 20cos
84/ sin cos cos 1 5 cos cos
210/ sin 2 1 2cos cos cos
140/ sin 3 cos cos
atan2 ,
acos 1
Change of variables:
Azimuth
Co‐latitude
First order
Second order
Third order
Co‐latitude

24. Image Sets under Directional Lighting UCL Centre for Digital Humanities
Seminar – 11th March 2015
Dr Lindsay MacDonald, 3DIMPact Research Group,
Department of Civil, Environmental and Geomatic Engineering, UCL 24
Hemispherical harmonics
First 16 modes
plotted on polar
plane
Green = positive
Blue = negative
Modelling intensity distributions
PTM
HSH
Order 1
HSH
Order 2
HSH
Order 3

25. Image Sets under Directional Lighting UCL Centre for Digital Humanities
Seminar – 11th March 2015
Dr Lindsay MacDonald, 3DIMPact Research Group,
Department of Civil, Environmental and Geomatic Engineering, UCL 25
Angular resolution
Spacing of lamps in dome
sets limit on fineness of
detail that can be resolved
in an angular sense.
Range of angles for all
neighbouring lamps is
12–28 with median 20.
Reflection transform imaging
Generalisation of PTM with enhancements:
• Basis functions (spherical harmonics)
• High dynamic range (HDR) imaging
• Virtual dome (movable light source)
• Highlight‐based calibration (spherical targets)
• More flexible file format *.rti
Specular highlight on black
sphere enables position of
light source to be determined

26. Image Sets under Directional Lighting UCL Centre for Digital Humanities
Seminar – 11th March 2015
Dr Lindsay MacDonald, 3DIMPact Research Group,
Department of Civil, Environmental and Geomatic Engineering, UCL 26
RTI in action
Photographer Elizabeth Minor and assistant Kierstin Sakai during RTI capture,
with raking light angle (PAHMA) – Hearst Museum, Berkeley, CA
Applications of RTI
Coins Rock art Cuneiform tablets
Fossils Byzantine glass
tesseræ
Marble friezes

27. Image Sets under Directional Lighting UCL Centre for Digital Humanities
Seminar – 11th March 2015
Dr Lindsay MacDonald, 3DIMPact Research Group,
Department of Civil, Environmental and Geomatic Engineering, UCL 27
Advantages of PTM/RTI in cultural heritage
• Non‐contact acquisition
• Convincing illusion of 3D shape
• Interactive visualisation
• Better discernment of surface
detail than physical examination
• No data loss due to shadows
and specular highlights
• Simple and achievable image
processing pipeline
• Higher resolution on object
surface than with 3D scanners
Marble capital
Museum of San Matteo
Factors affecting quality of PTM/RTI
• Spatial resolution of images
• SNR and dynamic range
• Number of light sources
• Fit of basis functions to actual
surface reflectance distribution
• Spectral resolution, i.e. ability to
reconstruct reflectance spectrum
The Antikythera Mechanism
Freeth et al (2006) Nature

28. Image Sets under Directional Lighting UCL Centre for Digital Humanities
Seminar – 11th March 2015
Dr Lindsay MacDonald, 3DIMPact Research Group,
Department of Civil, Environmental and Geomatic Engineering, UCL 28
Part 2 – Reconstructing height of surface
Very few surfaces are perfectly planar.
We are interested in 2½D surfaces, i.e. flat with relief.
Try to make a digital terrain map (DTM)
Use principle of ‘shape from shading’, aka photometric stereo.
Surface normals
• A normal N to a surface S at point P is a
a vector perpendicular to the tangent
plane touching surface at P.
• For a set of points satisfying S x,y,z 0, a normal at
x,y,z on surface is gradient formed by partial first
derivatives wrt each axis:
• Where surface is defined by z S x,y , the normal is:
, , , ,
T
̂ ̂
N
S
P
, , 1
T
, , 1 T

29. Image Sets under Directional Lighting UCL Centre for Digital Humanities
Seminar – 11th March 2015
Dr Lindsay MacDonald, 3DIMPact Research Group,
Department of Civil, Environmental and Geomatic Engineering, UCL 29
Photometric stereo
For a Lambertian surface, from which incident light is
scattered equally in all directions, the luminance of
reflected light is given by vector dot product:
∙ | | cos
∙
Three equations are needed to
solve system, by illuminating the
surface in successive images from
three lighting directions with
incident vectors L1, L2, L3 :
L1
L2
L3
Test object – Chopin terracotta
Tier 1 lamps – lowest Tier 5 lamps – highest

30. Image Sets under Directional Lighting UCL Centre for Digital Humanities
Seminar – 11th March 2015
Dr Lindsay MacDonald, 3DIMPact Research Group,
Department of Civil, Environmental and Geomatic Engineering, UCL 30
Effect of specular values on photometric stereo
V
N
S
N’
P
Normal distorted away from view vector Computed normal for spherical surface
Finding normals by avoiding specular values
Intensity vs lamp number at one pixel Intensity values sorted into ascending order
Choose subset with
slope similar to cosine

31. Image Sets under Directional Lighting UCL Centre for Digital Humanities
Seminar – 11th March 2015
Dr Lindsay MacDonald, 3DIMPact Research Group,
Department of Civil, Environmental and Geomatic Engineering, UCL 31
Normals and albedo
False colour: X in R, Y in G, Z in B
Gradients
Slopes in X and Y directions
are given by partial derivatives
of height wrt X and Y:
Compute intensity gradients
from normals:

32. Image Sets under Directional Lighting UCL Centre for Digital Humanities
Seminar – 11th March 2015
Dr Lindsay MacDonald, 3DIMPact Research Group,
Department of Civil, Environmental and Geomatic Engineering, UCL 32
Simple summation of gradients
Cumulative sums of P gradients
in two directions across horizontal midline.
Cross‐sections of P and Q gradients
across horizontal midline.
Horizontal and vertical image summations
Summation along rows Summation down columns

33. Image Sets under Directional Lighting UCL Centre for Digital Humanities
Seminar – 11th March 2015
Dr Lindsay MacDonald, 3DIMPact Research Group,
Department of Civil, Environmental and Geomatic Engineering, UCL 33
Measuring height at a few points
Height measuring gauge
Difference from reference height
Mean of two summations Error range from 0.96 to 7.48 mm

34. Image Sets under Directional Lighting UCL Centre for Digital Humanities
Seminar – 11th March 2015
Dr Lindsay MacDonald, 3DIMPact Research Group,
Department of Civil, Environmental and Geomatic Engineering, UCL 34
Profile of reconstruction
The clay disc on which Chopin’s head is moulded is tilted on a diagonal
axis from upper left to lower right. Thus summation of gradients has
stretched the scale of the relief, while compressing the height of the disc.
View from south‐east at zero elevation, parallel to the X‐Y plane.
Height reconstruction by Fourier transform
Technique yields 3D surface that is continuous and is recognisably Chopin, but is
distorted over the whole area with the height greatly amplified. Also there is a
false undulation of base with period of approximately one cycle over whole width.
Frankot R.T. and Chellappa R. (1988) A method for enforcing integrability in shape from shading algorithms,
IEEE Trans. on Pattern Analysis & Machine Intelligence 10(4):439‐451.

35. Image Sets under Directional Lighting UCL Centre for Digital Humanities
Seminar – 11th March 2015
Dr Lindsay MacDonald, 3DIMPact Research Group,
Department of Civil, Environmental and Geomatic Engineering, UCL 35
Replacing inaccurate low frequencies
Smooth surface of hump produced by
interpolation of measured points.
Log(power) distribution of spatial
frequencies of hump gradients.
Log(power) distribution of spatial
frequencies of photometric gradients.
The low spatial frequencies of gradients from Frankot‐Chellappa
integration can be replaced by corresponding frequencies from hump.
Linear combination of spatial frequencies
Blended over a radial distance in the range 1.5 to 4.0
cycles/width by a linear interpolation (lerp) function.
Blending functions α and 1‐α. Oblique view of reconstruction

36. Image Sets under Directional Lighting UCL Centre for Digital Humanities
Seminar – 11th March 2015
Dr Lindsay MacDonald, 3DIMPact Research Group,
Department of Civil, Environmental and Geomatic Engineering, UCL 36
Difference from reference height
Elevation view Error range from ‐1.31 to +1.92 mm
Using a laser scanner
Arius 3D colour laser scanner Rendering from point cloud

37. Image Sets under Directional Lighting UCL Centre for Digital Humanities
Seminar – 11th March 2015
Dr Lindsay MacDonald, 3DIMPact Research Group,
Department of Civil, Environmental and Geomatic Engineering, UCL 37
Conclusions on height reconstruction
• Photometric stereo provides excellent normals but lacks overall scale.
• Scale can be obtained from a few discrete height measurements.
• Results are much higher in visual quality than laser scan with texture map.
Many applications in
producing surrogates.
Scarab of steatite with gold band
Petrie Museum UC11365
CloudCompare
matching of
point clouds
from Arius 3D
scanner and
reconstruction
from dome
image set.
Part 3 – Specular reflectance distribution
The world is not filled with Lambertian surfaces!
All real objects have some gloss or sheen.
Specular (from Latin speculum) is mirror‐like reflection.
Aim to model surface reflection
as sum of diffuse body colour
plus specular component.

38. Image Sets under Directional Lighting UCL Centre for Digital Humanities
Seminar – 11th March 2015
Dr Lindsay MacDonald, 3DIMPact Research Group,
Department of Civil, Environmental and Geomatic Engineering, UCL 38
Roman medallionTest object
64 images
8x8 mosaic
Detail 200x200
Pixel sampled
from below eye

39. Image Sets under Directional Lighting UCL Centre for Digital Humanities
Seminar – 11th March 2015
Dr Lindsay MacDonald, 3DIMPact Research Group,
Department of Civil, Environmental and Geomatic Engineering, UCL 39
Variety of colour in a single pixel
Same camera
Same object
Same illumination
Same point
Same pixel
Same scaling
64 incident light directions
Intensity distribution at one pixel
• Vector of 64 values
• Low values similar to
cosine (Lambertian)
• Few high values near
specular direction

40. Image Sets under Directional Lighting UCL Centre for Digital Humanities
Seminar – 11th March 2015
Dr Lindsay MacDonald, 3DIMPact Research Group,
Department of Civil, Environmental and Geomatic Engineering, UCL 40
Finding surface normal from distribution
• Sort 64‐value distribution
into ascending order
• Lowest values in shadow
• High values near specular
• Middle values are good
approximation of cosine
• Use regression over
selected vectors
Plotting 3D vector distribution at pixel
• V is view vector
• N is normal vector
• S is specular vector
• Lamp vectors shown in
red are excluded.
• Lamp vectors shown in
blue are selected to
calculate normal by
regression.
V
N
S

41. Image Sets under Directional Lighting UCL Centre for Digital Humanities
Seminar – 11th March 2015
Dr Lindsay MacDonald, 3DIMPact Research Group,
Department of Civil, Environmental and Geomatic Engineering, UCL 41
Albedo Normal
Calculate ‘specular quotient’
• Ratio at each pixel of
actual intensity/diffuse
• ~1 for matte areas
• >>1 for shiny areas

42. Image Sets under Directional Lighting UCL Centre for Digital Humanities
Seminar – 11th March 2015
Dr Lindsay MacDonald, 3DIMPact Research Group,
Department of Civil, Environmental and Geomatic Engineering, UCL 42
Plot ‘specular quotient’ vs angle
• High values near
specular peak.
• Falling to asymptote of 1
with increasing radial
angle from peak.
• Plotted here for 3x3
pixel cell, giving 9x64 =
576 values.
Fitting angular distribution of specular intensity
• Various functions
for BRDF models in
computer graphics.
• Lorentzian function
chosen for its broad
flanks:
1
1 ⁄

43. Image Sets under Directional Lighting UCL Centre for Digital Humanities
Seminar – 11th March 2015
Dr Lindsay MacDonald, 3DIMPact Research Group,
Department of Civil, Environmental and Geomatic Engineering, UCL 43
Fitting angular distribution of specular intensity
• First fit linear flank.
• Then fit Lorentzian
function for values
above flank.
• Four parameters in
combined model:
1 ⁄
Polar plot of Lorentz distribution
• Diffuse values for 64
lamps calculated by
cosine (blue).
• Specular values for 64
lamps calculated by
Lorentzian (red).
• Sum is good match to
the intensity data.

44. Image Sets under Directional Lighting UCL Centre for Digital Humanities
Seminar – 11th March 2015
Dr Lindsay MacDonald, 3DIMPact Research Group,
Department of Civil, Environmental and Geomatic Engineering, UCL 44
Image components of specular model
Specular
amplitude
Specular
width
Flank
slope
Flank
offset
Photographic image Modelled image

45. Image Sets under Directional Lighting UCL Centre for Digital Humanities
Seminar – 11th March 2015
Dr Lindsay MacDonald, 3DIMPact Research Group,
Department of Civil, Environmental and Geomatic Engineering, UCL 45
• To obtain visual realism the
directionality of the lighting
must be considered.
• The Lorentzian function provides
a good basis for modelling the
specular component of reflectance.
• With a continuous function of
angle, views can be interpolated
between the original photographs.
Conclusions on specular rendering
Overall conclusions
Sets of images with structured light provide a
much richer representation than a single image
1. Interactive visualisation and rendering
2. 3D reconstruction of the object surface
3. Modelling of specular highlights
There are many applications in cultural heritage
for digitising and display of objects that are
flattish with surface relief:
– coins, medals, fossils, rock art, incised tablets,
bas reliefs, engravings, canvas paintings, etc.
Islamic handbag, c.1310, Mosul, Iraq
Courtauld Gallery, London