This document discusses using computational grids for resource-intensive digital humanities projects. It provides two examples of projects that would benefit from using a grid: 1) comparing OCR text to ground truth which requires large amounts of memory, and 2) aligning multiple OCR streams and generating error patterns which is computationally intensive. The document raises questions about preparing code written in Java to run on a grid, whether the programming language matters, how input/output operations and storage work on a grid, and if thread-based programs would be better suited for a grid.

1. Greedy enough for the Grid?
Matteo Romanello
Centre for Computing in the Humanities
KCL

2. Overview
● PhD in Digital Humanities: co-supervised between CCH
and dCS at KCL
● Information extraction from Classics-related materials
● This talk: pragmatic approach to the use of the Grid from a
newbie/user perspective
● NGS UK: “aims to enable coherent electronic access for
UK researchers to all computational and data based
resources and facilities required to carry out their
research, independent of resource or researcher location.”

3. Grid and scalability
● Computational intensive processes are
particular greedy in terms of resource use
● (simple) processes dry a *normal* machine
● Size of text corpora causes performance
problems

4. Example 1
● Code written in Java/Groovy
● Comparing the OCR output with the groundtruth
● String comparison for alignment (line by line)
● Need to execute batch processes (doc by doc,
page by page, line by line etc.)
● 1024mb of heap space to be released for Java to
run the process on a small size sample (not to talk
about an XSLT transformation on a 7mb XML file)

5. Example 2
● Code in Java (author Federico Boschetti)
● Alignment of multiple OCR streams
● Generation of error patterns for each OCR
engine
● Output → a new OCR transcription resulting
from the merging of the input OCR streams
● Merging based on the probabilistic likelihood of
a character to be correct provided the error
pattern of each OCR stream

6. Get ready for the Grid
● Does the prog language matter? (my code is
already in Java...)
● How I/O operations work on the Grid? How to
rely on Grid storage?
● Better having thread-based programs?
● How to prepare my own code to be ran on a
grid, i.e. gridifying my application

7. Thank you for your attention!