AmCAT is a document management and analysis platform for the social sciences and humanities. It allows users to input, manage, and analyze documents through features like keyword analysis, linguistic processing, and manual annotation. AmCAT is open source, uses open standards, and aims to provide open access to data. It is built using Django and separates business logic from presentation, with all data and methods exposed through a REST API for accessibility from scripts.

1. AmCAT3
Using Django for a scientific document analysis
website: Tastypie, unit tests, R, open platforms
and open questions
Wouter van Atteveldt (VU Amsterdam)

2. AmCAT

What is AmCAT?

Design considerations

Open data and the publication cycle

Tables, TastyPie, and R

Unit tests

3. What is AmCAT?

Document management and analysis

Aimed at social sciences and humanities

Input: scraping, uploading

Management: projects, selections

Analyses: keyword analysis, linguistic processing
(lemmatizing etc), manual annotation

Open source, open standards, open access

8. Design Choices

Default Django: web site backed by a database

AmCAT: database with a web front end

9. Design Choices

Default Django: web site backed by a database

AmCAT: database with a web front end

Data should be accessible from outside

ORM should be usable without web site code

DB should be final authentication/authorisation

10. Design choices

Separate 'apps' for business, presentation

Custom authentication middleware and user
management

save() and update() with using=

database-specific code for creating users

We don't actually like this too much...

All data and methods (should be) exposed
through web service API

11. Open data and Publication Cycle
AmCAT Navigator
(web site)
REST API
ORM
(web service)
(django)
Relational
SPARQL External scripts
DB
End point (Python, R, ...)

12. Open access publication cycle
Source: Analysis: Publication:
DANS/AmCAT3
(Linked) data R, matlab, ... e.g. Sweave PDF + hyperlinks
Web service + Latex
Structured data?
'data link'
from site
Links back to

13. Tastypie + Datatables

Django Model-based REST api

Jquery datatables with AJAX call

The good news:

It works

Unified point of entry for tables in website and
scripts

The bad news:

Tastypie code horribly redundant

(Unless we're doing it wrong!)

14. Unit tests

Web pages tough to test well

Move as much code as possible from
presentation to business layer

Trivial views need less testing

Regular python modules easy to test

Our choices:

Put all unit tests in the 'target' module

Put more complicated integration tests in tests/
package

15. Bonus slide: Plugins

Django (model)forms as interface description
for plugins

Plugins callable from web site, as web service,
and from cli

Single point of entry for actions

(relation with REST data modification?)