Django in the Office: Get Your Admin for Nothing and Your SQL for Free

Django in the Office:
Get Your Admin for Nothing
and Your SQL for Free
(Department of Molecular & Cellular Biology)

11.8.2011

“I wish I had a database”

source: Bureaucratics by Jan Banning

custom websites/databases

pre-2004/05 2005+

Django: python-based web framework

source: http://xkcd.com/353/

Python and Django: Under-Hyped, Widely Used

Django: Scalable (without magic and superclouds)

Firefox‟s Add-Ons Site Commenting System

250k add-ons 17,000 requests/second
150 million views/month 40 million users/month
500+ million api hits/day 450,000 websites
(rewritten from CakePHP) 15 million profiles
75 million comments

(stats from June 2011) (stats from 10/2010)

can start small: instrument scheduler

(stats from July 2011. initial launch in jan/feb 2009)

Django: Maintenance/Security

Django
- 2011 – 2 updates
- 2010 – 2 updates
- 2009 – 2 updates

- Only update is the core Django code (python files)
- No changes to your code
- No changes to the database
- No changes to HTML templates
- No changes to static or uploaded files

Python and Django: Online and Local Support

Python: “For Dummies” - list

>>> l = ['cat', 'fish', 7, 'eats’, 108]
>>> l.sort()
>>> for x in l:
... print x
7
108
cat
eats
fish

>>> len(x)
5

Python: “For Dummies” - dict

>>> d = {1:'Bob', 2:'Susan', ‘buckle’:’shoe'}
>>> d.keys()
[1, 2, ‘buckle’]

>>> for k,v in d.iteritems():
... print k, v.capitalize()
1 Bob
2 Susan
buckle Shoe

>>> print d.get(4, ‘not found’)
not found

what about in the office?

story of a 1-day, limited-use project
(“raw example/bad naming, etc”)

Wednesday: The Request

• 200+ trainees

• 40 different labs or offices

• 8 training dates

• 2 types of training

• 5 trainee classifications

Wednesday: The Response

Let‟s make a database!



You can manage it through a website admin!




Sortable column headers!





You can have searching and filters!






We can send personalized emails with RSVP links!!






We can send personalized emails with RSVP links!!

Just in case, you can download it all back to Excel!

Wednesday: “Over Promising”

Thursday: Resource Constraints

1 FTE dependent on the 77 bus

Thursday: Response -> Task List (9:30am)

1- Let‟s make a database!

2 - You can manage it through a website admin!

3 - Sortable column headers!

4 - You can have searching and filters!

5 - We can send personalized emails with RSVP links!!

6 - Just in case, you can download it all back to Excel!

a bit of setup: choose database (9:15am)

In the settings.py file:

DATABASES = {
'default': {
'ENGINE': 'django.db.backends.sqlite3',
# Add 'postgresql_psycopg2', 'postgresql', 'mysql', 'sqlite3' or 'oracle'.
'NAME': '/db/mcb_hcom.db3',
'USER': '',
'PASSWORD': '',
'HOST': '',
'PORT': '',
}
}

a bit of setup: start an “app” (9:20am)

>python manage.py startapp hcom_training

Data models: models.py (still 9:20am)

/hcom_training/
models.py -> SQL tables!
admin.py

a bit of setup: choose database (9:15am)

In the settings.py file:

INSTALLED_APPS = (
#...
'hcom_training',
#...

)

data models: think (9:30 to 9:40)

What would C.J. Date do?

data models: 1st think (9:30 to 9:45)

TrainingSession
iname
Trainee training type
first name training datetime
last name room
email (etc, etc)
location
lab_or_office
special
confirmed_training_date
id_md5 LabOffice
(etc., etc.) name

TraineeMessage SpecialDesignation
trainee name
time_sent
to_address
msg

data models

class Trainee(models.Model):
""" Trainee Information """

fname = models.CharField(„First Name‟, max_length=50)

lname = models.CharField(„Last Name‟, max_length=50)

email= models.EmailField(blank=True)

confirmed_training_date = models.BooleanField(default=False)

location = models.ForeignKey(LocationInfo) # training session

(+ 13 more fields)

data models

class LocationInfo(models.Model):
""" Training Session Information """

name = models.CharField(max_length=255)

training_datetime = models.DateTimeField()

room = models.CharField(max_length=100)

(+2 more fields)

models.py: create your tables (10:15am)

> python manage.py validate
0 errors found
> python manage.py syncdb
Creating tables ...
Creating table hcom_training_special
Creating table hcom_training_laboffice
Creating table hcom_training_locationinfo
Creating table hcom_training_trainee
Creating table hcom_training_traineemessage
Installing custom SQL ...
Installing indexes ...
No fixtures found

models.py

>python manage.py sqlall hcom_training

CREATE TABLE `hcom_training_locationinfo` (
ìd` integer AUTO_INCREMENT NOT NULL PRIMARY KEY,
`name` varchar(255) NOT NULL,
`training_type` varchar(100) NOT NULL,
`training_datetime` datetime NOT NULL,
...

CREATE TABLE `hcom_training_trainee` (
ìd` integer AUTO_INCREMENT NOT NULL PRIMARY KEY,
`fname` varchar(50) NOT NULL,
`lname` varchar(50) NOT NULL,
èmail` varchar(75) NOT NULL,
`confirmed_training_date` bool NOT NULL,
`location_id` integer,
`lab_or_office_id` integer NOT NULL,
...

2 files: data models + admin

/hcom_training/
admin.py -> web-based admin,
sortable columns,
searching and filters

admin.py: 1 line of code (“You can manage it through a website admin!”)

admin.site.register(Trainee)

admin.py: 3 lines of code (“Sortable column headers!”)

class TraineeAdmin(admin.ModelAdmin):
list_display = ('lname', ’fname', 'email', 'confirmed_training_date’,
'completed_training’)
admin.site.register(Trainee, TraineeAdmin)

admin.py: 4 lines of code (“You can have searching and filters!)

search_fields = ('lname', 'fname', 'email', ‘location_room’)

admin.py: 5 lines of code
search_fields = ('lname', 'fname', 'email', ‘location_room’)
list_filter = ('confirmed_training_date', 'completed_training', 'location')

admin.py: 5 lines of code


list_display = ('lname', 'fname', 'email',
'confirmed_training_date', 'completed_training')

search_fields = ('lname', 'fname', 'email',
'location__room')

list_filter = ('confirmed_training_date',
'completed_training', 'location')


2 Files: Models & Admin

../hcom_training/
admin.py -> web-based admin,
sortable columns,
searching and filters

admin.py -> just scratching the surface

list_display = ('lname','fname', 'email', 'active', 'confirmed_training_date','completed_training', 'special',
'approver_or_shopper', 'lab_or_office', 'hands_on_training_date', 'demo_training_date',)

list_editable = ('completed_training', )

search_fields = ('lname', 'fname', 'email',)
list_filter = ('active', 'confirmed_training_date', 'completed_training','has_hands_on_training',
'has_demo_training','special','approver_or_shopper', 'location',)

readonly_fields = ( 'has_hands_on_training', 'has_demo_training', 'last_update', 'id_md5',
'num_emails_sent','confirmation_link' )

inlines = [ TraineeMessageInline, ]

fieldsets = [
('name / email', {'fields': [( 'fname','lname',), 'active', 'email', 'special',
'lab_or_office','last_update', 'num_emails_sent',]})

,('training', {'fields': ['confirmed_training_date', 'completed_training', 'training_order',
'approver_or_shopper', 'location', ('has_hands_on_training', 'hands_on_training_date',)
, ('has_demo_training', 'demo_training_date'), ]})

,('Notes', {'fields': [ 'notes', 'id_md5' ,'confirmation_link' ]})
]

models.py -> shell -> ADD

>python manage.py shell

>>> t = Trainee(fname='Raman', lname='Prasad'
, email='rprasad@fas.harvard.edu')
>>> t.save()

>>> print t.id, t.lname, t.fname, t.email
1457 Prasad Raman rprasad@fas.harvard.edu

>>> t.delete()

models.py -> shell -> QUERY

>>> l = Trainee.objects.filter(completed_training=False,
special__name='Faculty')

>>> l.count()
23

>>> for t in l:
... print t.fname, t.lname

(publicly listing tardy faculty members = bad idea!)

models.py -> shell -> EDIT

>>> new_location = LocationInfo(
name='special faculty training' ,
training_type='Hands-On Shopper'
, training_date='2011-11-15'
, training_time='10:00'
, room='NW 404')

>>> new_location.save()

>>> for t in l:
... t.location = new_location
... t.save()

models.py -> direct database access

>python manage.py dbshell

SQLite version 3.7.6
Enter ".help" for instructions
Enter SQL statements terminated with a ";"
sqlite>.tables
hcom_training_laboffice
hcom_training_locationinfo
hcom_training_special
hcom_training_trainee
hcom_training_traineemessage

models.py -> dumpdata

>python manage.py dumpdata hcom_training --
indent=4
{
"pk": 8,
"model": "hcom_training.locationinfo",
"fields": {
"training_type": "Shopper Demo",
"room": "Biolabs room 1080 (Lecture Hall)",
"training_date": "2011-06-02",
"training_time": "10:00:00",
"week_num": "3",
"name": "Shopper Demo: Thursday June 2nd 10am -12pm @ Biolabs room 1080 (Lecture Hall)"
}
},
{
"pk": 1277,
"model": "hcom_training.trainee",
"fields": {
"demo_training_date": "2011-05-20",
"notes": "",
"completed_training": true,
"lab_or_office": 279,
"confirmed_training_date": true,
"has_demo_training": true,
"last_update": "2011-05-23 10:39:14",
"lname": "Akhmetova",
"approver_or_shopper": "SHOPPER",
"location": 3,
"fname": "Laila",
"active": true,
"training_order": 14,
"has_hands_on_training": false,
"id_md5": "d759175de8ea5b1d9a2660e45554894f",

models.py -> but there‟s more!

>python manage.py dumpdata hcom_training
–-format=xml --indent=4
{<object pk="9" model="hcom_training.locationinfo">
<field type="CharField" name="name">special faculty training</field>
<field type="CharField" name="training_type">shopper</field>
<field type="DateField" name="training_date">2011-11-15</field>
<field type="TimeField" name="training_time">10:00:00</field>
<field type="CharField" name="week_num"></field>
<field type="CharField" name="room"></field>
</object>
<object pk="1277" model="hcom_training.trainee">
<field type="CharField" name="fname">Laila</field>
<field type="CharField" name="lname">Akhmetova</field>
<field type="BooleanField" name="active">True</field>
<field to="hcom_training.special" name="special" rel="ManyToOneRel">356</field>
<field type="CharField" name="email">lakhmet@fas.harvard.edu</field>
<field type="DateField" name="hands_on_training_date"><None></None></field>
<field type="BooleanField" name="has_hands_on_training">False</field>
<field type="DateField" name="demo_training_date">2011-05-20</field>
<field type="BooleanField" name="has_demo_training">True</field>
<field type="BooleanField" name="confirmed_training_date">True</field>
<field type="BooleanField" name="completed_training">True</field>
<field to="hcom_training.locationinfo" name="location" rel="ManyToOneRel">3</field>
<field type="CharField" name="approver_or_shopper">SHOPPER</field>
<field to="hcom_training.laboffice" name="lab_or_office" rel="ManyToOneRel">279</field>
<field type="IntegerField" name="training_order">14</field>
<field type="TextField" name="notes"></field>
<field type="DateTimeField" name="last_update">2011-05-23 10:39:14</field>
<field type="CharField" name="id_md5">d759175de8ea5b1d9a2660e45554894f</field>
</object>

models.py -> loaddata

>python manage.py dumpdata hcom_training >
hcom_2011_1106.json

>python manage.py loaddata hcom_2011_1106.json

Installed 750 object(s) from 1 fixture(s)

Data In/Out

- Web-based Admin

-Python Shell or Python Scripts/Programs

- dumpdata / loaddata (json, xml, yaml)

- database shell / db specific commands (MySQL, sqlite, etc)

All for 2 files!!







personalized emails: message

Dear Raman Prasad,

You have been selected as an individual who will be placing orders via the Harvard Crimson Online
Marketplace or "HCOM" procurement system on behalf of your lab or group.

For REQUIRED Hands-on Shopper training, please save the SAVE THE DATE:

Time: May 18th, 2011 at 10am
Location: Science Center, Rm. 226.

The training will last 2 hours and lunch will be provided.

To RSVP, please click on the following link:

http://mcb.harvard.edu/hcom/confirm/2812e5cf6d8f21d69c91dddeefb792a7/

If for any reason, you are not able to attend this training, please contact me immediately to make
alternate arrangements.

Thank you.

personalized emails: message as a template

Dear {{ trainee.fname }} {{ trainee.lname }},

You have been selected as an individual who will be placing orders via the Harvard Crimson Online
Marketplace or "HCOM" procurement system on behalf of your lab or group.

For REQUIRED Hands-on {{ trainee.training_type }} training, please save the SAVE THE DATE:

Time: {{ trainee.location.training_date|date:"F jS, Y at P" }}
Location: {{ trainee.location.room }}

The training will last 2 hours and lunch will be provided.

To RSVP, please click on the following link:

{% url view_hcom_confirmation trainee.id_md5 %}

If for any reason, you are not able to attend this training, please contact me immediately to make
alternate arrangements.

Thank you.

personalized emails: use the template

for trainee in Trainee.objects.all():

msg = render_to_string('confirmation_email.txt’
, { 'trainee': trainee } )

personalized emails


msg = render_to_string('confirmation_email.txt',
{ 'trainee': trainee } )

subject = 'Required HCOM Training at %s' % trainee.location.name

from_email ='finance_office@mcb.harvard.edu‟
to_addresses = [ trainee.email ]
bcc = ['raman_prasad@harvard.edu']

send_mail(subject, msg, from_email, to_addresses, bcc=bcc)

Source: https://docs.djangoproject.com/en/dev/topics/email/

personalized emails: the link

http://mcb.harvard.edu/hcom/confirm/2812e5cf6d8f21d69c91dddeefb792a7/

urlpatterns = patterns(
'hcom_training.views'

, url(r'^hcom/confirm/(?P<trainee_id>(w){32})/$',
'view_hcom_confirmation'
, name='view_hcom_confirmation')
)

personalized emails: simplified view

def view_hcom_confirmation(request, trainee_id):
"""Simple view showing trainee confirmation page"""

trainee = Trainee.objects.get(id_md5=trainee_id) # get trainee
trainee.confirmed_training_date = True # set confirmed to true
trainee.save() # save the information

# display web page
return render_to_response('hcom_templates/hcom_confirm_page.html'
, { „trainee‟: trainee}
, context_instance=RequestContext(request))

personalized emails: the view

def view_hcom_confirmation(request, trainee_id):
"""Simple view showing trainee confirmation page"""
lu = {}
# Find the trainee; make sure he/she exists
try:
trainee = Trainee.objects.get(id_md5=trainee_id)
lu.update({ 'trainee' : trainee })
except Trainee.DoesNotExist:
lu.update({ 'ERR_MSG' : True, 'ERR_trainee_not_found' : True })
return
render_to_response('hcom_templates/hcom_confirm_page.html', lu, context_instance=RequestCont
ext(request))

# Check if the person has already confirmed
if trainee.confirmed_training_date: # Yes, err message
lu.update({ 'ERR_MSG' : True , 'ERR_already_confirmed' : True })
else:
# New confirmation, update database
trainee.confirmed_training_date = True
trainee.save()
# show confirmation page to use
return
render_to_response('hcom_templates/hcom_confirm_page.html', lu, context_instance=RequestCont
ext(request))

export as Excel

used python xlwt library

style_info_cell = easyxf('pattern: pattern solid, fore_colour white; align: wrap on;')

book = xlwt.Workbook(encoding="utf-8") # make Excel workbook
sheet1 = book.add_sheet('hcom training') # add a sheet

row_num = 2
sheet1.write(row_num, 1, trainee.lname, style_info_cell) # lname
sheet1.write(row_num, 2, trainee.fname, style_info_cell) # fname
sheet1.write(row_num, 3, trainee.email, style_info_cell ) # email
(…)
row_num += 1

Example: https://webapps.sciences.fas.harvard.edu/mcb/mcb-control-panel/hcom_training/

Project Results

- Emails sent out by 3:30/4pm after an hour of Q/A

Time savings

- RSVPs started coming. Database updated as email recipients
clicked link

- 35 people re-scheduled. No need to update shared
spreadsheet. Staff used dropdown box in admin to change
dates and add notes.

- Able to send reminder emails before each training date

Project Results

- Filters used to generate attendance spreadsheets

- Tracked “completed training” via the admin

- System used as new employees come on board

- Home on time

- BUT: Should have been a 1.5 to 2 day project

Spreadsheets -> Databases

- Genome Modification Center

- 46 tables
- 2,389 objects for 500+ projects

- Life Sciences Course Tracker

- 37 Tables
- 1,293 objects including 798 semesters of course data

Spreadsheets -> Databases

Conclusion: Django Models + Admin can take you far….

Spreadsheets -> Databases -> Models

But there‟s more!!

>python manage.py inspectdb <your existing db>

Learning more .. .

Django official site – Work through the Tutorial!!
https://www.djangoproject.com/

Python
http://python.org

Dive into Python
http://www.diveintopython.net/toc/index.html

Contact
raman_prasad@harvard.edu

Django in the Office: Get Your Admin for Nothing and Your SQL for Free

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Django in the Office: Get Your Admin for Nothing and Your SQL for Free

Similar to Django in the Office: Get Your Admin for Nothing and Your SQL for Free (20)

More from Harvard Web Working Group

More from Harvard Web Working Group (20)

Recently uploaded

Recently uploaded (20)

Django in the Office: Get Your Admin for Nothing and Your SQL for Free