You\'ve decided to make the switch to Drupal. Fantastic! Only one problem: you have to figure out how to move your content from the old database to Drupal. Although there are many import/export modules available it\'s sometimes good to know what\'s happening behind the scenes. This session will walk you through my adventures of porting community Web sites into Drupal. The talk will include: * exporting usable data from your old site; * using CCK to create the right home for your new content; * using existing import modules (specifically: import html, node import and user import); and * importing content \"by hand\" using MySQL command line magic. We will also touch on some of the headaches I ran into in keeping data synchronized on very active community sites during the development phase. This session is perfect for people who are preparing to migrate their Web site to Drupal and also people who are new to database management but want to know more about how things work behind the scenes. Of course if you\'re already a pro at data migration, please bring your stories and suggestions!

1. Moving In
how to port your content from * to drupal
emma jane hogbin

2. the sock economy
drupal.org/user/1773
photo by: morten.dk

3. there are three steps to success
on (drupal) moving day...

4. 1. start with a
fresh drupal installation
photo: julishannon

5. 2. be neat and tidy
to avoid disaster later
photo by: bringo

6. 3. aim to have happy users
at all stages of the move
photo by: waderockett

7. What does it mean to move?
1. Assessing the databases
2. The Export
3. Preparing Drupal for the new content (CCK)
4. Import modules
5. Importing content “by hand”

8. 1. Assessing the databases

9. examine your database
­ what is the db software?
­ is there a tool to export content?
­ what formats can you export content to?
­ how is your content stored in the database?
­ do you need to move user accounts as well?

10. examine drupal’s db
­ what content type(s) are you using?
­ what are the tables you need to place content into?

11. `
Using MySQL
Command Line Magic

12. Drupal 5 tables

13. mysql> describe node;
+­­­­­­­­­­+­­­­­­­­­­­­­­­­­­+­­­­­­+­­­­­+­­­­­­­­­+­­­­­­­­­­­­­­­­+
| Field | Type | Null | Key | Default | Extra |
+­­­­­­­­­­+­­­­­­­­­­­­­­­­­­+­­­­­­+­­­­­+­­­­­­­­­+­­­­­­­­­­­­­­­­+
| nid | int(10) unsigned | NO | PRI | NULL | auto_increment |
| vid | int(10) unsigned | NO | PRI | 0 | |
| type | varchar(32) | NO | MUL | | |
| title | varchar(128) | NO | MUL | | |
| uid | int(11) | NO | MUL | 0 | |
| status | int(11) | NO | MUL | 1 | |
| created | int(11) | NO | MUL | 0 | |
| changed | int(11) | NO | MUL | 0 | |
| comment | int(11) | NO | | 0 | |
| promote | int(11) | NO | MUL | 0 | |
| moderate | int(11) | NO | MUL | 0 | |
| sticky | int(11) | NO | | 0 | |
+­­­­­­­­­­+­­­­­­­­­­­­­­­­­­+­­­­­­+­­­­­+­­­­­­­­­+­­­­­­­­­­­­­­­­+
12 rows in set (0.00 sec)

14. mysql> describe node_revisions;
+­­­­­­­­­­­+­­­­­­­­­­­­­­­­­­+­­­­­­+­­­­­+­­­­­­­­­+­­­­­­­+
| Field | Type | Null | Key | Default | Extra |
+­­­­­­­­­­­+­­­­­­­­­­­­­­­­­­+­­­­­­+­­­­­+­­­­­­­­­+­­­­­­­+
| nid | int(10) unsigned | NO | MUL | | |
| vid | int(10) unsigned | NO | PRI | | |
| uid | int(11) | NO | MUL | 0 | |
| title | varchar(128) | NO | | | |
| body | longtext | NO | | | |
| teaser | longtext | NO | | | |
| log | longtext | NO | | | |
| timestamp | int(11) | NO | | 0 | |
| format | int(11) | NO | | 0 | |
+­­­­­­­­­­­+­­­­­­­­­­­­­­­­­­+­­­­­­+­­­­­+­­­­­­­­­+­­­­­­­+
9 rows in set (0.00 sec)

15. plan your move!
­ write yourself notes describing the process
­ cross off the steps as you complete them

16. 2. Exporting useful data

17. export tools
­ every database has its own export tool
­ if you are exporting from MySQL use mysqldump
­ export to CSV where possible
(comma separated values)

18. exporting content: the dump
http://dev.mysql.com/doc/refman/5.0/en/mysqldump.html
You can also use PHPMyAdmin if it’s installed!
MySQL to MySQL
ejhogbin@bilbo:~$ mysqldump ­u root ­p [dbname]
> oldsite.sql
MySQL to CSV (local machine only)
ejhogbin@bilbo:~$ mysqldump ­u root ­p
­­fields­terminated­by=,
­­tab=[outputdirectory]
­­tables [dbname] [tablename] > table.txt
Import into MySQL
ejhogbin@bilbo:~$ mysql ­u root ­p [newdatabase]
< oldsite.sql

19. 3. Preparing Drupal
for the new content (CCK)

20. Installing CCK
­ modules (CCK, image, date, etc)
­ creating content types

21. 4. Import modules

22. Drupal import tools
http://drupal.org/project/Modules/category/64
­ html_import, node_import and user_import
­ importing from WordPress? I used wp2drupal (4.7); try wordpress_import

23. import_html
­ modules: path, import_html
­ in Ubuntu: sudo apt­get install php5­xsl
(yes, even though it says XML support is enabled in the phpinfo() test file)
­ review the settings (keep the defaults)
­ copy the site to a local directory
(use wget to generate a flat site if necessary)
­ proceed with the import
­ set the path to the local directory where your static site lives
­ check “more settings” to set the user and default settings for each node
­ play with XSL template file if necessary (and you feel like an XML pro)
­ click “next” and choose the files to import
­ click import
­ check for errors
­ enjoy your imported content

24. start page: import_html

25. choose what to import

26. choose the files to import

27. success!
(I’m actually overwriting the initial import on this second attempt.)

28. node_import
­ modules: node_import
­ prepare your CSV file to import
­ prepare your Drupal installation to receive the imported data
­ proceed to: admin/content/node_import
­ choose the CSV file that contains the data
­ choose your content type (e.g. page or story)
­ match the data in your CSV file to data
­ run the import
­ node_import runs on cron and may take
several hours for very large amounts of content

29. upload your CSV file

30. match data to drupal fields

31. set node author

32. user_import
­ basically the same as node_import
­ upload your CSV file and match fields onto drupal’s database
­ user profile data must be uploaded separately into nodes
(ask me later, we’re probably running short on time)

33. passwords: oh the headaches
­ if you’re lucky they’re stored as plain text in the other db
­ if they’re encrypted hopefully it’s with MySQL’s md5() function
­ drupal stores passwords with the md5() mysql function
­ back up the user table and convert passwords from plain text to md5()
mysql> create table users2 select * from users;
mysql> update users2 set pass = md5(pass);

34. 5. Importing content “by hand”

35. importing plain vanilla content
1. import the data into temporary tables
2. using the insert/select command copy the data into
node and node_revisions
3. update the sequences table to show the next nid

36. importing CSV data
mysql > create table tmpdata
(tmpid int, key1 int, key2 int);
mysql > load data local infile
'/home/emmajane/data/data.csv'
into table tmpdata
fields terminated by ','
optionally enclosed by 'quot;'
lines terminated by 'n'
(tmpid, key1, key2);

37. create “story” node stubs
mysql> insert ignore
node (nid, vid, uid, title, type)
select tmpdata.nid, tmpdata.nid,
tmpdata.uid, tmpdata.pagetitle,
quot;storyquot;
from tmpdata;

38. insert the body/description into
node_revisions
mysql> insert ignore
node_revisions (nid, vid, uid, title,
body, timestamp, format)
select tmpdata.nid, tmpdata.nid,
tmpdata.uid, tmpdata.pagetitle,
tmpdata.pagecontent,
unix_timestamp(now()), 1
from tmpdata;

39. update sequences (v5)
mysql> select nid from node
order by nid DESC limit 1;
+­­­­­­­+
| nid |
+­­­­­­­+
| 18325 |
+­­­­­­­+
mysql> update sequences
set id = 18325
where name = “node_nid”;
mysql> update sequences
set id = 18325
where name = “node_revisions_vid”;

40. SRSLY! Do not forget!
Bad things happen when you
fail to update sequences.
photo by: croma

41. importing special content
1. import the data into temporary tables
2. using the insert/select command copy the data into
node and node_revisions
3. copy special content into CCK tables
4. update the sequences table to show the next nid

42. CCK content
­ use Drupal to create CCK tables
­ examine the structure
­ copy your content into the new tables

43. cck tables
mysql> show tables;
| content_type_ad |
| content_type_faq |
| content_type_forum |
| content_type_page |
| content_type_pattern |
| content_type_quotes |
| content_type_story |

44. cck tables
mysql> describe content_type_pattern;
+­­­­­­­­­­­­­­­­­­­­­­­­­­­+­­­­­­­­­­­­­­­­­­+­­­­­­+­­­­­+­­­­­­­­­+­­­­­­­+
| Field | Type | Null | Key | Default | Extra |
+­­­­­­­­­­­­­­­­­­­­­­­­­­­+­­­­­­­­­­­­­­­­­­+­­­­­­+­­­­­+­­­­­­­­­+­­­­­­­+
| vid | int(10) unsigned | NO | PRI | 0 | |
| nid | int(10) unsigned | NO | | 0 | |
| field_byline_value | longtext | YES | | NULL | |
| field_intro_value | longtext | YES | | NULL | |
| field_intro_format | int(10) unsigned | NO | | 0 | |
| field_materials_value | longtext | YES | | NULL | |
| field_finished_size_value | longtext | YES | | NULL | |
| field_gauge_value | longtext | YES | | NULL | |
| field_notes_value | longtext | YES | | NULL | |
| field_notes_format | int(10) unsigned | NO | | 0 | |
| field_contributor_nid_uid | int(11) | YES | | NULL | |
+­­­­­­­­­­­­­­­­­­­­­­­­­­­+­­­­­­­­­­­­­­­­­­+­­­­­­+­­­­­+­­­­­­­­­+­­­­­­­+

45. Moving Content Summary
1. Assess the situation and plan your move
2. Export content to a useful format
3. Create the new home for your content
4. Use the import modules OR
5. Importing content “by hand”