Migrate is part of the core starting with Drupal 8. This presentation will teach you the anatomy of a migration from Drupal 6 to Drupal 8.

1. migrating to drupal 8

2. a brief history
drupal.org/upgrade

3. update vs. upgrade
what types drupal “renewing” we have?

4. drupal update

5. drupal update
minor version update
7.24 > 7.25

6. drupal update
minor version update
7.24 > 7.25
drupal upgrade

7. drupal update
minor version update
7.24 > 7.25
drupal upgrade
major version upgrade
7.x > 8.x

8. drupal update
minor version update
7.24 > 7.25
drupal upgrade
major version upgrade
7.x > 8.x

9. the migrate module
“in service” since 2009
drupal.org/project/migrate

10. the migrate module
• How it worked?
• Migrations = classes extending Migration.
• Main elements: source, destination, map,
mappings, “hooks” (prepareRow, prepare,
complete, createStub, etc).
• Each migration has to extend the Migration
class or one of its successors.

12. migrate in D8 core

13. disclaimer

14. disclaimer
• the migrate system is under heavy
development right now.

15. disclaimer
• the migrate system is under heavy
development right now.
• some of the features or APIs may change in
the future

16. disclaimer
• the migrate system is under heavy
development right now.
• some of the features or APIs may change in
the future
• not all the current work is pushed to 8.x.

17. disclaimer
• the migrate system is under heavy
development right now.
• some of the features or APIs may change in
the future
• not all the current work is pushed to 8.x.
• The work is in the sandbox at
https://drupal.org/sandbox/chx/2105305

18. credits
Károly Négyesi (chx)
Mike Ryan (mikeryan)
Moshe Weitzman (moshe weitzman)
Ben Dougherty (benjy)

19. drupal 8 migration

20. note

21. note
• While a significant portion of the code and
the interaction between the elements is
brand new, the actual migrate-y code is
coming straight from D7: highwater marks,
track changes, id map, this is here to

22. note
• While a significant portion of the code and
the interaction between the elements is
brand new, the actual migrate-y code is
coming straight from D7: highwater marks,
track changes, id map, this is here to
• The new interaction allows for really nice
and powerful migrations but at the same
time we are most definitely not reinventing
wheel.

23. structure/modules

24. Migrate
core/modules/migrate/

25. Migrate
core/modules/migrate/
• provides general API for all migrations

26. Migrate
core/modules/migrate/
• provides general API for all migrations
• provides interfaces and base classes for all
migration plugin components (source, destination,
process, id_map, row).

27. Migrate
core/modules/migrate/
• provides general API for all migrations
• provides interfaces and base classes for all
migration plugin components (source, destination,
process, id_map, row).
• provides a plugin manager for manipulation on
migration plugins.

28. Migrate
core/modules/migrate/
• provides general API for all migrations
• provides interfaces and base classes for all
migration plugin components (source, destination,
process, id_map, row).
• provides a plugin manager for manipulation on
migration plugins.
• provides the migrate configurable (configuration
entity type).

29. Migrate Drupal
core/modules/migrate_drupal/

30. Migrate Drupal
core/modules/migrate_drupal/
• the first module using the new Migrate API.

31. Migrate Drupal
core/modules/migrate_drupal/
• the first module using the new Migrate API.
• kind of migrate_d2d successor.

32. Migrate Drupal
core/modules/migrate_drupal/
• the first module using the new Migrate API.
• kind of migrate_d2d successor.
• migrates out-of-the-box from Drupal 6 and 7 into
Drupal 8.

33. Migrate Drupal
core/modules/migrate_drupal/
• the first module using the new Migrate API.
• kind of migrate_d2d successor.
• migrates out-of-the-box from Drupal 6 and 7 into
Drupal 8.
• Defines migrations for all system components:

34. Migrate Drupal
core/modules/migrate_drupal/
• the first module using the new Migrate API.
• kind of migrate_d2d successor.
• migrates out-of-the-box from Drupal 6 and 7 into
Drupal 8.
• Defines migrations for all system components:
• Drupal 6 settings (site name, slogan, roles, etc)

35. Migrate Drupal
core/modules/migrate_drupal/
• the first module using the new Migrate API.
• kind of migrate_d2d successor.
• migrates out-of-the-box from Drupal 6 and 7 into
Drupal 8.
• Defines migrations for all system components:
• Drupal 6 settings (site name, slogan, roles, etc)
• Content definitions (vocabularies, node types, etc)

36. Migrate Drupal
core/modules/migrate_drupal/
• the first module using the new Migrate API.
• kind of migrate_d2d successor.
• migrates out-of-the-box from Drupal 6 and 7 into
Drupal 8.
• Defines migrations for all system components:
• Drupal 6 settings (site name, slogan, roles, etc)
• Content definitions (vocabularies, node types, etc)
• Content (noded, terms, users, etc).

37. understanding migrations

38. migrations are configurables

39. small peek
into configurables

40. what is a configurable?

41. what is a configurable?
• “Configurables” are configuration entities.

42. what is a configurable?
• “Configurables” are configuration entities.
• In Drupal 8 the content is separated from
configuration. Both are classes and are sharing
the same ancestor: the Entity class.

43. what is a configurable?
• “Configurables” are configuration entities.
• In Drupal 8 the content is separated from
configuration. Both are classes and are sharing
the same ancestor: the Entity class.
DrupalCoreEntityEntity

44. what is a configurable?
• “Configurables” are configuration entities.
• In Drupal 8 the content is separated from
configuration. Both are classes and are sharing
the same ancestor: the Entity class.
DrupalCoreEntityContentEntityBase
DrupalCoreEntityEntity

45. what is a configurable?
• “Configurables” are configuration entities.
• In Drupal 8 the content is separated from
configuration. Both are classes and are sharing
the same ancestor: the Entity class.
DrupalCoreEntityContentEntityBase
DrupalCoreEntityEntity
DrupalCoreConfigConfigEntityBase

46. what is a configurable?
• “Configurables” are configuration entities.
• In Drupal 8 the content is separated from
configuration. Both are classes and are sharing
the same ancestor: the Entity class.
DrupalCoreEntityContentEntityBase
DrupalCoreEntityEntity
DrupalCoreConfigConfigEntityBase

47. what is a configurable?
•
•
•
•
A configurable is the way Drupal 8 stores the
configuration of a specific functionality. E.g. the the
definition of a node type is stored in a configuration
entity of type “node_type”.
Configuration entity types are annotated classes, meaning
that the object meta information is stored in annotation
rather than in info hooks - as it was in Drupal <= 7.
Imagine configurables as entities storing their data in
config YAML files rather than DB.
The “fields” of a configurable are the public properties
exposed by the configurable object.

48. what is a configurable?

49. how it’s stored?
example

50. migration plugins
parts implemented by specific migrations

51. source plugins
• plugins returning information and data from
the source of migration.
• usually: the list of fields, the source iterator
(used retrieve data from source).
• each migration should configure a source.

52. destination plugins
• are handling data at the destination: import,
rollback.
• different plugins for different destination
components: entity, config, etc.
• are defined in the base module (migrate) as
destination is always drupal 8 but if necessary it
can be extended.
• each migration should specify a destination.

53. id map plugin
• plugins of this type are handling and storing the
•
•
•
relation between primary IDs of source and
destination.
without this, rollback and continuous migrations are
impossible.
in 99% of the cases you’ll use the sql id map plugin
(Sql) that keeps the map of each migration in a table.
table name migrate_map_MIGRATION_ID

54. processors
• plugins that are performing small but very specialized
operations against values to be migrated.
• Some simple examples: DefaultValue, Concat, etc.
• The most important interface method:
transform().

55. the anatomy of a migration
migrating user roles from a dupal 6 site

56. creating the config file
config/migrate.migration.d6_user_role.yml
relative to core/modules/migrate_drupal

57. config .yml file content

58. config .yml file content
• id: same as the last part of filename (d6_user_role)

59. config .yml file content
• id: same as the last part of filename (d6_user_role)
• sourceIds: Source fields, providing a primary ID.

60. config .yml file content
• id: same as the last part of filename (d6_user_role)
• sourceIds: Source fields, providing a primary ID.
• source: configure the source of data, usually the
source plugin to be used

61. config .yml file content
• id: same as the last part of filename (d6_user_role)
• sourceIds: Source fields, providing a primary ID.
• source: configure the source of data, usually the
source plugin to be used
• process: describe the list of processors to be applied
per destination field.

62. config .yml file content
• id: same as the last part of filename (d6_user_role)
• sourceIds: Source fields, providing a primary ID.
• source: configure the source of data, usually the
source plugin to be used
• process: describe the list of processors to be applied
per destination field.
• destination: destination configuration, usually the
destination plugin.

63. id

64. id
• this is the configurable unique id.

65. id
• this is the configurable unique id.
• it must be exactly as the same as the last
part of filename: d6_user_role.

66. id
• this is the configurable unique id.
• it must be exactly as the same as the last
part of filename: d6_user_role.
id: d6_user_role

67. sourceIds

68. sourceIds
• look in D6 schema to find the role primary ID.

69. sourceIds
• look in D6 schema to find the role primary ID.
• lines 107 - 115 of drupal/modules/user/user.install.

70. sourceIds
• look in D6 schema to find the role primary ID.
• lines 107 - 115 of drupal/modules/user/user.install.
$schema['role'] = array(
'description' => 'Stores user roles.',
'fields' => array(
'rid' => array(
'type' => 'serial',
'unsigned' => TRUE,
'not null' => TRUE,
'description' => 'Primary Key: Unique role id.',

71. sourceIds

72. sourceIds
• use TypedData identifiers for data type.

73. sourceIds
• use TypedData identifiers for data type.
• Here are the .yml lines that we need to add.

74. sourceIds
• use TypedData identifiers for data type.
• Here are the .yml lines that we need to add.
sourceIds:
rid:
type: integer

75. sourceIds
• use TypedData identifiers for data type.
• Here are the .yml lines that we need to add.
sourceIds:
rid:
type: integer
Note: sourceIds will be removed in the near future
and the source plugin will set also the primary id.

76. source

77. source
• we need to implement a source plugin first, that
provides the list of fields and the iterator by
querying the D6 backend.

78. source
• we need to implement a source plugin first, that
provides the list of fields and the iterator by
querying the D6 backend.
• let’s see how it should look (code).

79. source
• we need to implement a source plugin first, that
provides the list of fields and the iterator by
querying the D6 backend.
• let’s see how it should look (code).
• add the source plugin id in the configuration .yml file.

80. source
• we need to implement a source plugin first, that
provides the list of fields and the iterator by
querying the D6 backend.
• let’s see how it should look (code).
• add the source plugin id in the configuration .yml file.
source:
plugin: drupal6_user_role

81. process

82. process
• process keys are destination “fields”.

83. process
• process keys are destination “fields”.
• for configurables: the public properties (except uuid)

84. process
• process keys are destination “fields”.
• for configurables: the public properties (except uuid)
• for content: the keys from
baseFieldDefinitions

85. process
• process keys are destination “fields”.
• for configurables: the public properties (except uuid)
• for content: the keys from
baseFieldDefinitions
• let’s see how it looks! (code).

86. process
• process keys are destination “fields”.
• for configurables: the public properties (except uuid)
• for content: the keys from
baseFieldDefinitions
• let’s see how it looks! (code).
process:
id:
label:
weight:
permissions:

87. destination

88. destination
• should point to the destination plugin.

89. destination
• should point to the destination plugin.
• in this case we’re importing into user_role entity,
so we’re passing also the entity_type argument.

90. destination
• should point to the destination plugin.
• in this case we’re importing into user_role entity,
so we’re passing also the entity_type argument.
destination:
plugin: entity
entity_type: user_role

91. running a migration
• via drush
• There will be a brief UI implemented in
core (to come!)

92. final notes

93. final notes
• Minor version updates are unchanged. Developers
continue to use hook_update_N() for those.

94. final notes
• Minor version updates are unchanged. Developers
continue to use hook_update_N() for those.
• Contrib and custom modules are encouraged to ship
with migrations of their data from D6/D7 to D8. Use
core modules as model.

95. final notes
• Minor version updates are unchanged. Developers
continue to use hook_update_N() for those.
• Contrib and custom modules are encouraged to ship
with migrations of their data from D6/D7 to D8. Use
core modules as model.
• The underlying Migrate API is source-agnostic.You
can easily migrate into D8 from MS SQL, Oracle, piles
of HTML files, XML feeds, CSV files, etc.

96. final notes
• Minor version updates are unchanged. Developers
continue to use hook_update_N() for those.
• Contrib and custom modules are encouraged to ship
with migrations of their data from D6/D7 to D8. Use
core modules as model.
• The underlying Migrate API is source-agnostic.You
can easily migrate into D8 from MS SQL, Oracle, piles
of HTML files, XML feeds, CSV files, etc.
• Similarly, Drupal 4.x and Drupal 5.x sites are able to
migrate using this same approach.

97. Questions?
Thank you.