10. the migrate module
• How it worked?
• Migrations = classes extending Migration.
• Main elements: source, destination, map,
mappings, “hooks” (prepareRow, prepare,
complete, createStub, etc).
• Each migration has to extend the Migration
class or one of its successors.
15. disclaimer
• the migrate system is under heavy
development right now.
• some of the features or APIs may change in
the future
16. disclaimer
• the migrate system is under heavy
development right now.
• some of the features or APIs may change in
the future
• not all the current work is pushed to 8.x.
17. disclaimer
• the migrate system is under heavy
development right now.
• some of the features or APIs may change in
the future
• not all the current work is pushed to 8.x.
• The work is in the sandbox at
https://drupal.org/sandbox/chx/2105305
21. note
• While a significant portion of the code and
the interaction between the elements is
brand new, the actual migrate-y code is
coming straight from D7: highwater marks,
track changes, id map, this is here to
22. note
• While a significant portion of the code and
the interaction between the elements is
brand new, the actual migrate-y code is
coming straight from D7: highwater marks,
track changes, id map, this is here to
• The new interaction allows for really nice
and powerful migrations but at the same
time we are most definitely not reinventing
wheel.
27. Migrate
core/modules/migrate/
• provides general API for all migrations
• provides interfaces and base classes for all
migration plugin components (source, destination,
process, id_map, row).
• provides a plugin manager for manipulation on
migration plugins.
28. Migrate
core/modules/migrate/
• provides general API for all migrations
• provides interfaces and base classes for all
migration plugin components (source, destination,
process, id_map, row).
• provides a plugin manager for manipulation on
migration plugins.
• provides the migrate configurable (configuration
entity type).
33. Migrate Drupal
core/modules/migrate_drupal/
• the first module using the new Migrate API.
• kind of migrate_d2d successor.
• migrates out-of-the-box from Drupal 6 and 7 into
Drupal 8.
• Defines migrations for all system components:
34. Migrate Drupal
core/modules/migrate_drupal/
• the first module using the new Migrate API.
• kind of migrate_d2d successor.
• migrates out-of-the-box from Drupal 6 and 7 into
Drupal 8.
• Defines migrations for all system components:
• Drupal 6 settings (site name, slogan, roles, etc)
35. Migrate Drupal
core/modules/migrate_drupal/
• the first module using the new Migrate API.
• kind of migrate_d2d successor.
• migrates out-of-the-box from Drupal 6 and 7 into
Drupal 8.
• Defines migrations for all system components:
• Drupal 6 settings (site name, slogan, roles, etc)
• Content definitions (vocabularies, node types, etc)
36. Migrate Drupal
core/modules/migrate_drupal/
• the first module using the new Migrate API.
• kind of migrate_d2d successor.
• migrates out-of-the-box from Drupal 6 and 7 into
Drupal 8.
• Defines migrations for all system components:
• Drupal 6 settings (site name, slogan, roles, etc)
• Content definitions (vocabularies, node types, etc)
• Content (noded, terms, users, etc).
41. what is a configurable?
• “Configurables” are migrations entities.
42. what is a configurable?
• “Configurables” are migrations entities.
• In Drupal 8 the content is separated from
configuration. Both are classes and are sharing
the same ancestor: the Entity class.
43. what is a configurable?
• “Configurables” are migrations entities.
• In Drupal 8 the content is separated from
configuration. Both are classes and are sharing
the same ancestor: the Entity class.
DrupalCoreEntityEntity
44. what is a configurable?
• “Configurables” are migrations entities.
• In Drupal 8 the content is separated from
configuration. Both are classes and are sharing
the same ancestor: the Entity class.
DrupalCoreEntityContentEntityBase
DrupalCoreEntityEntity
45. what is a configurable?
• “Configurables” are migrations entities.
• In Drupal 8 the content is separated from
configuration. Both are classes and are sharing
the same ancestor: the Entity class.
DrupalCoreEntityContentEntityBase
DrupalCoreEntityEntity
DrupalCoreConfigConfigEntityBase
46. what is a configurable?
• “Configurables” are migrations entities.
• In Drupal 8 the content is separated from
configuration. Both are classes and are sharing
the same ancestor: the Entity class.
DrupalCoreEntityContentEntityBase
DrupalCoreEntityEntity
DrupalCoreConfigConfigEntityBase
47. what is a configurable?
•
•
•
•
A configurable is the way Drupal 8 stores the
configuration of a specific functionality. E.g. the the
definition of a node type is stored in a configuration
entity of type “node_type”.
Configuration entity types are annotated classes, meaning
that the object meta information is stored in annotation
rather than in info hooks - as it was in Drupal <= 7.
Imagine configurables as entities storing their data in
config YAML files rather than DB.
The “fields” of a configurable are the public properties
exposed by the configurable object.
51. source plugins
• plugins returning information and data from
the source of migration.
• usually: the list of fields, the source iterator
(used retrieve data from source).
• each migration should configure a source.
52. destination plugins
• are handling data at the destination: import,
rollback.
• different plugins for different destination
components: entity, config, etc.
• are defined in the base module (migrate) as
destination is always drupal 8 but if necessary it
can be extended.
• each migration should specify a destination.
53. id map plugin
• plugins of this type are handling and storing the
•
•
•
relation between primary IDs of source and
destination.
without this, rollback and continuous migrations are
impossible.
in 99% of the cases you’ll use the sql id map plugin
(Sql) that keeps the map of each migration in a table.
table name migrate_map_MIGRATION_ID
54. processors
• plugins that are performing small but very specialized
operations against values to be migrated.
• Some simple examples: DefaultValue, Concat, etc.
• The most important interface method:
transform().
55. the anatomy of a migration
migrating user roles from a dupal 6 site
56. creating the config file
config/migrate.migration.d6_user_role.yml
relative to core/modules/migrate_drupal
58. config .yml file content
• id: same as the last part of filename (d6_user_role)
59. config .yml file content
• id: same as the last part of filename (d6_user_role)
• sourceIds: Source fields, providing a primary ID.
60. config .yml file content
• id: same as the last part of filename (d6_user_role)
• sourceIds: Source fields, providing a primary ID.
• source: configure the source of data, usually the
source plugin to be used
61. config .yml file content
• id: same as the last part of filename (d6_user_role)
• sourceIds: Source fields, providing a primary ID.
• source: configure the source of data, usually the
source plugin to be used
• process: describe the list of processors to be applied
per destination field.
62. config .yml file content
• id: same as the last part of filename (d6_user_role)
• sourceIds: Source fields, providing a primary ID.
• source: configure the source of data, usually the
source plugin to be used
• process: describe the list of processors to be applied
per destination field.
• destination: destination configuration, usually the
destination plugin.
74. sourceIds
• use TypedData identifiers for data type.
• Here are the .yml lines that we need to add.
sourceIds:
rid:
type: integer
75. sourceIds
• use TypedData identifiers for data type.
• Here are the .yml lines that we need to add.
sourceIds:
rid:
type: integer
Note: sourceIds will be removed in the near future
and the source plugin will set also the primary id.
77. source
• we need to implement a source plugin first, that
provides the list of fields and the iterator by
querying the D6 backend.
78. source
• we need to implement a source plugin first, that
provides the list of fields and the iterator by
querying the D6 backend.
• let’s see how it should look (code).
79. source
• we need to implement a source plugin first, that
provides the list of fields and the iterator by
querying the D6 backend.
• let’s see how it should look (code).
• add the source plugin id in the configuration .yml file.
80. source
• we need to implement a source plugin first, that
provides the list of fields and the iterator by
querying the D6 backend.
• let’s see how it should look (code).
• add the source plugin id in the configuration .yml file.
source:
plugin: drupal6_user_role
83. process
• process keys are destination “fields”.
• for configurables: the public properties (except uuid)
84. process
• process keys are destination “fields”.
• for configurables: the public properties (except uuid)
• for content: the keys from
baseFieldDefinitions
85. process
• process keys are destination “fields”.
• for configurables: the public properties (except uuid)
• for content: the keys from
baseFieldDefinitions
• let’s see how it looks! (code).
86. process
• process keys are destination “fields”.
• for configurables: the public properties (except uuid)
• for content: the keys from
baseFieldDefinitions
• let’s see how it looks! (code).
process:
id:
label:
weight:
permissions:
89. destination
• should point to the destination plugin.
• in this case we’re importing into user_role entity,
so we’re passing also the entity_type argument.
90. destination
• should point to the destination plugin.
• in this case we’re importing into user_role entity,
so we’re passing also the entity_type argument.
destination:
plugin: entity
entity_type: user_role
91. running a migration
• via drush
• There will be a brief UI implemented in
core (to come!)
93. final notes
• Minor version updates are unchanged. Developers
continue to use hook_update_N() for those.
94. final notes
• Minor version updates are unchanged. Developers
continue to use hook_update_N() for those.
• Contrib and custom modules are encouraged to ship
with migrations of their data from D6/D7 to D8. Use
core modules as model.
95. final notes
• Minor version updates are unchanged. Developers
continue to use hook_update_N() for those.
• Contrib and custom modules are encouraged to ship
with migrations of their data from D6/D7 to D8. Use
core modules as model.
• The underlying Migrate API is source-agnostic.You
can easily migrate into D8 from MS SQL, Oracle, piles
of HTML files, XML feeds, CSV files, etc.
96. final notes
• Minor version updates are unchanged. Developers
continue to use hook_update_N() for those.
• Contrib and custom modules are encouraged to ship
with migrations of their data from D6/D7 to D8. Use
core modules as model.
• The underlying Migrate API is source-agnostic.You
can easily migrate into D8 from MS SQL, Oracle, piles
of HTML files, XML feeds, CSV files, etc.
• Similarly, Drupal 4.x and Drupal 5.x sites are able to
migrate using this same approach.
