1. BUILD AND DEPLOYMENT
MANAGEMENT

2. PURPOSE OF CURRENT
TRAINING
Why do we need to manage builds
Theory and practice of build and deployment
management
Why do we need to manage deployments
2

3. CONTENTS
 Build management
 What is build?
 Why do we need to build?
 Build types.
 Tools and their specifics: Ant, Maven, make, etc
 Tips
 Deployment management
 Build vs deployment
 Why do we need to deploy?
 How to deploy and when?
 Tools that might be useful 3

4. MAIN CONCEPTS
4

5. WHAT IS BUILD
From Process To
of conversion
run at the end-user
Source code Artifacts workstation
5

6. WHY DO WE NEED TO BUILD?
Build
RESULT OF
Configuration
 defined set of components having specific versions
 components = functional units
 components and their versions are chosen to meet
specific objectives or tasks
Main objectives
6
Functionality Performance

7. WHY DO WE NEED TO BUILD?
Conclusion. Why we need to build is:
 To incorporate additional functionality (feature)
 To reach defined level of performance (as an
example of non-functional requirement)
And what about data?
 Data has its own lifecycle
 Which is incorporated into the database integration
process
7

8. WHY DO WE NEED TO MANAGE
BUILDS?
 When application grows in size, it becomes more
complex
 As a result, it turns out that routine daily activities require
additional management
 Organizing build process is one of such activities
 Therefore, build process also requires management
 What means complex application?
 From configuration management point of view, most
complex application incorporate all possible variant 8
properties

9. BUILD VS VARIANT
Build:
install.msi
Variant:
AppName-1.1.8.en.x32_beta.msi
Real world example:
HTC_Touch_Diamond2_WM6.5_RUU_Topaz_S_HT
C_RUS_2.16.411.2_Radio_Sign_Topaz_61.44
tc.25.32_4.49.25.17_Ship.exe 9

10. CONNECTION BETWEEN
CONFIGURATION AND VARIANTS
Functional requirements for the variant being built
#1 #2 #3 … #N
version Maturity
Platform
variant OS
Localization
build Build type
configuration + source code changes
(changeset) 10
static runtime

11. WHAT IS VARIANT
Variant is the result of the build
characterized by the set of specific
properties and their values
11

12. VARIANT PROPERTIES
 Version
 Revision
 Date
 Build type: debug or release
 Operating system, hardware platform (x32, x64, …)
 Localization (en, ru, ua, …)
 Platform (development, test, integration, production)
 Maturity (alpha, beta, release candidate, stable)
 Issue number (fix#387)
 License (MIT, GPL, BSD, CC, …)
 Project ID (EPM-SIGN) + Discipline ID (PD) 12

13. EXAMPLE.
PLAIN DOCUMENT Build #2
Properties:
• Language (en)
• Date (24-12-2010)
• Revision (4)
Ctrl + N Ctrl + S Ctrl + S Ctrl + S
Build #1
Properties:
13
• Language (en)
• Date (22-12-2010)
• Revision (3)

14. DEMO EXAMPLE
14

15. EMERGED QUESTIONS
 Why open source repositories are not structured
properly?
 How to assign unique marker (version) for our
changes?
 What repository model should we use?
 How do we need to manage experimental changes
or development?
 What is initial codebase?
15

16. BUILD MANAGEMENT
16

17. WHY DO WE NEED TO MANAGE
BUILDS?
 Builds should be CRISPy!
 CRISP means:
 Complete
 Repeatable
 Informative
 Schedulable
17
 Portable

18. HOW TO MANAGE BUILD
PROCESS?
Define what build tasks you need
Define what build types you need
Choose build phases you will need
Choose build tool
Structure your build file 18

19. HOW TO MANAGE BUILD
PROCESS?
Define what build tasks you need
Define what build types you need
Choose build phases you will need
Choose build tool
Structure your build file 19

20. BUILD TASKS BY PRIORITY
Compilation
Deployment
 DB integration
 Unit testing
 Code coverage
 Static analysis
 Source code metrics
 Dynamic analysis
 Documentation generation
 … 20

21. LET’S TALK ABOUT DIFFERENCES
BETWEEN BUILDS AND
DEPLOYMENTS
21

22. BUILD VS DEPLOYMENT
 Deployment is the process of software installation
(software system available to use)
 While build is the process of installation artifacts creation
 Is this always true?
 Not really
 For interpreted languages (no compilation phase) there might
be no significant difference between build and
deployment
 But it is more convenient to think of all related activities
(unit-testing, code coverage, inspections, etc) as a build, not
deployment.
22

23. RESULTS OF THE BUILD
 Executables
 .exe, .bin, .msi, .cab, …
 Libraries
 .dll, .lib, .so, …
 Archives
 .zip, .tar.gz, .bz2, .7z, .ear, .jar, .war, .sar, …
 Packages
 .rpm, .deb, … 23

24. RESULTS OF DEPLOYMENT
 Installed
application
 Updated database
 Added incremental changes
 Unpackaged library
 Loading and installing dependencies
…
24

25. BUILD VS DEPLOYMENT
Build Deployment
 Runs on single machine  Might be executed on
several machines
 Does not require remote  Requires connection to
connection the remote server
 Creates application  Installs application
installation
 Makes sense always  Makes sense only in
case of distributed or
remote application
25
 Primary  Secondary

26. HOW TO MANAGE BUILD
PROCESS?
Define what build tasks you need
Define what build types you need
Choose build phases you will need
Choose build tool
Structure your build file 28

27. BUILD TYPES
CLASSIFICATION BY:
Use
•Private (developers build)
•Integration (centralized build)
•Release (delivery to the end-user)
29

28. BUILD TYPES
CLASSIFICATION BY:
Structure, stage, result
• Pre-build (ensuring zero build errors: checking
space, clean, …)
• Post-build (build verification tests, security
tests, …)
• Clean (rebuilding whole project from scratch)
• Incremental (only changed files and artifacts)
• Broken 30

29. BUILD TYPES
CLASSIFICATION BY:
Staging
• Lightweight (recompilation, unit-
tests, …)
• Heavyweight (deployment, system
tests, inspections, …)
31

30. HOW TO MANAGE BUILD
PROCESS?
Define what build tasks you need
Define what build types you need
Choose build phases you will need
Choose build tool
Structure your build file 32

31. BUILD PHASES
 Pre-build
 Clean
 Init
 Check disk space
 Build
 Compilation
 Deployment  Post-build
 Inspections  Documentation generation
 Checking build errors
 Security tests
 Tagging
33

32. HOW TO MANAGE BUILD
PROCESS?
Define what build tasks you need
Define what build types you need
Choose build phases you will need
Choose build tool
Structure your build file 34

33. BUILD TOOLS
GNU Autotools Autotools alternatives
• make • pymake
• automake • CMake
• autoconf • Cons
• autoheader • SCons
• libtool • qmake
• gettext • makepp
• gcc • JAM
• pkg-config • waf
35

34. BUILD TOOLS
Java based
• Ant
• Maven
• Gant
.NET based
• MSBuild
• Nant
• Byldan
• NMaven
Other
• Phing (PHP)
36
• Rake (Ruby)

35. GNU AUTOTOOLS
 Did you build applications from sources on UNIX
platform?
 What steps did you need to accomplish this?
1. ./configure
2. make
3. make install
 When does actual compilation happens?
 Right. During make command execution
 make is the father of all build tools
 Heart of the build with make is the makefile
38

36. GNU AUTOTOOLS
MAKEFILE EXAMPLE
target1 target2 target3 : prerequisite1 prerequisite2
command1
command2
command3
myprogram: main.o part1.o part2.o
gcc -o myprogram main.o part1.o part2.o
part1.o: part1.c part1.h header.h
gcc -O -c part1.c
part2.o: part2.c header.h
gcc -O -c part2.c
main.o: main.c header.h
gcc -O -c main.c
39
clean:
rm -f myprogram main.o part1.o part2.o

37. GNU AUTOTOOLS LIST
 autoconf
 creates a configuration script for a package from a
template file
 From Makefile.in to Makefile
 automake
 Creates template file from another template file
 From Makefile.am to Makefile.in
 libtool
 helps manage the creation of static and dynamic
libraries
 gettext
internationalization and localization library
 gcc (GNU compiler collection)
40
 default compiler used for building from sources

38. APACHE ANT
 Implemented in Java
 For the purpose of building java applications
 And for the purpose of make replacement
Advantages over make:
 Cross-platform, does not require buildfile generation
 Resolves circular dependencies
 Works with file hierarchies
 Easily determines outdated files
 Conforms to “java way of thinking”
41

39. APACHE ANT BUILDFILE
EXAMPLE
42

40. APACHE MAVEN
Principles:
 Convention over configuration
 Declarative execution
 Reuse of build logic
 Coherent organization of
dependencies
Features:
 Project Object Model
 Extensive plugin architecture
 Predefined build lifecycle
43
 Dependencies management

41. APACHE MAVEN BUILDFILE
EXAMPLE
44

42. MAVEN VS ANT
Maven Ant
 Description of project  Development of a build
(convention over configuration) script per project
 Invocation of defined goals  Invocation of project
(targets) specific targets
 Project knowledge  "Just" the build process
 build lifecycle, standard  too complex scripts
project layout
 reusable  scripts are not reusable
plugins, repositories
45

43. HOW TO MANAGE BUILD
PROCESS?
Define what build tasks you need
Define what build types you need
Choose build phases you will need
Choose build tool
Structure your build file 46

44. BUILD FILE STRUCTURE
Tasks Targets
 compile
Compilation
 deploy
Deployment  deploy-db
 DB integration  run-tests
 Unit testing  check-coverage
 Code coverage  run-inspections
 Static analysis  gather-metrics
 Source code metrics  performance-tests
 Dynamic analysis
 generatedoc 47
 Documentation generation
 …  …

45. BUILD FILE TEMPLATE
48

46. BUILD TYPES AND BUILDFILE
STRUCTURE
Use types: private, integration, release
49

47. LOCAL BUILD VS PRIVATE
BUILD?
 Local build: What is the difference?
 Compilation Omitting unnecessary steps:
 Unit-testing Compilation
Deployment
 Private build: Database integration
 Compilation Loading initial data
 Deployment Unit-testing
 Database integration Simple static analysis
 Loading initial data
 Unit-testing 50
 Simple static analysis

48. LOCAL BUILD
Local build is required after minor source code changes for
the purpose of having executable artefact
Compilation
Deployment
Database integration
Loading initial data
Unit-testing
Simple static analysis
51
ant compile run-tests

49. PRIVATE BUILD
Private build assumes that changes should be deployed to
the application container or web-server
Compilation
Deployment
Database integration
Loading initial data
Unit-testing
Simple static analysis
ant compile deploy deploy-db load- 52
data run-tests inspections

50. PRIVATE BUILD
53

51. INTEGRATION BUILD
Integration build assumes presence of continuous
integration practice as a key part of software lifecycle
Compilation
Deployment
Database integration
Loading initial data
Unit-testing
Static analysis
Dynamic analysis
54
Documentation generation
Gathering metrics

52. INTEGRATION BUILD
55

53. RELEASE BUILD
Release build is the result of development will be visible to
the end user
Compilation
Deployment
Database integration
Loading initial data
Security tests
integrity checks
56
performance tests

54. RELEASE BUILD
57

55. DEPLOYMENT MANAGEMENT
58

56. DEPLOYMENT TYPES
 Static
 when the container starts
 Hot
 Ability to deploy/undeploy deployables into a running
container
 Incremental
 Deploying only changed files/artifacts
 Remote
 Requires connection via specified protocol
 Local
59
 Does not require connection

57. DEPLOYMENT PROTOCOLS
FTP/SFTP SSH/SCP
No protocol
Rsync
(copy/move)
Mail/e-mail
(!)
60

58. DEPLOYMENT PHASES
AND TASKS
 Deployment
 Deploy
 Restart application server
 Database integration
 Generate ORM models, mapping
 Dump data
 Instantiate DB (DDL usage)
 Data load into DB (DML usage, CRUD operations)
 Migrate data, structure or both
61

59. INCREMENTAL DEPLOYMENT
 Rsync
 synchronizes files and directories from one location to
another
 SVN export only changed files
 svn diff –summarize /path1 /path2
 Deploy only files modified/added since last revision
 svn status
In most cases it’s just a…
HEADACHE! 62

60. SIMPLE DEPLOYMENT
WORKFLOW
WC (working copy, trunk)
local
deployment
instance db 63

61. LOCAL DEPLOYMENT
WORKFLOW
WC (working copy, trunk)
local deployment
dev instance test instance prod instance
64
dev db test db prod db

62. REMOTE DEPLOYMENT
WORKFLOW
+ + +
dev test prod
local deployment
WC (working copy, trunk)
remote deployment
+ + +
65
dev test prod

63. MANAGE YOUR DEPLOYMENTS
1. Define what deployment type you will use
(inherits from corresponding build type)
2. Define what deployment tasks you need
3. For each deployment type prioritize chosen
tasks
4. Define destinations
(production, test, development)
5. Map destinations and deployment types
6. Configure deployment destinations
environment 66

64. DEVELOPMENT WORKFLOW
EXAMPLE
67

65. DEVELOPMENT WORKFLOW
EXAMPLE
Deployed application WC (working copy) Repository
svn up
r1049
ant deploy-private
r1050
r1051
r1052
development changeset (#146)
Second deployed …
manual merge
application
svn up r1126
ant deploy-private –Dinstance=2 resolve conflicts
svn commit
OK? yes
68
no
make corrections manual merge
svn commit

66. DEVELOPMENT WORKFLOW
EXAMPLE NOTES
 Does this example remind you something?
 Yes! Distributed version control
 What is the point?
 Svn working copy is the repository
 While deployed application is a working copy
 Difference is that all operations between
„repository‟ and „working copy‟ are manual
 And operations history is not tracked
69

67. DEVELOPMENT WORKFLOW
EXAMPLE NOTES
 Why do we need two builds?
 To ensure there are no integration errors
 Because of “Don‟t break the build“ rule
Advantages:
 You see your source code many times during merge
 You can find errors/inconsistencies easily
 Keeping unfinished development both safe and actual
Flaws:
 A lot of time spent before commit
 Works only for web-projects written in interpreted
languages 70

68. CONCLUSION
71

69. BEST PRACTICES
 Separate build from IDE
 Run faster tests first
 Introduce staging into build process
 Fail build fast
 Organize build targets by priority
 Build components separately
 Recursive build
 Centralization of project assets
 Put sources to sources (VCS)
 Put binaries to binaries (dedicated repositories)
 Consistent directory structure
72
 Develop project structure conventions

70. BUILDS AND DEPLOYMENT MANAGEMENT
1. Define what build/deployment tasks you need
2. Define what build types you need
3. Choose build/deployment phases you will need
4. Prioritize chosen tasks with regard to
corresponding build types
5. Choose build tool
6. Structure your build file
7. Define deployment destinations
8. Map build types and deployment destinations
9. Configure deployment destinations environment 73

71. RECOMMENDED READING
1. The Build Master: Microsoft's Software Configuration
Management Best Practices By Vincent Maraia
74

72. RECOMMENDED READING
2. Ant: The Definitive Guide, 2nd Edition by Steve Holzne
75

73. RECOMMENDED READING
3. Maven: The Definitive Guide by Timothy M. O'Brien
76

74. USEFUL LINKS
1. http://freshmeat.net/articles/build-and-release-management -
Build and release management
2. http://onjava.com/pub/a/onjava/2003/12/17/ant_bestpractices.ht
ml - Ant best practices
3. http://www.sonatype.com/books/mvnref-book/reference/public-
book.html - Maven complete reference
4. http://repo.exist.com/dist/maestro/1.7.0/BetterBuildsWithMaven.
pdf - Great book about Maven
5. http://kent.spillner.org/blog/work/2009/11/14/java-build-
tools.html - Ant vs Maven
6. http://martinfowler.com/articles/rake.html - article about rake by
Martin Fowler
7. http://www.finalbuilder.com/finalbuilder.aspx - FinalBuilder home
page
77