ארגונים ברחבי העולם מגבירים את השימוש בתהליכי DevOps לטובת שיפור היתרון התחרותי שלהם, הורדת סיכונים והפחתת עלויות פיתוח. כיום ניתן ליישם את ההצלחה של ה-DevOps בעולם מסדי הנתונים, על ידי ביצוע אוטומציה של תהליכי הפיתוח והעברה בין סביבות, אכיפת מנגנוני אבטחה, והפחתת הסיכונים הכרוכים בתהליך.
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Challenges and Best Practices of Database Continuous Delivery
1.
2. ▪ You will be on mute for the duration of the event
▪ Please type a message in the Questions box in the
Control Panel if you can’t hear us (please check your
speakers and GoToWebinar audio settings first)
▪ If you have questions during the session, please submit
them on the Q&A bar on your GoToWebinar dashboard
and we will address them at the end
▪ A recording of the full webinar will be put up online
Before we Begin
3. Yaniv Yehuda
Co-Founder & CTO at DBmaestro
Spent the last years raising awareness about the challenges around database
development and deployment, and how to support database Continuous Delivery.
Joined NessPRO as a product manager for the line of SAP complementary products and
DevOps solutions for the DB.
Shachar Furman
Product Manager, Central Systems, NessPRO
About the presenters
4. Outline
▪ The Adoption of Continuous Delivery
▪ The Reasons why the Database is Often Left Behind
▪ Single Source of Truth for your Database
▪ Best Practices for Including the Database in CD
▪ The Bigger Picture – Automation and Stopping the Line
▪ Q&A
5. NessPRO is Ness Technologies’ products group and the sole representative in
Israel of more than 30 international and Israeli organizations which develop
software products designated for the enterprise market.
About NessPRO
9. The pain–Fortune 1000 by IDC
Application
Downtime Cost $2B/Y
Deployments/month 2x
Growth
Accelerate Delivery by 20% Compliance & Audit
Enforcement
Infra Failure
Hourly Cost $100K
IDC DevOps Best Practices metrics: Fortune 1000 Survey, December 2014
Loss of
Reputation
11. ▪ Every business is an IT business
▪ Customers demand that you deliver new features faster
− Agile Development
− Process Automation
− DevOps
▪ Can’t wait 6 months for that next waterfall release…
▪ If you don’t, your competitor probably will
DevOps & CD: a must for every company
13. ▪ More rapid changes
▪ Fewer changes backed out
▪ Better collaboration
▪ Fewer defects
▪ Ultimately better service
▪ Happy customers
▪ Profitability
How Do I Measure Success?
16. Only 13% are actually performing basic CD practices for the DB!!!
Manual work:
cant scale, cant match CD frequency
not repeatable, prone to error
Continuous Delivery is big and getting bigger, but...
17. Manual steps lead to Chaos in Emergency Times
90%
Rate this as a risk factor, but
53%
Break the process and test urgent hot-fixes in pre-
production
18. Old adage but true
– The database is often neglected and therefore can become the
weakest link
– Manual processes
Database/Code Silos exist…
– Don’t always communicate effectively
– Need to follow same procedures & best practices
Essential from a compliance and business point of view
Should be the strongest link
Is the database the Weakest link in a chain ???
21. What is the problem?
▪ Root Causes for issues:
− Challenging manual source control process
− Static deployments code overrides and configuration drift
− Dynamic deployments tools unaware of version control
− No release automation red-flags – don’t know when to stop the line…
22. File Version Control Process
Today: Two isolated processes
DB Development Process
Check-Out
Script
Modify Script
Get updated
Script from DB
Check-In
Script
Compile
Script
in DB
Debug Script
in DB
?
?
?
?
A
A’
Version control is out of sync from the database and cannot act as
a Single Source of Truth
23. 90%Rate this as a risk factor, yet
72%Admit database may not be in sync with the source repository
24. ▪ Challenges:
■ Code-overrides
■ Working on the wrong revisions
■ Scripts do not always find their way to the version control solution
■ Out of process updates go unnoticed
■ Hard to locate outdated update scripts
▪ Playing safe? What we really need:
■ The upgrade script
■ The actual code of the object
■ A roll-back script
Scripts & version control
25. X
1.11.1.11.11.21.31.41.51.61.7
Int QA Stage Prod
Database Deploy Script
Dev
Dev
Dev
Model
1.1 1.2
1.2 1.3
1.3 1.4
1.4 1.5
1.5 1.6
1.6 1.7
1.11.11.41.7
1.1 1.2
1.2 1.3
1.3 1.4
1.4 1.5
1.5 1.6
1.6 1.7
1.1 1.2
1.2 1.3
1.3 1.4
1.4 1.5
1.5 1.6
1.6 1.7
Out of Process
Change
X
X
X
X
X
? 1.1.1
X
Challenge with static scripts…
Configuration drift…
26. 60%Of those manually building scripts have to fix or tweak them regularly as part of a deployment process
27. 80%
Rate this as a risk factor, yet
71%
Follow manual processes to create their database deploy scripts
28. ▪ Scripts
− Hard to test in their entirely (holistically)
− Hard to test due to colliding dependencies
− Need to run in a specific order…
− Much harder to deal with project scope changes
▪ Scripts, unless super sophisticated:
− Unaware of changes made in the target environment
− Time passed from their coding to the time they are run
− Potentially overriding production hot-fixes or work done in parallel by another team
Scripts are static…
29. Test cases using compare & sync tools:
An index exists in source (QA) but not in target (Production)
What should we do? Add the index or not?
Using tools
32. 70%of those using compare & sync tools have to review and fix the results as they can't always trust them to automatically deploy
correctly
33. A compare & sync tool:
▪ Is unaware of any changes that occurred before the time it ran
▪ Has no knowledge of changes that took place at the target environment
▪ Does not leverage version control for more information
▪ Unable to deal with conflicts & merges between different teams
▪ Requires manual inspection
▪ Requires detailed knowledge regarding
each change as part of the process
So…no automation…
We fear for automating problems into
production and a major risk!!!
Challenges
38. 1.11.21.31.41.51.61.7
*
Int QA Stage Prod
Dev
Dev
Dev
Model
1.1 1.2
1.2 1.3
1.3 1.4
1.1 1.7
1.1.1 1.7
1.1 1.1 1.11.41.7
Out of Process
Change
1.1.11.7 1.1.11.7
Validate
1.4 1.5
1.5 1.6
1.6 1.7
Configuration Drift prevention / conflict identification
and Validated execution
1.4 1.5
1.5 1.6
1.6 1.7
OR
Baseline aware analysis
Validated execution / Build & deploy on demand
Validate
39. ■ Understand the nature of the changes
■ Raise red flags on conflicts
■ Support out-of-process changes
■ Utilize baseline aware analysis
Safety Net Deployment Automation
40. If we had the index in the baseline (previous version) and no longer in Dev (i.e. - removed)
=> we should take it down from production…
(Deploy Change)
Deploying changes if needed
Development Baseline
Previous Label /
Production Golden Copy
Production
41. Development Baseline
Previous Label /
Production Golden Copy
Production
BUT… If no index in baseline => someone else added it to Production…
we should protect the NEW index on production!!!
(Protect Target)
Or protecting target environment…
44. Continuous Delivery Pipeline Builder
• Define a process
• Automate the process
• Prevent/Alert out of process changes
45. Raise red flags to stop the line…
if requires human intervention
Impact Analysis! Not Damage Control…
46. Automate “everything”
– Package the deployment of database changes along with all your other
application components to give a unified picture
Move the process upstream
– Easily promote the same package (including database changes!) from one
environment to the next, handling environment-specific differences automatically
Create the deployment pipeline
For successful CD:
47. ▪ Requited integrations into entire continuous delivery pipeline:
■ IDEs (Oracle, Microsoft, MySQL in dev)
■ Source control (MsTFS, Git, Subversion etc.)
■ Task based development and ticketing systems (Jira, MsTFS, IBM RTC
etc.)
■ Build tools (Jenkins, bamboo etc.)
■ ARA tools (IBM uDeploy, CA release automation, Chef etc.)
Integrated CD world…
48. ▪ Focusing on changes rather than managing changes and
dealing with re-work, boosted overall productivity of 250
developers. We estimate we were able to do 15% more with the
same resources.
▪We went from several fix-centric deployments a day, to one
feature-centric deployment a week.
▪The amount of incidents in production has declined as well. We
had 20% less incidents.
▪CIO @ Credit Card company
“
Testimonials - Efficiency
“
49. ▪ Regulation requirement - SOX and derived 357
▪Auditing - change management approach with change
management auditing. You have to do it anyway, do it
automatically and efficiently.
▪Change management in production itself is regulatory
required (ITIL). But you cannot ensure it without managing
the whole process starting at Dev.
▪We have to comply with regulation - but the business
benefits from it.
▪CIO @ Credit Card company
“
Testimonials - Regulation
נספרו הינה קבוצת מוצרי התוכנה של נס טכנולוגיות והנציגה הבלעדית בישראל של למעלה מ-30 חברות בינלאומיות וישראליות המפתחות מוצרי תוכנה המיועדים לשוק הארגוני.
הקבוצה פעילה בשוק הישראלי למעלה מ-40 שנה ומשרתת מאות לקוחות.
בנוסף למכירת מוצרי התוכנה במודלים עסקיים שונים. מתמחה נספרו גם באינטגרציה, שיווק, שירות, הדרכה ותחזוקה של מוצרים אלו.
אנחנו מייצגים פתרונות ממגוון רחב של תחומים – DevOps, Cloud solutions, Mobile, אבטחת מידע, שיווק דיגיטלי, משלימים עבור SAP, העולם הפיננסי, ניהול ידע ועוד.
להלן חלק מהיצרנים אותם נספרו מייצגת. בין החברות אותנו אנו מייצגים ניתן למצוא את DBmaestro, EPI-USE, CA, דוקומנטום של EMC, Evolven, Sybase ועוד רבים נוספים.
הצלחה עסקית - מוכרים בהצלחה ליותר מלקוחות 500 - 2000 הגדולים בעולם
IT organizations that have tried to custom adjust current tools to meet DevOps practices have a failure rate of 80%, thus making tool replacement and/or addition a critical requirement
The average cost percentage (per year) of a single application's development, testing, deployment, and operations life cycle considered wasteful and unnecessary is 25%
Development teams are the leading sponsors of DevOps teams, with operations and architecture teams close behind
There are significant acceleration advantages for IT leaders that decide to create a DevOps team or center of excellence versus a less-organized DevOps organizational approach
Average total cost of unplanned application downtime per year is $1.25b - $2.5b
Hourly cost of an infrastructure failure is $100K/hour to $500K – $1 M
Average number of deployments/month is expected to double in two years
DevOps-led projects will accelerate the delivery of capabilities to the customer by 15%–20%
Over the next two years, DevOps will adopt security, compliance, and audit
The average cost percentage (per year) of a single application's development, testing, deployment, and operations life cycle considered wasteful and unnecessary is 25%
Development teams are the leading sponsors of DevOps teams, with operations and architecture teams close behind
There are significant acceleration advantages for IT leaders that decide to create a DevOps team or center of excellence versus a less-organized DevOps organizational approach
הצלחה עסקית - מוכרים בהצלחה ליותר מלקוחות 500 - 2000 הגדולים בעולם