Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Similar to The Changing Role of a DBA in an Autonomous World
Similar to The Changing Role of a DBA in an Autonomous World (20)
More from Maria Colgan
More from Maria Colgan (8)
Recently uploaded
Recently uploaded (20)
The Changing Role of a DBA in an Autonomous World
- 1. The Changing Role of the DBA in an Autonomous Database World Maria Colgan Master Product Manager Mission Critical Database Technologies April 2019 Copyright © 2019, Oracle and/or its affiliates. All rights reserved.
- 2. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Safe Harbor Statement The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, timing, and pricing of any features or functionality described for Oracle’s products may change and remains at the sole discretion of Oracle Corporation.
- 3. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Today DBAs Spend More Time on Maintenance vs Innovation 3 85% of security breaches occurred after the CVE was published - DB Maestro Security 85% 91% experience unplanned data center outages - Healthcare IT News Reliability 91% 72% of IT Budget is spent on Generic Maintenance tasks vs Innovation -ComputerWorld Maintenance 72%
- 4. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | 4 How will the Autonomous Database change things
- 5. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Oracle Autonomous Database 5 Self-Driving Automates all database and infrastructure management, monitoring, tuning Self-Securing Protects from both external attacks and malicious internal users Self-Repairing Protects from all downtime including planned maintenance
- 6. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Autonomous Transaction Processing 6 Autonomous Database | Optimized by Workload Available Since March 2018 Available Since August 2018 Autonomous Data Warehouse ORACLE AUTONOMOUS DATABASE
- 7. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | 7 Autonomous Optimizations | Specialized by Workload Autonomous Transaction Processing Autonomous Data Warehouse Row Format Optimizes Response Time Creates Indexes Columnar Format Optimizes Complex SQL Creates Data Summaries Plan Stability and Run Away Query Prevention
- 8. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | • An expert system that implements indexes based on what a performance engineer skilled in index tuning would do • It identifies candidate indexes and validates them before implementing • The entire process is full automatic • Transparency is equally important as sophisticated automation – All tuning activities are auditable via reporting 8 Automatic Indexing
- 9. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | What Autonomous Database means for DBAs 9 • Tasks Specific to the Business – Architecture, planning, data modeling – Data security and data lifecycle management – Application related tuning – End-to-End service level management • Tactical Operations – Configuration and tuning of systems, network, storage – Database provisioning, patching – Database backups, H/A, disaster recovery – Database optimization Value Scale Innovation Maintenance
- 10. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | What Autonomous Database means for DBAs 10 Removes tactical drudgery, more time to innovate • Tasks Specific to the Business – Architecture, planning, data modeling – Data security and data lifecycle management – Application-related tuning – End-to-End service level management • Tactical Operations – Configuration and tuning of systems, network, storage – Database provisioning, patching – Database backups, H/A, disaster recovery – Database optimization Value Scale Maintenance Innovation
- 11. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | ADB Responsibility Customer Responsibility Service Provisioning ✓ Network, Storage and Server Infrastructure ✓ OS Administration ✓ Backup and Restore ✓ Patching ✓ Service Continuity (HA) and Disaster Recovery ✓ Database Configuration and Tuning ✓ Security and Compliance ✓ User Management, Role and Permissions ✓ Database Schema Management ✓ Data Security Policies ✓ Data Loading and Integration ✓ BI Services and Tools Integration ✓ Cloud Services Management ✓ 11
- 12. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | 12 What can I do to prepare for this evolution
- 13. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | Data Modeling End-to-end Service Level Management 13 Data Security Application Tuning Evolving the DBA Role
- 14. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | 14 Data Modeling
- 15. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | Data Modeling • A good data model requires knowledge of the underlying business as well as relational database principle – Requires working closely with business users to learn how data is used – Understanding how the data will be used helps determine best model – No special data-modelling techniques required for Autonomous – Follow best practices for DW / ATP schema design • A sound data model – Reduces risk of poor query performance – Improves time to market – Accommodates future extensions 15 Understand The Benefits of a Good Architecture
- 16. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | • Take advantage of Oracle SQL Developer Data Modeler • A free diagramming and data modeling tool – Logical and relational modeling – Versioning – DDL Generation – Compare and Synch • Reverse Engineer: – Existing Schemas , DDLs (not just Oracle) • Publish your diagrams and data dictionaries – HTML, PDF, SVG or database reporting repository 16 Data Modeling What can I do to prepare?
- 17. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | • Most popular data format for new web applications is JSON • Storing JSON documents in the database greatly simplifies application development as the same schema-less data representation can be use in Application and the Database • Just because the schema is now in a JSON document, doesn’t mean you shouldn’t have a entity model • Don’t be afraid to bring modeling to the “wild west” of JSON • If you already have JSON take advantage of Oracle JSON Data Guide – Discovers the structure of collection of JSON documents 17 Not Just Relational Model Data Modeling
- 18. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | 18 Data Security DATABASE SECURITY
- 19. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | Security Managed by the Customer • Ongoing security assessments • Users & Privileges • Sensitive data discovery • Data protection • Activity auditing Security Managed by Oracle • Network security and monitoring • Strong OS and platform security • Database patches and upgrades • Administrative separation of duties • Data encryption by default 19 In the Cloud, Security is a Shared Responsibility Security
- 20. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | • Identify your assets – Know what data you have and where • Secure all databases – Ensure there are no insecure settings, default passwords etc. – Remove unnecessary privileges – Determine what data should be masked in dev and test environments – Determine what data should be redacted or dynamically masked in applications – Encrypt your data What sensitive data do I have? How much? Where is it? What data should be mask? What data should be redact? Who has access? Security What can I do to prepare? Sensitive Data Discovery Data Protection
- 21. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | 21 Announcing Privilege Analysis now available in EE (no option required)
- 22. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | Oracle Database Security Assessment Tool (DBSAT) • Understand how (in)secure your database is – Report on overall security status – Find the users, entitlements, and risks – Discover sensitive data • Actionable Assessment Reports – Summary and detailed information – Prioritized recommendations – Mapping to EU GDPR and CIS Benchmark • Stand-alone light weight tool: Quick, Easy • FREE to current Oracle customers Assess Your Security Profile Before Hackers Do Database Securely Configured? Users? Entitlements? What Sensitive Data do I have? For Oracle Databases 10g and later Collaborate 19 - San Antonio 22
- 23. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | 23 End-to-End Service Level Management
- 24. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | • You can manage cloud services via cloud service console (web-based UI) 24 End-to-End Service Level Management Learn to Manage cloud services
- 25. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | 25 • Can also manage cloud services via cloud API’s • Cloud API’s provide full control over cloud services – Every command available in service console (UI) is also available via API’s • Available interfaces: – REST API – CLI – SDK’s for Java, Python, etc End-to-End Service Level Management APIs available for all ADB Operations
- 26. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | End-to-End Service Level Management • Autonomous Database will patch databases in a rolling fashion • Goals is to have applications continue to operate without errors and within the specified response time objectives while this patching occurs • Steps to prepare for Application Continuity are: 1. Use Services for location transparency 2. Configure TNS connection string to enables transparent retries and load-balancing 3. Make sure your application is Fast Application Notification aware • If you use Oracle client driver – you already have this • If you use a 3rd party driver – follow the instructions in our white paper 4. Enable application continuity if you are currently on Oracle Database 12c or higher 26 Application Continuity
- 27. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | New White Paper Continuous Availability Application Checklist for Continuous Service for MAA Solutions oracle.com/technetwork/database/options/clustering/applicationcontinuity/ adb-continuousavailability-5169724.pdf 27
- 28. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | 28 Application Tuning DBA
- 29. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | Application Tuning 29 Same Rules Apply • Take the time to understand the business process and what’s important – No point improving a SQL statement if no one cares that it takes an hour • Connection management is just as important in the cloud – Having thousands of connects logging on and off the database every second won’t scale • Hard parsing thousands of statements a second won’t scale • Get familiar with new functionality in the latest versions of the database that will improve end-user experiences – Real-Time Materialized Views – JSON support – Approximate Analytic Queries – REST APIs – Etc.
- 30. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | 30 Where should I start
- 31. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | New Project or Applications 31 • Simplicity and faster time-to-market for NEW applications – All new applications should go on Autonomous Database • Autonomous Database – Eliminates dependence and delays on others for servers, storage, and databases – Eliminates database tuning, auto adapts to changing workload – Provides advanced SQL and PL/SQL capabilities to accelerate developer productivity App Dev
- 32. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | Customers Looking to Apply Machine Learning 32 • Customers who want to gain better insights into their business – Customers looking to gain better insights into their business via Machine Learning • Autonomous Database – Provides integrated Machine Learning enabling real-time predictive capabilities – ML models can be built on ADW where there is a large historical data set – ML models can then be used to make real- time predictions on active data in ATP Machine Learning
- 33. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | Determine Fans Most Likely to Renew Season Tickets 1. Build ML model in ADW where we have a detailed history of all fan activities using in- Database ML algorithms DBMS_DATA_MINING.CREATE_MODEL( model_name => 'SEASON_TKS_MODEL', mining_function => dbms_data_mining.classification, data_table_name => 'FAN_DETAIL_TAB', case_id_column_name => 'FAN_ID', target_column_name => 'BUY_SEASON_TKS', settings_table_name => 'GLM_SETTINGS'); 2. Apply the ML model via a simple SQL query on ATP to predict which fan likely to buy SELECT prediction_probability(BUY_SEA_TKS, ‘Yes’ USING 3500 as bank funds, 40 as age, ‘Married’ as marital_status) FROM dual; 33 Using simple two-step Machine Learning (ML) process Collaborate 19 - San Antonio
- 34. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | 34 What about existing apps?
- 35. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | Migration to Autonomous Database • A logical migration for Autonomous must be performed because: – Database must be PDB, upgraded to 19c, and encrypted – Any changes to Oracle shipped stored procedures or views must be found and reverted – All uses of CDB admin privileges must be removed – All legacy features that are not supported must be removed (e.g. legacy LOBs) • Migration uses Data Pump to move database data into new Autonomous DB – GoldenGate replication can be used to keep database online during process 35
- 36. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | Autonomous Database Schema Advisor • ADW Schema Advisor is a light-weight PL/SQL Package – Simple to Install and execute – Installs in the Database (11.2+) to be analyzed • Generates a report highlighting the Schema objects that: – Can be migrated – Cannot be migrated – Will migrate with changes • Available for download – MOS Doc 2462677.1 36 Schema ADW Schema Advisor Database Feature Restrictions DWCS Lockdown Rules, Datatype Restrictions Report
- 37. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | 37 The Future of the DBA
- 38. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | Summary: Autonomous Database and the DBA • Goal for DBA’s: Develop stronger relationships with business teams and focus on delivering solutions • Autonomous Database does the mundane maintenance task – provisioning, securing, patching, tuning etc. • But there are still major technical responsibilities required for Autonomous Database – Database Schema Management – Data Security Policies – Cloud Services Management – Data Loading and Integration – Application Tuning 38
Editor's Notes
- Don’t believe us? Let’s look at the statistics 72% of IT Budget is spent on Generic Maintenance tasks vs Innovation 85% of security breaches occurred after the CVE was published 91% experience unplanned data center outages And what’s scariest of all is that 80% of the outages experienced today are due to human error
- In order to help change these statistics Oracle has introduce the Autonomous Database, a new category of cloud server which automates the complete lifecycle management of an Oracle Database with the help of Machine Learning. So, how does the Autonomous Database change things for you?
- 5
- Both ADW and ATP share the Autonomous Database platform of Oracle Database 18c on our Exadata Cloud infrastructure. The difference is how the services have been optimized within the database. When you start loading data into the autonomous database, we store the data in the appropriate format for the workload. If it is ADW, then we store data in columnar format as that’s the best format for analytics processing If it is ATP, then we will store the data in a row format as that’s the best format for fast single row lookups Query optimization: for analytics workload, we automatically parallelize the query execution to access large volumes of data in a short amount of time to answer biz questions If it is a transaction processing system, then we will automatically detect missing indexes and create them for you. Regardless of the workload we need to keep optimizer statistics current to ensure we get optimal execution plans. With ADW we are able to achieve this by gather statistics as part of all bulk load activities. With ATP, where data is add using more traditional insert statements statistics are automatically gathered periodically. As the data volumes change, or new access structures is created, there is the potential for an execution plan to change and any change could result in a performance regression so we use Oracle SQL Plan Management to ensure that plans only change for the better.
- Automatic Indexing is an expert system that implements indexes based on what a performance engineer skilled in index tuning would do. But unlike a human it is able to work 24 hours a day, 7 days a week, 52 weeks a year. It also takes full responsibility for its decisions and hence its decisions are validated before they are implemented to ensure proposed indexes really do improve performance. The entire process is fully transparent to the DBA via a detailed report of what happens each time automatic indexing kicks in. It’s a simple 6 step processing: 1.Capture Periodically captures SQL statements into a SQL repository Include plans, bind values, execution statistics, etc. (AWR for SQL only) 2.Identify Candidates Identify candidate indexes that may benefit the newly captured SQL statements Creates index candidates as unusable, invisible indexes (just metadata) 3.Verify Ask the optimizer if index candidates will be used for these statements Index candidates not used by the optimizer are automatically dropped Complete creation of chosen indexes and run captured statements to verify that the indexes did improve performance 4.Confirm If performance is better for all statements, the indexes are marked visible If performance is worse for all statements, the indexes are dropped If performance is worse for some, the indexes are marked visible except for statements that regressed 5.Validation Each Use The validation of the new indexes continues for other statements, online First session that runs each affected SQL validates benefit, and avoids index if none 6.Monitor Index usage is continuously monitored Automatically created indexes that have not been used in a long time will be dropped Rebuilds decaying indexes
- 10
- Highlight Findings related to: Oracle Best Practices CIS Oracle Database Benchmark EU GDPR What does DBSAT Check? Security Configuration Data Encryption Auditing Policies Fine-grained Access Control Database and Listener Configuration OS File permissions Security Patches Users and Entitlements User Accounts, Privileges and Roles Sensitive Data Which type, where, how many - To discover sensitive data in the database, DBSAT looks for column names and column comments.
- The business is going to look to you to determine what database cloud service they should use You will be in charge of, and in control of, the end-to-end service levels You need to know what each Database Cloud Service offers you in terms of Availability Security Performance Scalability Take advantage of trial accounts to get familiar with provisioning and using a cloud environment Oracle connection pools, for example, use FAN to receive very fast notification of failures, to balance connections following failures, and to balance connections again after the failed components are repaired. So, when a service at an instance starts, the connection pool uses the FAN event to route work to that resource, immediately. When a service at an instance or node fails, the connection pool uses the FAN event to immediately interrupt applications to recover. FAN is essential to prevent applications from hanging on TCP/IP timeouts. Application Continuity masks outages from end users and applications by recovering the in-flight work for impacted database sessions following outages. Application Continuity performs this recovery beneath the application so that the outage appears to the application as a slightly delayed execution.
- Back to Penny
- In step 1, we create a ML Model, ‘season_tks_model’, which classifies the likelihood of fans purchasing a season ticket based on detailed history – which would typically reside in ADW. (the settings table is there to decide what to to when certain columns are null – for example). In step 2, we apply the model to determine the probability of fans most likely to renew their season ticket. In the example shown, we want to know the probability of a 40-year old married person with $3,500 in the bank will purchase. By applying ML classification model (in this example), we can more readily identify top prospects for a sale from a long list of names/leads.
- Has standard GoldenGate restrictions or rowids, nested tables, identity columns, etc.
- ADW Schema Advisor provides guidance on Oracle Schemas to be migrated to ADW Analyzes only the Schema objects, not the workload Generates an easy to understand report Easy to use, fast, command-line tool Analyzes the metadata, does not depend on Data size