Recommandé
Recommandé
Contenu connexe
Tendances
Tendances (20)
Similaire à Cloud Native Testing, 2020 Edition: A Modern Blueprint for Pre-production Testing of Microservices
Similaire à Cloud Native Testing, 2020 Edition: A Modern Blueprint for Pre-production Testing of Microservices (20)
Plus de OlyaSurits
Plus de OlyaSurits (9)
Dernier
Dernier (20)
Cloud Native Testing, 2020 Edition: A Modern Blueprint for Pre-production Testing of Microservices
- 1. Cloud Native Testing, 2020 Edition: A Modern Blueprint for Pre-production Testing of Microservices Jon Edvald, CEO & Co-founder @JonEdvald
- 2. What we’ll talk about • Walk through di erent types of tests • Deep-dive on integration tests • Spinning up test environments • Good practices for integ/end-to-end tests • Scaled-down blueprint and quick summary
- 3. chaos testing exploration monitoring A/B testing canarying feature flagging config tests benchmarking integration testing shadowing fuzz tests unit tests property based tests contract testing functional tests security tests TESTING IN DEV PRE-PRODUCTION TESTING TESTING IN PRODUCTION observability end-to-end testing
- 4. chaos testing exploration monitoring A/B testing canarying feature flagging config tests benchmarking integration testing shadowing fuzz tests unit tests property based tests contract testing functional tests security tests TESTING IN DEV PRE-PRODUCTION TESTING TESTING IN PRODUCTION observability end-to-end testing unit tests • Crucial for most projects, but… • Often overused. Consider functional tests where applicable. • Use for stateless, side-effect free functions. • Don’t strive for 100% coverage.
- 5. chaos testing exploration monitoring A/B testing canarying feature flagging config tests benchmarking integration testing shadowing fuzz tests unit tests property based tests contract testing functional tests security tests TESTING IN DEV PRE-PRODUCTION TESTING TESTING IN PRODUCTION observability end-to-end testing fuzz tests • Good complement to unit tests, for stateless functions. • Esp. important for e.g. C/C++ code (that may not be memory safe). • Use frameworks/sanitizers to facilitate. property based tests
- 6. chaos testing exploration monitoring A/B testing canarying feature flagging config tests benchmarking integration testing shadowing fuzz tests unit tests property based tests contract testing functional tests security tests TESTING IN DEV PRE-PRODUCTION TESTING TESTING IN PRODUCTION observability end-to-end testing functional tests • Often preferable to unit tests, since they have broader coverage and mimic real-world use. • Should still be fast to run, and require limited setup/scaffolding. • Only use fakes/mocks/stubs when necessary, considering integ tests instead.
- 7. chaos testing exploration monitoring A/B testing canarying feature flagging config tests benchmarking integration testing shadowing fuzz tests unit tests property based tests contract testing functional tests security tests TESTING IN DEV PRE-PRODUCTION TESTING TESTING IN PRODUCTION observability end-to-end testing benchmarking • Use for specific performance- sensitive functions, and maybe full test runs. • Translating results to test successes or failures can be difficult.
- 8. chaos testing exploration monitoring A/B testing canarying feature flagging config tests benchmarking integration testing shadowing fuzz tests unit tests property based tests contract testing functional tests security tests TESTING IN DEV PRE-PRODUCTION TESTING TESTING IN PRODUCTION observability end-to-end testing contract testing • Fast alternative to integration testing, but… • Provides limited coverage, only tests at a schema level. • Doesn’t really test actual code, how data is processed etc. • Pact is a good framework for this.
- 9. chaos testing exploration monitoring A/B testing canarying feature flagging config tests benchmarking integration testing shadowing fuzz tests unit tests property based tests contract testing functional tests security tests TESTING IN DEV PRE-PRODUCTION TESTING TESTING IN PRODUCTION observability end-to-end testing config tests • Very important for Kubernetes (too easy to do the wrong thing). • Apply org policies for applications and infrastructure. • Use e.g. conftest, OPA, and/or Terraform Sentinel.
- 10. chaos testing exploration monitoring A/B testing canarying feature flagging config tests benchmarking integration testing shadowing fuzz tests unit tests property based tests contract testing functional tests security tests TESTING IN DEV PRE-PRODUCTION TESTING TESTING IN PRODUCTION observability end-to-end testing security tests • Again, very important for Kubernetes, for the same reasons. • Includes image/CVE scanning, black box scanning, cluster linters. • Try e.g. Starboard, Claire, clusterlint.
- 11. chaos testing exploration monitoring A/B testing canarying feature flagging config tests benchmarking integration testing shadowing fuzz tests unit tests property based tests contract testing functional tests security tests TESTING IN DEV PRE-PRODUCTION TESTING TESTING IN PRODUCTION observability end-to-end testing shadowing • Quite powerful, but difficult to get right. • Service meshes can help a lot with this. • Easiest for stateless / side- effect free / independent API calls.
- 12. chaos testing exploration monitoring A/B testing canarying feature flagging config tests benchmarking integration testing shadowing fuzz tests unit tests property based tests contract testing functional tests security tests TESTING IN DEV PRE-PRODUCTION TESTING TESTING IN PRODUCTION observability end-to-end testing • Great way to thoroughly test a system, from code through configuration. • Can be run on your full stack, or individual services talking to other production APIs. • Has been frowned upon due to slow feedback loops, but there are ways to accelerate... integration testing / end-to-end testing
- 13. Integration testing cloud native applications
- 15. Test environments • Spinning up preview environments implicitly tests your con guration. • Running tests in those environments provides a lot of coverage. • Tools like garden, Flux, Okteto can help with the setup, and some also facilitate running tests
- 17. Instrumentation • Observability helps a lot when debugging. • Emit plenty of tracing events across your code with OpenTelemetry.
- 18. Managing state • We need to make sure test suites don’t step over each other. • Partition your data, by e.g. creating a test organization/project/tenant/user for each suite, then cleaning up. • Alternative: Add metadata to requests/data and clean up a er.
- 19. Accelerating tests • Safely run multiple test suites in parallel with partitioned data. • Keep a test environment running while developing. • Trigger individual tests/suites directly from dev machine instead of waiting for a full pipeline.
- 21. chaos testing exploration monitoring A/B testing canarying feature flagging config tests benchmarking integration testing shadowing fuzz tests unit tests property based tests contract testing functional tests security tests TESTING IN DEV PRE-PRODUCTION TESTING TESTING IN PRODUCTION observability end-to-end testing
- 22. chaos testing exploration monitoring A/B testing canarying feature flagging config tests benchmarking integration testing shadowing fuzz tests unit tests property based tests contract testing functional tests security tests TESTING IN DEV PRE-PRODUCTION TESTING TESTING IN PRODUCTION observability end-to-end testing chaos testing • Test the resilience of your applications. • Suggest using in testing envs, growing to all environments incl. production. • Start by running manually, watching for failures. • Use tools and services designed for this, e.g. Gremlin, Litmus.
- 23. chaos testing exploration monitoring A/B testing canarying feature flagging config tests benchmarking integration testing shadowing fuzz tests unit tests property based tests contract testing functional tests security tests TESTING IN DEV PRE-PRODUCTION TESTING TESTING IN PRODUCTION observability end-to-end testing exploration • Important, especially with new features/applications and when test coverage isn’t very high. • May make sense as part of pre- prod review, depending on the application and team, and whether you have preview environments.
- 24. chaos testing exploration monitoring A/B testing canarying feature flagging config tests benchmarking integration testing shadowing fuzz tests unit tests property based tests contract testing functional tests security tests TESTING IN DEV PRE-PRODUCTION TESTING TESTING IN PRODUCTION observability end-to-end testing A/B testing & feature flagging • Another approach to progressive delivery. • May need a little work on application level, but tools like LaunchDarkly can help.
- 25. chaos testing exploration monitoring A/B testing canarying feature flagging config tests benchmarking integration testing shadowing fuzz tests unit tests property based tests contract testing functional tests security tests TESTING IN DEV PRE-PRODUCTION TESTING TESTING IN PRODUCTION observability end-to-end testing canary rollouts • Generally good practice. How much to rely on it depends on your application and processes. • Service meshes are helpful here, and tools like Flagger.
- 26. chaos testing exploration monitoring A/B testing canarying feature flagging config tests benchmarking integration testing shadowing fuzz tests unit tests property based tests contract testing functional tests security tests TESTING IN DEV PRE-PRODUCTION TESTING TESTING IN PRODUCTION observability end-to-end testing monitoring • Definitely do this! Monitor resource usage, infra, clusters etc. • Use for prod + staging at least, but ideally any shared dev/test clusters. • Prometheus is good for Kubernetes, or commercial services.
- 27. chaos testing exploration monitoring A/B testing canarying feature flagging config tests benchmarking integration testing shadowing fuzz tests unit tests property based tests contract testing functional tests security tests TESTING IN DEV PRE-PRODUCTION TESTING TESTING IN PRODUCTION observability end-to-end testing observability • Great for debugging. • Use in all environments if possible, also handy during dev + testing. • Use e.g. Jaeger, Honeycomb, Lightstep.
- 28. And now to summarize: our testing blueprint
- 29. chaos testing A/B testing canarying feature flagging config tests benchmarking shadowing fuzz tests property based tests contract testing TESTING IN DEV PRE-PRODUCTION TESTING TESTING IN PRODUCTION end-to-end testing unit tests functional tests security tests integration testing observability exploration monitoring
- 30. Further reading • Testing microservices the sane way—Cindy Sridharan • Unit testing is overrated—Alexey Golub • Write tests. Not too many. Mostly integration.— Kent C Dodds • Contraption Maker - 🙃 • garden.io / medium.com/garden-io