Transcription
How not to do bad things fasterTest fast and benefit continuously – QA to support CDGwen Stewart – independent presentation
What we are going to cover– Why are you doing CD anyway? Benefits? Costs?– Quality assurance of the software process – as well asthe product– Lessons learnt . (how not to do it)– What can you do?
How did I get here? 4 goes at working with/implementing/designing aCD process to far– 2010 Expedia – 2 teams 2 products– 2012 Microsoft – 4 teams 2 products– 2014 Amazon – 4 teams 1 product– 2016 BBC – 19 teams 4 products– 2019 . CPA Global
Why do it anyway? Move fast and break things? DORA Metrics (Google state of devops report 2019)– Lead time for changes– Deployment frequency– Time to restore service– Change failure rate
Why do it? To protect yourself from yourself– Hide deployment risk from customers– Avoid death by 1000 cuts
Martin Fowler definition You’re doing continuous delivery when:– Your software is deployable throughout its lifecycle – supported byfeature flags and live like environments, automated coverage– Your team prioritizes keeping the software deployable overworking on new features – zero tolerance – no false negatives– Anybody can get fast, automated feedback on the productionreadiness of their systems any time somebody makes a change tothem – automated tests run anywhere – and appropriate forstage/environment – plus live like environments– You can perform push-button deployments of any version of thesoftware to any environment on demand – automated pipeline
2 key paths to follow
Quality Assurance and Testing Quality Assurance of the software lifecycle– Decouple deploy and release– Automation to support the process Quality Assurance and testing of the product– Decouple test and deploy– Automation to support the product
The 5 continuities Continuous feedback – from monitoringContinuous integration – regular mergeContinuous testing – automated tests on merge Continuous delivery – auto deploy to prod after merge Continuous development – frequent automated releaseof deployed features to customers Continuous feedback – from monitoring Thanks to Jean-Paul Varwijk
Copyright: Jez Humble
Ship broken code!?
Feature FlagsCopyright Stackify
Copyright: Hyperbole and a half
Test pyramidE2E testsFunctional testsAPI testsUnit tests
We have something that looks a bit like this
And this happens
The release cycle of DOOM
Maybe a new pipeline will help
Copyright Amazon
Test pyramid
So now we only have the critical things
The Pipe is blocked Keep the pipeline green!– Test code is treated like production code, reviews, unittests, deploy (via CD etc)– No tolerance of failed tests – the pipeline needs toALWAYS be green– No waiting for one more change there will be anotherrelease in an hour
Copytight Redbubble
Monitoring and learning from production
Positive feedback loop
Key Principles Understand the customer lived experience Put as much effort into the current site as into newfeatures Decouple deployment and feature launch Everything is behind a feature flag – EVERYTHING Automation at all levels (pyramid) Zero tolerance
QA specifics to implementFeature flags and a tidy up policyShip small changes regularly - No waiting for one last changeThe test product is as important as the product.Keep the pipe GREENAutomation is critical but bad automation is worse than noautomation Automation to support delivery as well as test the product Absolutely NO testing in the pipe! Really
Is it working? Lead time for changesDeployment frequencyTime to restore serviceChange failure rate Bugs in productionDORA metrics – state of devops report 2019
Key things for any software developmentprocess CommunicationDisciplineBe “a”gileIt’s never a low risk one line config change
Thank youhttps://uk.linkedin.com/in/gwen-stewart-71b9144
Further reading– html– devops-2019.pdf– https://continuousdelivery.com/– Search for Jez Humble videos on YouTube
You're doing continuous delivery when: -Your software is deployable throughout its lifecycle -supported by feature flags and live like environments, automated coverage -Your team prioritizes keeping the software deployable over working on new features -zero tolerance -no false negatives
