Agile Testing Days 2021

This is the first in-person conference for me in almost two years. Crazy times! Below are my notes from the talks that I attended at the Agile Testing Days Conference 2021 in Potsdam, Germany.

Be warned, these are just notes, and basically something to help me remember the conference content, so context is probably missing for the casual reader.

Agile Testing Days - Tutorial Day

Keynote: “Test Reporting”; Paul Holland & Huib Schoots

• software testers and devs mean different things when using the term “tests”
• testers view automated tests as being of the level the baseline/sanity tests
• testing is a much broader term; testers test systems using exploratory (and other) methods and do things much more thoroughly than software devs usually do
• decision makers need information, not data
  • a solution: tell stories about the status of the product, mention risks

Agile Testing Days - Day 1

Keynote: “Coming to terms with intelligence in machines”; Dagmar Monett

• defining intelligence in humans is difficult and has changed over time
• culture often defines how people view what intelligence is
• AI community isn’t worried that machines will take over the world
• machine learning isn’t artificial intelligence
• AI is just software

“The Struggle with Learning How to Code”; Maaike Brinkhof

• meta talk; not about steps to learn how to code
• talk used Miro (!)
• talk is about perfectionism, changing beliefs, overcoming oneself
• got into testing team at first job after manager recommended it
• revelation: one can solve problem on paper before programming it on computer
• kept failing when learning to code; couldn’t deal with feelings; couldn’t deal with failing when learning; needed to learn to break through the barrier and see the problem behind the problem
• why fail? Expected the solution to “just come”; didn’t break down the problem
• smart people don’t just “get” things all at once; they tend to put in the work to understand something
• being a perfectionist is unhelpful; setting high expectations that one can never achieve
• needed to learn to become comfortable with failing
• ingredients: time, patience, expect to feel like shit and continue anyway
• learning styles: learning from books didn’t work for her; a Udemy course did work -> the lower tempo in the course forced her so slow her thinking down and had more repetition. Felt accomplisment.
• used components of stoicism to overcome destructive emotions
• has concept of “circle of control”; what’s in your control? -> Your mind
• stoicism helped: brought confidence, less perfectionism, comfort at work
• why are YOU failing (or feel like you are failing)? Is it time? Your approach? Can you find out the problem behind the problem? There isn’t a magic solution.
• https://www.maaikebrinkhof.nl/resources-code/
• https://exercism.org -> programming exercises, and real people as mentors

“Unit Testing and TDD from the tester perspective”; Alex Schladebeck

• used to believe that only developers write unit tests and that it is easy and not for testers
• came from impression from simple examples about TDD etc., from course materials etc., and view from outside
• lead to: delegation of responsibility, lack of empathy, lack of interest, frustration
• got curious: began learning, reading blogs, going to workshops
• testers ask good questions when pairing with devs
• learned that:
  • daily work is much harder than the simple examples one sees for TDD
  • big decisions happen in small amounts of code
  • behaviour change is hard, especially in a project context
• found out that:
  • starting TDD can be very hard and slow
  • naming things is hard
  • TDD is a powerful concept
• started having better conversations about TDD
• improved communication in the team and company
• had to wait for “the magic” to happen that the idea takes off
• create a safe learning environment
  • quality dojo -> a safe space to learn and practice
  • process is more important than product
  • ask questions
  • welcome diversity: beginners and experts
  • “yes and” rather than “no, but”
• stay curious, challenge beliefs, we are stronger when we work together

Keynote: “The Power of Coaching for Leading Test Teams”; Maryam Umar

• mentoring is long term; coaching is short term
• in mentoring, mentee drives sessions; in coaching, coach drives sessions
• mentoring is about giving guidance; coaching is about training and upskilling
• how one phrases things changes the mindset one has; e.g. replacing a negative word with a positive one can change perspective

“A journey through an adapted zero-bug policy”; Simone Colosimo

• his first talk in English; is doing really well!
• backlog management
  • defer non-priority bugs to fix them later
  • ‘spring cleaning’ to reduce backlog
  • low value given to reported bugs
  • wasn’t helpful long-term
  • end of 2020 the company had over 1000 bugs
• solution
  • put quality and customer back to centre of project
  • apply zero bug policy concept applied to their context
• zero bug policy
  • defn: a problem is either a bug or it isn’t
  • a bug has higher prio than new developments or improvements
  • in their context was too rigid
  • risk of pushback from team
• zero bug policy adapatation
  • product owner/tech lead decides if bug should be closed or not
  • bugs have to be fixed within 30 days of opening
  • be honest, be accountable
  • honest
    • if is important, fix it
    • never defer bugs
    • don’t fix it? Better to close it
  • accountable
    • increase accountability of decision makers
    • need to communicate the decision to whoever reported bug
    • what if product quality decreases? -> decision maker must justify the decision
• advantages
  • easier prioritisation
  • team morale improved
  • reporter satisfaction increased
  • reduced technical debt
• risks
  • velocity was reduced
  • no explanation about why a bug was closed -> need to explain and communicate decisions
  • reduced reporting on bugs -> communicate honestly
• Rollout of the strategy
  • involve the stakeholders
  • obtain engagement and support from executives
  • support the first steps attentively
  • empower key team members
  • define metrics and communicate their trends regularly
• Metrics
  • trend of number of open bugs
  • ratio of “won’t fix” bugs vs fixed bugs
  • bugs that breach the “within 30 days” policy
  • average time to fix bugs
  • average time to decide if a bug should be fixed or not
• Results
  • definitely an improvement, but some areas still need work: good to know where
• Improvements still needed
  • reduce time to decision
  • improve quality of reported bugs
  • challenge the requirements -> find bugs earlier, at requirements time
  • trigger alerts when bugs are stuck in same state
• Qualitative results
  • mindset changed; bug fixing is a new habit
  • ownership and accountability by product owner etc.
  • awareness of quality
  • strategy is so well accepted in the company that people now make memes about it

“Dear Diary: How to remote onboard and get productive???”; Jan Reimann

• changed company during pandemic; was hard to onboard at the new company completely remotely
• problems:
  • being completely oneself
  • extroverted vs introverted
  • how do others perceive me?
  • what are cultural specifics to the company?
    • are there safe areas?
    • implicit cultural
  • lack of trust
  • hard to get to know other team members
  • how to demonstrate skills?
  • lack of spontaneity
    • have to schedule meetings
    • reading between the lines is harder
    • no body language to read
    • asynchronous communication
• goals:
  • get to know all teams in company
  • let others understand where you come from, what your background is, what your interests are, etc.
  • get productive quickly
• solutions:
  • communication
    • key to everything
    • explain
      • … yourself
      • … your home situation (so people know when you are available)
      • … you are there (and are going to stay; people need to know when they can count on you)
    • listen
      • … to everyone (everyone can help you/has something to contribute)
      • … and understand (try to understand what people have told you; don’t be afraid not to understand something; don’t be scared to ask questions when new)
      • … culture buddy (someone to aid the new person up to speed and help them through the “culture jungle”)
    • ask
      • … for clarification
      • … and leave comfort zone
      • … “if you are able to draw it; you’ve understood it” -> try to draw a diagram of something as a communication aid to see if you’ve understood concepts/architecture/product etc.
  • transparency
    • show
      • what and how you work
      • potential connections
      • e.g. work-with-me blockers (signal to colleagues to work with him)
      • e.g. daily into slack (put tasks for the day into the slack channel; informs team members what’s happening, can get feedback from colleagues)
    • engage
      • and be a role model
      • and add value
      • e.g. “hospitation” (like a student teacher sitting at the back of a class taking notes and learning how to be a teacher; sitting in on meetings of new group to learn how things work)
      • e.g. TDD ensembles (doing tasks in a team session; also doing katas in a team session)
  • visibility
    • establish
      • thematic connections
      • e.g. book club
    • contribute
      • to company culture
      • e.g. take ownership (if you have an idea of how to add to company culture)
      • e.g. just do it
      • e.g. play together (having fun together; helps team building)
      • e.g. align your work with company goals

Keynote: “It is time for Toxic Leaders to come out of their closet”; Raj Subrameyer

• had been a toxic leader in an organisation
• faced his own shame at his behaviour
• keep these four steps in mind as a manager
  • run 1:1 meetings with direct reports
  • show vulnerability
  • put ego aside and listen
  • take notes
• afterwards reflect and work out what needs to change to stop being a toxic leader
• how to leaders become toxic?
  • can happen very gradually
• who is a toxic leader?
  • someone who abuses the leader-follower relationship causing the org to be worse off after they interacted with them
• how do you know you are a toxic leader?
  • constantly compare with others
  • self worth is driven by latest results
  • cannot see other’s success
  • constantly stressed, anxious and depressed
  • deteriorating mental and physical health
  • become secretive
  • withhold info to ensure success
  • lack proper sleep
  • constant fear of being ignored
  • take credit for other’s work
  • more concerned about yourself than others
• but no-one is 100% toxic!
• environments that lead to toxic leadership:
  • stress
  • emotional numbness
  • insecurity
• life shapes itself based on things you say yes to and the things you say no to
• your insecurity is your worst enemy
• patterns to look out for
  • becoming defensive
  • have last word on everything
  • … (missed points on rest of slide)
• how to get out of being a toxic leader:
  • be open and honest about your problems
  • notice patterns and triggers
  • open up to a trusted person
  • show that you are vulnerable
  • put ego aside
  • shift mindest from fame and money to serving and impacting people
• how do organisations reduce toxic leadership?
  • symptoms:
    • workplace bullying
    • counterproductive behaviour
    • psychological distress
    • depression and burnout
  • solutions:
    • psychological safety
      • make it a priority
      • facilitate everyone speaking up
      • establish norms for handling failure
      • create space for new ideas (even wild ones)
      • embrace productive conflict
    • pay to quit programs (pay people, e.g. one month’s salary, to leave if they want to quit the job)
      • gives financial security to leaver
      • company stops losing money on reduced productivity
    • anonymous help hotline
    • have open conversations
• has infoq article about toxic leadership

Agile Testing Days - Day 2

Keynote: “Agile Comes with a Responsibility for Sustainability.”; Jutta Eckstein

• what is sustainability?
• it’s not only the environment, but also social/people, economic/profit, environment/planet
• sustainability is the combination of these factors
• think holistically if one thinks sustainably
• software can solve some sustainability issues, but more computing uses more and more energy; projected 21% of total energy consumption by 2030
• Google is planning on being climate-neutral by 2030
• ethics in computing is important; what is the system being used for? Is the system making a positive contribution to society.
• usability and accessibility are important aspects when creating software
• important to have multiple perspectives
• important to recognise own biases (personal and company-wide)
• explore environmental/social/economic issues in retrospectives. What is the impact of the software on environment, society and economic issues?
• one can use the agile principles as a guide with the goal to provide value sustainably
• change the questions in and around the product:
  • people - watch out for diversity and inclusion and any bias
  • planet - accountability for overall energy consumption
  • profit - responsibility for eventual harm of the product

“Breaking into the IT World remotely without being broken”; Thomas Spengler

• when new to IT, there is lots of new terminology to learn; makes communication difficult at first
• getting up to speed when being completely remote is difficult
• remote work amplified challenges
• safe spaces helpful as part of onboarding process
• patience important; take your time as a new employee to learn and orient yourself
• when onboarding company has a plan of where the new empolyee “should be” at this stage of their introductory training
• feelings of success when onboarding are important for further development and morale
• remote/online socialising and bonding important for new team members; helps with employee retention

“Testing the Pyramid with Toggles”; Marc Vila

• toggles in this context means feature toggles
• goal: to always have software in a releasable state
• was inspired by book: The DevOps Handbook
• uses trunk based dev at his company
• CI pipeline triggered by each commit
• if pipeline passes, then code goes to production
• each test pyramid level is executed by a different stage of the pipeline
• feature toggles:
  • allows “dark launching”
  • use in this context four categories: release, test, ops, permission
  • benefits:
    • decouple deploy and release
    • control rollouts
    • kill switch
    • enable test in production
    • can commit often on work-in-progress features
• test toggled feature with toggle on and with toggle off (except for new tests added for the toggled feature)
• need to handle missing or extra features in tests
  • turn on/off feature-specific tests given toggle state
• can cause a combinatoric explosion in number of tests
• how to test several toggles?
  • define types (feature status) for toggles: implementable, deployable, releaseable
• have “can deploy”, “can release”, “can implement” states
  • can deploy (can go to prod, but not be on)
  • can release (prod, and is on)
  • can implement (able to be added to the code base)
• use toggle types to structure which tests are run
  • only the implementable toggles within dev
  • only deployable toggles within staging
  • only releaseable toggles within prod
• when removing toggles, have tests in the end that the toggle has been removed from the code
• slides: https://bit.ly/3qsx9IF

Keynote: “Limitless within our boundaries”; João Proença

• too much choice can be a hindrance
• restrictions can be more liberating and one can be more easily creative
• the paradox of choice:
  • having too many choices blocks decisions
  • but we tend to prefer having many choices
• The Paradox of Choice: TED talk by Barry Schwarz
• decision fatigue: when one has lots of decisions, one can get tired just from simple decision taking
• no choice is bad
• some choice is better than no choice
• a lot of choice is not necessarily better than just some choice
• … yet we tend to want more choice!
• interrelationship between creativity and focus
• too much choice can be detrimental to creativity
• constraints give focus
• can apply constraints to the exercise of deciding on features/tests/quality aspects to give more focus and decide on the most important things to do. Makes things more achievable.
• setting up the right kinds of constraints sets up the team to help guarantee success

“The lone tester survival guide”; Patryk Oleksyk

• as a lone tester one has work challenges
  • won’t have all info
  • will have lots of work to do
  • will have to be flexible
• can use different kinds of testing approaches with different teams/organisations
  • traditional (waterfall)
  • rapid software testing
  • modern testing
• can use mind maps for:
  • product coverage outline
  • quality criteria (what quality the org wants to achieve)
  • feature spec
  • note risks
  • personally, in order to get up to speed on a project
• five orders of ignorance -> knowledge discovery framework
• questioning skills are important
  • asking why to get to the root cause
  • 5Ws to create test ideas
  • use leading questions to prompt respondant to elaborate
• use knowledge of experts to help level up
• have consistent daily habits; helps in work life and non-work life

“The relationship between quality and testing”; Janet Gregory

• quality is everyone’s responsibility. Improve quality, you automatically improve productivity - M. Edwards Deming
• if you focus on quality, the speed will come. If you focus on speed, the quality will be lost
• quality is vague and often hard to describe
• when thinking about quality, we think about the customer, the process that went into the creation of the product, and the product’s value
• how to build quality into product?
• development lifecycle:
  • discover -> plan -> understand -> build -> deploy -> release -> observe -> learn -> (return to start)
  • part of holistic testing
• discover
  • test the ideas
  • determine value
• plan
  • identify risks
  • test assumptions
  • create testable stories
• understand
  • example mapping
  • prototypes
  • BDD
  • determine what to observe/monitor
• build
  • automate tests
  • instrument code
  • test stories and features
• deploy
  • test infrastructure
  • run automated tests
  • test the pipeline
  • test quality attrs
  • test the system
• release
  • test in production
  • use feature toggles or blue/green env
• observe
  • how do customers use product?
  • monitor for warnings and errors
• learn
  • hypothesise and adapt
• need to think about what kind of testing one is doing at each stage of the delivery pipeline and what is appropriate at which stage
• what are quality attrs? What is their priority?
  • reliability
  • accessibility
  • performance
  • scalability
  • flexibility
• grow culture of quality in team
  • whole team commitment to quality
  • make connections
  • share knowledge
  • iterate in small batches
  • visualise your work
  • experiment

Keynote: “How to be an Ally to Non-binary Folk in Tech”; Bruce Hughes

• listening to how people describe themselves is a good skill to have
• to be an ally, you need to empathise and understand
• if you make a mistake, correct yourself and move on
• learn from your mistakes

Agile Testing Days - Day 3

Keynote: “How to nudge your way through agile testing”; Ard Kramer

• cognitive bias codex
• nuding is an intervention that is easy and cheap to avoid
• nudge theory
• nudging comes from:
  • behavioural economics
  • psychology
  • political theory
  • marketing sales
• nudge criteria
  • choice
  • intervention
  • easy to avoid
• ethics:
  • is nudging manipulation?
  • need to think about why one is trying to influence people
• can we nudge people to make better choices about quality and testing in our organisation?
• default option:
  • need to think about how one uses default options in e.g. websites
  • can influence people by specifying a default option
• commitment through consistency
  • small consistent steps to gain commitment in an activity
• anchoring
  • e.g. specifying a high price initially (as an anchor) and then specifying a smaller price -> people think new price is cheaper than the original. The initial price is an anchor.
  • also possible to have characteristics as anchors. E.g. positive or negative characteristics first in a list of characteristics.
  • can use an anchor to set tone in a meeting; the first perspective can set the tone for an entire meeting
• decoy effect
  • have a decoy option (e.g. a really expensive option) to make the actually wanted option (that one wants the customer to buy) look more reasonable.
• the Zeigarnik effect
  • the ability to remember incomplete tasks better than complete tasks
  • also the reason songs get stuck on one’s head
  • can use this effect by leaving a task open at the end of the day and plan to finish it the next day: one thinks about it subconciously over night -> also can work for team tasks
• activate unconscious behaviour
  • try to change people’s behaviour by subtle hints to trigger the desired behaviour
  • e.g. silent zones in trains -> make them look like libraries (i.e. be quiet)
  • e.g. make boxes for delivered bikes look like plasma TVs so delivery people don’t throw them around

“Growing an Experiment-driven Quality Culture”; Elisabeth Hocke

• want to improve quality (quality is value)
• had a lack of transparency across organisation
• a lack of knowledge in organisation
• wanted to scale up company a lot and wanted to address these issues
• decided to run experiments to work out how to achieve this
• important to decide when an experiment is finished
• made clear that wanted to learn from experiments
• gained awareness of what testing and quality is
• improved knowledge within the teams
• everyday business got in the way of the experiments
• not everyone was on board
• quality is a team responsibility
• there is a need to make a system change so that behaviours contributing to quality are rewarded
• team needs to work together to focus on quality: it’s a whole-team issue

“Multi-dimensional Test automation pyramid for microservices”; Shuqi Jiang

• at a new company, it turned out that devs had their own tests and testers had their own tests and teams hadn’t communicated between each other what they were doing
• also unclear what API tests/integration tests were and what they meant in each context
• also had manual tests which took days to run
• UI tests ran hours
• could decouple some UI tests from backend so that subset of UI tests ran much quicker
• also important to decouple tests from domains and services in backend tests to isolate things and speed tests up and increase test coverage
• can also define contracts between different services and then define tests between them -> “contract tests”
• test automation pyramid is a guide to aid discussion about automated tasks, how to prioritise them and make automation visible to the whole team
• now test objectives are clear; terminology now consistent across org
• when want to change: be clear about what want to achieve
• need to focus on testing the boundaries of the domain -> aids decoupling and isolation
• trust low-level tests -> allows one to avoid unnecessary e2e and integration tests -> can focus on happy paths

Keynote: “The Tester’s Learning Toolkit”; Vera Baum

• to become an expert, one needs to apply deliberate practice
• one doesn’t have to become an expert; just doing your job is completely ok
• one can be an expert in a given domain; and one isn’t necessarily an expert in another (even if similar) domain
• one can only transfer general knowledge between domains
• how to level up from novice, to apprentice, to crafter and then to expert?
• novices
  • supporting novices
    • offer rules
    • monitoring
    • more teaching
    • training in self-observation
    • instructional feedback
  • be aware of:
    • too much information
    • lack of structure
  • training novice
    • go for the basics
    • teaching paths
    • look for help and feedback from a trainer
    • start with one topic to build initial structures
• apprentice
  • attributes
    • considerable amount of experience in coping with real situations
    • knowledge is situational
    • understanding of environment
  • supporting an apprentice
    • move from teaching goals to learning goals (apprentice drives learning more)
    • fading additional guidelines
    • start with one domain to build first structures
  • be aware of
    • too high complexity of tasks
    • not enough help with choosing learning goals
    • still relying on too many guidelines
  • training apprentice
    • join communities (e.g. conferences)
    • try to find the right talks
    • go for more specific literature (not only basic)
    • talk to others
    • other media (podcasts, videos, etc.)
• crafter
  • attributes
    • takes years to achieve this level
    • knowledge of variety of whole situations fed from practical experience
    • learning should lead to reaching overall goals
  • supporting a crafter
    • confront them with a variety of teaching and learning methods
    • variety of real life situations
    • can reflect on those situations
  • be aware of
    • staying too long in your comfort zone
    • not looking for feedback
  • training crafter
    • start training others
    • change job?
    • talking at conferences
    • look for conference talks of more experienced people
    • go for your own learning methodology (how to find own content etc.)
    • go for your own style
    • experiment
• expert
  • attributes
    • no more rules and guidelines are necessary; create them!
    • intuitive reaction to situations
    • non-analytical stage of performance (basically instintively know what to do)
  • supporting an expert
    • try to analyse their actions
    • encourage teaching
  • be aware of
    • training novices (a lot of experts shouldn’t train novices)
  • training experts
    • specific methodologies present themselves to expert
    • approach method for each situation
    • go into different field
• (self-)reflection is key

“Chaos engineering bottom up”; Bart Szulc

• Chaos engineering
• works with a distributed system
• can be stressful to work with if one of the systems has a failure, when other systems depend on them
• want to discover problems before they happen
• combination of services can cause unforeseen and unpredictable problems
• therefore wanted to implement chaos engineering in their team
• run experiments on system to see if it can survive the various scenarios, e.g. certain service failures and how that affects the entire system
• running experiments can also be stressful for people as well as the systems that are being tested

“Growing Quality from Culture in Testing Times”; Tom Young

• if have proper culture in team then quality will grow on its own
• high quality software can be created as a direct consequence of a focus on quality culture
• not being afraid to try; not being afraid to fail
• need buy-in from whole team and management to implement quality culture
• need a definition of “done” and a definition of “ready” for the product
• have a team charter: define common values in team
• team buy-in to charter and values from new members is a positive sign
• hill charts: good way to visualise risks in a project
• hill chart issues
• important to be mentally present at work
• delivering user value
• show up, deliver, be kind, repeat

Keynote: “What does the ‘Coach’ in ‘Quality Coach’ mean?”; Vernon Richards

• quality coaching: look at problems, try to mitigate problems
• how to accelerate individuals, teams, even the business
• active listening:
  • pay attention
  • withhold judgement
  • reflect
  • clarify
  • summarise
  • share
  • is focussed; try to understand how the person is feeling
  • builds trust and empathy
• classic traps when active listening
  • being uncomfortable with silences
  • being preoccupied with thoughts about your own life
  • thinking ahead about what you will do for the coachee after the session
  • wanting to ask questions to satisfy your own curiosity
  • feeling the need to take part and share your own experiences
  • feeling the need to “solve” things and have all the answers
  • thinking ahead to your next question
  • making snap judgements or assumptions about your client or their situation
• unconditional positive regard
• coach asks open questions so that the coachee thinks of solutions on their own
• teaching is opposite of coaching in the sense that teaching directs exactly what to do; coaching is non-directive: the coachee knows how to solve the problem(s), they just need the right questions to be asked
• mentoring is between teaching and coaching (in terms of directness)
• Egan’s 3-stage model
  • explore the situation -> what’s going on?
  • define the situation -> what do you want instead?
  • manage the situation -> what are you going to about it?
• use powerful questions to help the coachee gain insight
• Goleman’s 6 Leadership Styles