Down Under – to LetsTest Oz

This trip to a conference was one of many ‘first times’. First time to Australia, first time to passing the equator, first time in Asia, albeit only airports, first time driving on ’the other side’. Also the first time I saw a kakatoe flying in the wild. And the one I’m not sure about to be proud of, first time in a karaoke bar.

I planned some days before the conference. I had to recover from a 28 hour flight. The flight started in Amsterdam, via Kuala Lumpur and Jakarta to Sydney. I stayed for two days in Sydney. I met with some people over there, who were also staying over in Sydney before the move on to the conference site, The Fairmont Resort in Leura.

I drove with Henke Andersson and Carsten Feilberg to Leura. When we arrived, we met other people already being there. As always, hugging time. You see your friends again after a long time. But one thing hit me directly. The scenic surrounding of the resort was astonishing. The entrance, but also the view outside. The mountains.

Hall Fairmont ResortMountains Fairmont Resort

We decide to get the car and drove to a viewpoint, just to see the environment. Later that day it was diner and meeting up with other friends.

Mountains FairmontMountains FairmontMountains FairmontMountains Fairmont

Monday – Tutorial Day
Keynote : James Bach – How do I know I’m context driven
In his speach, James gave us some information on how to know that you are a context driven tester. We as testers always have an opinion. That is fine, but be ready to be challenged. Context driven testers most of the time respond with “it depends”. Explain where or what it depends on. We all know that good testing depends on skills. Skills you can learn. As a context driven tester you must be open to learn. The community respects you for that.
Another thing is that before you come up with a strategy, you must have understanding of the situation you’re dealing with. It is not always the case that the situation is not the clear. Sometimes it happens that you do bad work, but it can have good side effects.
Context driven testing is a paradigm (a world view), a community (the people), an approach (specific testing heuristics).

Tutorial : James Bach and Anne Marie Charrett – Coaching testers
This one day tutorial was given not only by James Bach, but also by Anne Marie Charrett. Both are well known for given coaching sessions to testers. They explain the process how they work with and that is was still not finished. The first question James asked was “What kind of coach are you?”. A drill sergeant, high standards person, grumpy. You must center yourself, knowing who you are. You need to know how to test to coach a tester. It is all about ‘expertise based coaching’, As a coach, as a tester. You have two identities.
The coaching space diagram depicts an arena within which coaching happens.
This arena consists of at least two people—the coach and student—and the elements of a coaching session wherein they interact. We talked about the elements

  • Context : The area where you work in. Anything outside of yourself that affects your choice
  • Self-Image : The current story of who you are.
  • Aspirations : Your motivating goals. You try to reach them, You still can do it
  • Expectations : Your critical requirements for yourself or others, they are non-negotiable. As a coach, you need to lower them first and raise them later, bit by bit.
  • Ability : Your ability to coach as well as your ability to test. What you can do and how well you do it.
  • Demeanor : You energy state as it appears to others. Attitude and emotions.
  • Self-evaluation : Your assessment of your own performance now and over time.
  • Coach : Your beliefs and biases about your student. Different in behavior to a woman compared to a man. How students should be coached? Deal with these biases

Important for a coaching session is energy. Without energy nothing much will happen. Much of a coaching session is about managing energy. In other words motivation. Dusting the coaching session, you have to read between the lines, what is their motivation. Cultivate the motivation with constructive responses. The purpose of “our coaching” is to set them free.
Sometimes you let them go, sometimes you interrupt. Manage the energy. Read it and evaluate it. Manipulate their aspirations. You can do better than this, I raise the expectations. Direction to where the motivation goes to. As a coach you can set the bar of expectation. See what the student does. You can coach a person that can transform your energy. (Million Dollar baby). You can as a coach give a task and observe (The Matrix). Note that coaching and training are totally different.
During the coaching session you will make make many observations. People who are self correcting, their learning process is going on. Get the student to be specific, go into details. Drilling down. Get the student to speak generally. Both types of speaking needs to be work on.
The tester has to explain themselves, what are you doing. Is the person you are coaching is better, move to a supportive role. Notice everything which is unusual. Is it specific, ask them to be more general. Is it too general, try them to be more specific. Ask questions with out them knowing all the answers. See them as follow up question of the answer which is given. (Socratic questions). Always be truthful to your student. Be positive.
When you have a coaching session, assign a task to the student. Assist him/her where needed. See where the discussion will lead to. We did two coaching sessions via Skype with another participant. You was not aware who that person was. In one coaching sessions you played the coaching role, in the other session you played the student role. When both sessions were finished, you get together with ‘your counterparts’ and discussed the session. Here we tried to find out if the session was having a positive direction, a neutral direction or a negative direction.

Tuesday – Conference, Day 1
Keynote : Keith Klain – How to talk to a CIO (about software testing)
Keith keynote was moved to this time slot, as Fiona Charles had to cancel. She had a cold and here voice let here down.
Keith talk was about his own experiences. Experiences about how to deal with top of the organization. With a CIO, you talk about budget and staffing. It is all about “something I want them to know” and “something I want them to do”. A CIO doesn’t care about your problems. He has his own problems. A CIO doesn’t talk about quality, so ask him about quality. Discuss it with him. We as testers are also bad about what we are doing from test point of view. You need to know what you are doing, the value of your work and the risks you are dealing with. Note that quality is NOT zero defects. We need to give the CIO confidence, because that is what he wants. We testers are deconstructive in a constructive SDLC (Software Development Life Cycle). If you talk to about testing, know your stuff. Keith showed a picture which covers it in its essence.


Know what you say, where you talk about. Be aware that a CIO is interested in issues and risks, not in bugs. You must learn to speak the language of the other expertise, in this case the language of the CIO. Avoid talking about yourself. Don’t even about the topic test itself. But on the other side, the way how a CIO runs his business is a great source of test ideas.
A CIO is surrounded with people he trust, where he can make decisions with. Gain credibility with who can influence decisions. Spent time with him/her.
To conclude; know what you outcome has to be when you talk to the CIO

Talk : Sigge Birgisson – QA as in Quality Assistance (not Quality Assurance)
Sigge started his talk with his position as a tester. He was most of the time the lone tester of the team. Currently in a team where the ration is 1 tester to 10 developers. He sees himself as now as a quality assistant. Assisting the team with test activities. But quality itself is owned by the team. You can not test quality in the product, you can build it in the product. So you focus on bug prevention in stead of bug detection. As a quality assistant, it is your task to get the developer fast to his goal, that is good software. That means a ‘person’ has to sit next to the developer.
As a team, we have to remind ourself frequently “what can go wrong”. Remind that with a quality mindset. When we start, we have a feature kick-off to talk about risk, scope of test in general, the appropriate testing. We have a QA kick-off before we start with development of the code. The developer knows what to do. QA kick-off are about decomposition of the different parts/states/interfaces of the feature.


Output from the QA kick-off sessions are the testing notes. When the code is delivered, we have several so called QA demo’s. Validation of what is build related to the test notes. Metrics, will be evaluated. What we discussed during QA kick-off and what we realised with development. We have retrospective disccussions to improve our QA kick-off session
What is tough about about quality assistant is that you have to know “all the things”. We have to coach smart people. Invest time in coaching developers in testing. Talk about risk early, often and on all levels. It is all about how to test, but NOT test (Think about this one)

ISO29119 discussion
This discussion was organized on short notice. Normally, Fiona Charles suppose to give here talk.
Four persons formed a panel. Henke Andersson, Ben Kelly, Keith Klain and. James Bach. Each did a 5 minute personal opinion about the topic. After that, it was open season. The audience was invited to ask questions to the members of the panel.
I did not make that many notes, only the numbers of the green, yellow and red cards. Yep, I was facilitating this session. One remark made by the audience was, should we make an anti-standard. Something like the Agile Manifesto. Got some member of the panel food for thought.

Talk : Martin Hynie – What’s In A Name? Experimenting with job titles.
Martin talked about an experiment (of 18 months) he conducted while he was working for a company related to the aircraft industry. They all work differently, but all use the same very complex software. We call this software ‘The Beast’. A lot of people needed to work with us on ‘The Beast’. Problem was that software testing was seen as a downstream activity. Development was not approachable. “No, you are testers”. It was just the word tester that kept the door closed. As an experiment, I decided to create a new group. I called it ‘Technology Research Group’. Note that is still a division of testing. Department head of Development comes to me; “Why is ‘Technology Research Group’ not involved in my projects”. We kept this group for 4 months operational. After that period we changed our names back to testers. No invitation anymore. Excluded from almost everything. Did the trick again. We created a team called ‘Business Analyst Group”. Next day, marketing and development wanted to work with us.
We were forming small groups for shared understanding, the so called ‘The Three Amigos’. A coder, a tester and a business analyst. We were creating new kinds of collaborations, ‘Trading Zones’

Trading Zones

In this way of working we found strange things. We all worked on the same, but each one has different values. We note that the approach of working had to change. We formed partnerships across the teams. One person in two groups, the business analyst.

Business Analyst

One person created common language, common understanding. We came information brokers, sort of centipedes of the organization. This business analyst person was somebody who worked for many years at the company. The strange thing was that he was recognized as a new person. Weird, because they were still the same persons only carrying a new job title. My group only that we were running the experiment, the rest of the organization did not. A reorganization stopped my experiment. The outcome of the experiment was that some job titles can work against you.

In the evening, I asked Rich Robinson if I could play the dice game with him. This was a real challenge. There were all kinds of dices on the table. Different in size, color, shape, you name it. With asking questions, you have to find out what the pattern is. This pattern is something Rich got from some local testers. It took me a while to crack it. The feedback I got was that I played it well. Note that it was not the game most of us played during the RST training. This game from Rich has several levels you have to pass as he explained later to me.

Wednesday – Conference, Day 2
Talk : Ben Kelly – What do you mean Agile Tester?
Ben directly popped the questions, “Is there a role for testing in Agile?” Yes there is, but there some points of attention. In an agile team, you as a tester have to focus on different kind of skills. You also focus on building the right thing, building it the right way. As a lone tester I was a person who was facilitating test between the two teams I was working with. While working with your developers, ask them simple questions. It is to avoid shallow agreements and hidden complexity. A question such as “What can go wrong with the user story?” Challenge them to think about it. When you pair with a developer to work on a user story, talk about the specifics. Ask about implementation. Deal with interesting observations. We use TDD, but in a different fashion. In our context it is Test Driven Design. You create a test which is later not used for regression purposes. These type of tests are created to help the developer to design/build the code.
In the scrum team, there must be a fundamental understanding of each others role. Ask the specialist for help when it is needed, when you are ‘stuck’. As a facilitator in the scrum team, you help other non-testers to do good testing. Main drive is to add more value to the team. Make your value visible in the team. You have more autonomy and flexibility as a tester. The whole team want to go to the final solution, ‘getting to done’.

Talk : Erik Petersen – A contextual framework for hiking testers
Erik gave an interactive presentation showing parts of websites to support his talk. With showing these websites, Erik triggered one of the important skills we have as an exploratory tester. Observation skills. Get your brain out of its comfort zone. As we have a thought experiment, describe what you see and then try to explain it for yourself. Try to build a model and experiment around that model. When we work within our model, try to think about it, try to understand it. Experiment around the model to try see if we can do more. During exploring, you describe the behavior in terms of the model. His framework is more or less covered in the picture I took from his presentation.


The last talk session of that day I did not participate. I sneaked out with Henke to go to the The Three Sisters. The rock formation standing in the glades at the city of Katoomba.


Keynote : Fiona Charles – The  Battle of Our Hearts and Minds
We as testers have to fight against a lot of misunderstanding. People are making statements like “Test is dead”. “You don’t look dead to me”, says Fiona. “Test is expensive, we must get rid of it”. “Yeah, sure”, reacted Fiona. Testing is all about value, not cost. When we dealing with uncertainty, we are dealing with reality. You can estimate, but that doesn’t tell anything. It is something we can hang on.
Test documentation is something which cost a lot of money, but what do we produce. Nothing. So don’t argue that the testers cost a lot of money. It is the documentation. We must go back to our principles, ‘What is testing?’ Know your stakeholders, know what they want. Write a test strategy such, that your stakeholder understands it. Make it a small document, not longer than 4 pages. Show your stakeholders that you have exercised your professional judgment the best way you can. We need to have a disciplined process to work with, one which add value. Things like skills, engagement, courage and disciple are needed as a tester. ….. Always question yourself.

Glad that Fiona could make it, giving this presentation. Her voice hold up, while she recovering from a cold. Wouldn’t miss it for the world.

Time to say goodbye. Most of the participant were leaving either to fly back home or to go to Sydney. I stayed at the resort. Luckily I was not the only one staying an extra night at the resort. We were with four of us. Terri Randle, Morris Ney, Anna Royzman and me. Also Kim Engel was with us, but she had to take the train in the evening. We had diner together. We talked about the the conference and other test related topics. That is what we do, unconferring but discussing about our profession. Kim had to go for her train as we went back to the resort.

To close, I want to bring up a tweet from Kim she sent out the other day. It was the blog from Colin Cherry, ‘How a Test Community Became a Family‘. Great blogpost. Reading it back again touched me. All the days it felt like a family gathering.
Big thanks to Anne Marie Charrett, David Greenlees and Henke Andersson. You did a amazing job organizing this. Hope there will be a sequel.

Note : Most of the blogpost is written on my way back home, a 23 hour flight from Sydney, via Singapore to Amsterdam.


2 Comments Add yours

  1. Hi Peter, Quite a extensive summary of your conference experience. Nice to read, seems you had a lot of inspiring moments. Great read

  2. Sigge says:

    Coming back to this and reading about my own talk is nice. Thank you for the good summary of the talk mate!

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s