Knowledge Management In Testing

You can find some screenshots and installation instructions in GitHub repository obsidian-km-testing-kit.

This is an ObsidianMD starter kit (vault) for testers and QA enthusiasts. The main 2 goals are:

Share my templates, findings and settings for ObsidianMD with other testers who also use that tool in their work.
To have a handy template in the cloud for myself. When I need to start a new testing project, I can download/clone this starter kit from GitHub. Without spending much time on settings and configurations I can quickly start using it for my testing needs in less than 5 min.

What is knowledge management in testing and why talk about it?

In this post, I wanted to share some of my findings related to testing (mostly exploratory), knowledge management and how I see the connection between them. I am still learning things, experimenting and tweaking my system, but It definitely feels so much better because

I believe I now have better organization of my work (it is more structured and “mindful”)
I was able to improve visibility, traceability and quality of my testing (I got some positive feedback from the team)
I have fun learning new things

There are probably a lot of definitions of knowledge management. I like to think about KM as a system that helps to solve problems which we face when dealing with large amount of information. That information is consumed, stored, searched, reused, shared, etc. Similarly when we talk about knowledge management in testing, we want to find the answers to some important questions which cover such aspects as:

test management approach? (do you test one feature per day, or you spread your time between multiple tasks and areas? how do you manage your time?)
testing method (what do you do to discover important risks, generate test ideas, avoid shallow testing?)
measuring progress (have you tested this area enough? can you explain why? )
reporting (what is the current progress on testing feature X, can you visualize it or present in a clear and concise form?)
collaboration (you tested feature Y a couple of months ago; can you share your notes with a new tester so that person can find all necessary resources, set up instructions and spend less time learning the product?)
traceability (do we have tests or for the requirement Z? can we trace the link between them and test results?)
visibility (what have you been testing last Monday? Can other team members see which cases/scenarios/areas you covered?)

Typical knowledge management issues I have always struggled with organizing files, folders, notes, bookmarks, being able to quickly find information I saved some time ago, organize tasks on a paper and logs. Here are some examples:

I can’t find the web link that I saved some time ago in my notes
I can’t find the file that I downloaded and saved some time ago in one of the folders
It is often hard to make a decision, in which folder should I put a new file because there can be different ways to structure folders (by date, by topic or area, by project etc.)
Multiple notes related to the same area are scattered across various folders and sometimes get lost.
I have too many folders and folder names are inconsistent
Some files are buried deep inside folder structure, so it is hard to find when need them.

Knowledge management issues in testing. When I started to work as a tester. The situation I described above got even worse. I was consuming lots of information: courses, articles, blog posts, slack and Twitter discussions. How should I manage this information? Then there is information related to your everyday testing activities. How do you capture, organize, store, share and retrieve it? Some examples

Test Management. What is your plan for this day? What form do you use to capture it. Will you be able to answer that plan in 2 weeks? In 1 year?
Test Planning. How do you write a concise and actionable test plan and test strategy? Will other people get value from it? Is it accessible and easy to read, navigate for other team members?
Logs. How do you capture which areas you have tested and which haven’t?
End-to-end testing/Reference list What are the features of the product you are testing? Can you see the overview (full list) which you could use to plan and prioritize tests?
Exploratory testing How do you capture random findings, clarifications, updates on acceptance criteria. For me, they usually come from discussions with developers, Github comments or Slack messages.
Exploratory testing It would be great to have some kind of checklists or guides (I learned later that they are called heuristics) which you can use when testing standard components like text field, input form, modal or web page? Is there a way to organize them effectively and retrieve them quickly when needed?
Measuring progress Which metrics can I use to visualize.
Productivity. Spending too much time on documents, notes and reporting will leave you very little time for actual testing. On the other hand, if you do not pay attention to planning, making notes, tracking your progress, reporting you will most likely will be missing and forget important things. Your job will not be transparent for others. You will not be able to learn from your mistakes.

Evolution of my exploratory testing approach and note-taking

There were lots of things that I learned along the way, but I’d like to mention 3 main sources which influenced my journey:

Rapid Software Testing methodology and Session-Based Test Management in particular. These are mostly publications, videos and resources created by such exploratory testing enthusiasts and advocates (James Bach, Michael Bolton, Elizabeth Hendrickson, Louise Gibbs and others)
ObsidianMD + Markdown. ObsidianMD is a personal knowledge management tool, markdown editor.
One particular article written by Alan Richardson aka EveilTester.

Here are other milestones in my test exploratory and knowledge management journey:

Random note taking and unstructured testing. Notes from learning and occasional testing notes are scattered across google docs, local txt files. Notes and files are usually very long and not structured.
Conflict. It is impossible to remember all incoming information. I should make notes and capture it somehow, but there is a problem. It takes too much time and it is hard to find that information later.
Markdown. I discovered Typora – markdown editor with minimalistic, very clean UI. It is very easy to use and markdown is so powerful. This is what works for me.
Rapid software testing and exploratory testing. The more I read, the more often I see people referring to James Bach and RST and emphasize the importance of exploratory testing. It turns out that checklists and templates that I was trying to build are called “heuristics” and other testers use them all the time!
Session-Based Test Management. I came across this article by y Jonathan Bach Session-Based Test Management – Satisfice, Inc.. Initially, it seems to complicated, time-consuming and not practical. But there is something in it. I should give it a try.
ObsidianMD. Your “IDE for thought”. It allows you to have a powerful knowledge base on top of a local folder of plain text Markdown files. For free. You have total control. You do not depend on a particular app or proprietary technology. There are hundreds of markdown editors, and markdown is future-proof. Everything is connected: graph, backlinks, outgoing links. With 25 core plugins and 200+ community plugins, it is so flexible and extensible.
ObsidianMD community. Wow! It turned out there is a whole universe around note-taking. There are so many things and solutions that are actually solving those problems that I mentioned before.
- Atomic notes,
- LYT,
- Zettelkasten,
- P.A.R.A.,
- maps of content,
- evergreen notes,
- various workflows,
- templates,
- script automatons,
- querying data in your knowledge base using syntax similar to MySQL,
- customizing appearance. And that knowledge is constantly growing and evolving as people continue to share ideas and build tools.
First attempt of implementing SBTM and RST using Obsidian. I can now use templates to quickly generate various types of notes: daily or monthly log, session note (report), feature note. I can use Git and sync my local knowledge base with GitHub repo!
Javascript and more sophisticated workflows. What makes obsidian is so flexible is that you can combine in your templates plain text and javascript that you can use to implement conditional logic, insert automatically dates, process user’s input, create modals, buttons, etc. I started to look up Javascript snippets and examples and introduce write my own scripts.
Find the balance. At some point, I started to realize that there should be a balance between how much time you spend tinkering with a favourite tool (sometimes it is so addictive you cannot resist) and how much time you do the real work.
Mindmaps and metadata. The deeper I dived into ObsidianMD ecosystem, the more I loved it. I started to use mind maps to capture test activities. There are at least 2 Obsidian plugins. The next step was adding YAML metadata to my notes which opens lots of possibilities for searching data, manipulating notes using templates, using plugins and custom scripts to build reports and analyze data.
The draft of my first test plan. I spent enormous time writing my first test plan for the product I am testing. No one asked me to do it but I knew I have to start building that skill. It is probably far from perfect but there are some things I really like about the final result.
- The test plan reflects what I actually do during my day-to-day testing and it guides my activity.
- It is written using markdown so there are lots of links that naturally connect different sections of the test plan.
- I used Git for version control. Since it is written in markdown it allowed me to publish the entire test plan on GitHub in a private repo. I can share it with my team members. Readme file represents a table of contents and all parts are connected by links.
- The test plan itself is a part of my knowledge base for this project. Logs, test sessions, templates, heuristics, charters – everything has its dedicated place and is interconnected. When the test plan or knowledge base is updated, all I need to do is commit changes and push to remote repo.

Does this make you a better tester?

Well, that’s a good question and I want you to answer it.

Like in many cases, it probably depends on context. We are all different. We work in different environments and companies.

For someone, what I shared in this article will not work at all. Others might find something helpful and borrow some parts. Who knows, maybe there will be testers who will be inspired by my experience and use it to build their own “frameworks” or systems.

There is no such tool that could make you a better tester. The tool is only an instrument. It can help you be a better tester. Your experience, hard work, mindset and talent – probably are major factors.

In my opinion, building knowledge management as a tester can benefit you because

There is a cumulative effect. This is your “second brain”. Imagine you can instantly pull any atomic note on testing session you did in the past. You can use the findings, examples and techniques from all of your previous work. That can be powerful.
Leveraging technology can save you lots of time. Plugins and your custom scripts can help you to achieve better test coverage, apply techniques and methodologies developed by other people (e.g. heuristics, test charters, SBTM), produce better reports, improve traceability and of test activities and make them more transparent.

Some visual examples.

I am planning to record several short videos to capture some typical workflows to show how I use ObsidianMD in my testing.

For now, I created a GitHub repository obsidian-km-testing-kit. This is a starter kit of Obsidian vault that can be cloned and be used to quickly start a new test project. The main benefits are

Real examples make it easier to understand the structure that I use.
Preinstalled plugins already have the necessary settings.
A set of templates and scripts that will allow creating sessions, features and daily notes.

You can find some screenshots and installation instructions in GitHub obsidian-km-testing-kit repository.

Maybe I will record short videos showing some typical workflows. Please let me know in the comments.

Overview of my system: folder structure, file naming
Workflow: daily note, reporting
Workflow: test session
Workflow: heuristics
Workflow: charters
Workflow: feature note
Workflow: Measuring progress
Workflow: test plan

Things to keep in mind

Security. There are security risks related to using community plugin. Obsidian uses Electron engine, which also has its own issues and vulnerabilities
ObsidianMD is 100% free for personal use. Commercial license is required for commercial use.