What is A/B Testing
Nowadays a lot of people talk about A/B Testing, especially in the tech industry, but what it is? why do we want to do A/B Testing? what is the outcome or what is the impact?
Often people confuse that words and it became hyped especially in the product team and marketing team, and it is not surprising that a lot of people actually confuse what is A/B Testing.
In simple words A/B Testing is a practice to compare something, but what do we want to compare then?
For example, there is a marketing person that have the question “what if we change this button color to red? Does it do something? What is the outcome?”, the marketing person will wondering the outcome and do a lot of assumptions, maybe do his own interview with some of his/her colleagues and ask “hey which you like between this old button and the new red button” some of them will think I like the new red button, some of them will say I like the old button, and actually you're already doing the A/B testing, so A/B testing is basically the practice when you want to do a comparison between two or more stuff
How to do A/B Testing?
In the tech industry most likely you won't be able to do manual A/B testing like the example that I mentioned above at the scale, you can do 5-10 people but you cannot do like 1000++ with that method, so to do that in the scale you might need help from other people.
Usually, the marketing or product guy will start with the question, what comparison they will do? where they will do it?
Why do we need to do A/B Testing?
In product management, as a product manager, you always need to think that you do not know everything, and everything needs to be learned by you, this is the case when you want to do development, you need to think that if you want to develop something you should think that you don't know about it and don't know about your user.
So that's why A/B Testing is necessary to do, you never know what is the outcome of your development whether it will increase the conversion or it will decrease the conversion.
To mitigate if the development is fail or the development cause the conversion rate to decrease we do A/B Testing so in this phase we actually do experiment with a small number of customer “Segmentation”. In this phase, we only want to do a test experiment whether the changes actually do good or it will do harm. Usually, the experiment will happen in the span of weeks or months depending on the needs.
If the result is good, we can implement the changes to all customers, if the changes actually do decrease the conversion we just can revert it back.
With A/B Testing we didn't risk losing a lot of conversions due to just experimenting with a small number of customers.
Example
I have a hypothesis that changing the button from grey to red might increase the conversion rate.
 |
|---|
| AB Testing Example |
So how we can do it? first of all, usually changing the button directly is too risky, because we don't know whether the changes will do good or bad (decrease the conversion rate) in this case we do need to do A/B Testing which includes the old design (grey button) with a new design (red button)
We might need help from the developer to implement this feature so there will be 2 buttons that have grey and red buttons.
If we can't have 2 same functional buttons on the same page then how we can show the different buttons to the customer? In this case, we should specify who will get the old button and who will get the new button.
We can do the experiment is several weeks, or a month if needed.
Conclusion
In conclusion, A/B testing is a practice commonly used in the tech industry, particularly in product management and marketing, to compare and evaluate different versions or variations of a product, feature, or design. Its purpose is to gather data-driven insights and determine the impact of changes on user behavior, conversion rates, or other relevant metrics. By conducting controlled experiments with a subset of users or customers, A/B testing allows companies to make informed decisions and mitigate risks associated with implementing changes at a larger scale. It provides a systematic approach to validate hypotheses, optimize user experiences, and drive improvements based on measurable outcomes. Ultimately, A/B testing helps teams make more data-informed decisions, enhance product performance, and achieve desired business objectives.