{"id":142,"date":"2021-07-14T14:24:24","date_gmt":"2021-07-14T14:24:24","guid":{"rendered":"https:\/\/www.figpii.com\/blog\/?p=142"},"modified":"2024-06-17T01:10:16","modified_gmt":"2024-06-17T01:10:16","slug":"quality-assurance-for-ab-testing-best-practices","status":"publish","type":"post","link":"https:\/\/www.figpii.com\/blog\/quality-assurance-for-ab-testing-best-practices\/","title":{"rendered":"Quality Assurance for AB Testing, Best Practices."},"content":{"rendered":"\r\n<p><a href=\"https:\/\/www.figpii.com\/blog\/ab-testing-guide\/\">A\/B tests<\/a> are frequently used to prove or disprove hypotheses that emanate from research.\u00a0<\/p><div id=\"ez-toc-container\" class=\"ez-toc-v2_0_74 ez-toc-wrap-left counter-hierarchy ez-toc-counter ez-toc-transparent ez-toc-container-direction\">\n<div class=\"ez-toc-title-container\">\n<p class=\"ez-toc-title\" style=\"cursor:inherit\">Table of Contents<\/p>\n<span class=\"ez-toc-title-toggle\"><a href=\"#\" class=\"ez-toc-pull-right ez-toc-btn ez-toc-btn-xs ez-toc-btn-default ez-toc-toggle\" aria-label=\"Toggle Table of Content\"><span class=\"ez-toc-js-icon-con\"><span class=\"\"><span class=\"eztoc-hide\" style=\"display:none;\">Toggle<\/span><span class=\"ez-toc-icon-toggle-span\"><svg style=\"fill: #999;color:#999\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" class=\"list-377408\" width=\"20px\" height=\"20px\" viewBox=\"0 0 24 24\" fill=\"none\"><path d=\"M6 6H4v2h2V6zm14 0H8v2h12V6zM4 11h2v2H4v-2zm16 0H8v2h12v-2zM4 16h2v2H4v-2zm16 0H8v2h12v-2z\" fill=\"currentColor\"><\/path><\/svg><svg style=\"fill: #999;color:#999\" class=\"arrow-unsorted-368013\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" width=\"10px\" height=\"10px\" viewBox=\"0 0 24 24\" version=\"1.2\" baseProfile=\"tiny\"><path d=\"M18.2 9.3l-6.2-6.3-6.2 6.3c-.2.2-.3.4-.3.7s.1.5.3.7c.2.2.4.3.7.3h11c.3 0 .5-.1.7-.3.2-.2.3-.5.3-.7s-.1-.5-.3-.7zM5.8 14.7l6.2 6.3 6.2-6.3c.2-.2.3-.5.3-.7s-.1-.5-.3-.7c-.2-.2-.4-.3-.7-.3h-11c-.3 0-.5.1-.7.3-.2.2-.3.5-.3.7s.1.5.3.7z\"\/><\/svg><\/span><\/span><\/span><\/a><\/span><\/div>\n<nav><ul class='ez-toc-list ez-toc-list-level-1 eztoc-toggle-hide-by-default' ><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-1\" href=\"https:\/\/www.figpii.com\/blog\/quality-assurance-for-ab-testing-best-practices\/#Well-written_documentation\" >Well-written documentation<\/a><ul class='ez-toc-list-level-5' ><li class='ez-toc-heading-level-5'><ul class='ez-toc-list-level-5' ><li class='ez-toc-heading-level-5'><ul class='ez-toc-list-level-5' ><li class='ez-toc-heading-level-5'><a class=\"ez-toc-link ez-toc-heading-2\" href=\"https:\/\/www.figpii.com\/blog\/quality-assurance-for-ab-testing-best-practices\/#1_What_are_your_target_pages\" >1. What are your target pages?<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-5'><a class=\"ez-toc-link ez-toc-heading-3\" href=\"https:\/\/www.figpii.com\/blog\/quality-assurance-for-ab-testing-best-practices\/#2_Problem_statement_and_your_solution_to_the_problem\" >2. Problem statement and your solution to the problem<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-5'><a class=\"ez-toc-link ez-toc-heading-4\" href=\"https:\/\/www.figpii.com\/blog\/quality-assurance-for-ab-testing-best-practices\/#3_List_your_variations\" >3. List your variations<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-5'><a class=\"ez-toc-link ez-toc-heading-5\" href=\"https:\/\/www.figpii.com\/blog\/quality-assurance-for-ab-testing-best-practices\/#4_Test_conditions_and_exclusions\" >4. Test conditions and exclusions<\/a><\/li><\/ul><\/li><\/ul><\/li><\/ul><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-6\" href=\"https:\/\/www.figpii.com\/blog\/quality-assurance-for-ab-testing-best-practices\/#An_Adequate_design\" >An Adequate design<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-7\" href=\"https:\/\/www.figpii.com\/blog\/quality-assurance-for-ab-testing-best-practices\/#What_to_test\" >What to test?<\/a><ul class='ez-toc-list-level-5' ><li class='ez-toc-heading-level-5'><ul class='ez-toc-list-level-5' ><li class='ez-toc-heading-level-5'><ul class='ez-toc-list-level-5' ><li class='ez-toc-heading-level-5'><a class=\"ez-toc-link ez-toc-heading-8\" href=\"https:\/\/www.figpii.com\/blog\/quality-assurance-for-ab-testing-best-practices\/#1_Is_it_even_loading\" >1. Is it even loading?<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-5'><a class=\"ez-toc-link ez-toc-heading-9\" href=\"https:\/\/www.figpii.com\/blog\/quality-assurance-for-ab-testing-best-practices\/#2_Does_is_look_right\" >2. Does is look right?<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-5'><a class=\"ez-toc-link ez-toc-heading-10\" href=\"https:\/\/www.figpii.com\/blog\/quality-assurance-for-ab-testing-best-practices\/#3_The_big_F_Functionality\" >3. The big F; Functionality<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-5'><a class=\"ez-toc-link ez-toc-heading-11\" href=\"https:\/\/www.figpii.com\/blog\/quality-assurance-for-ab-testing-best-practices\/#4_Are_you_breaking_existing_elements\" >4. Are you breaking existing elements?<\/a><\/li><\/ul><\/li><\/ul><\/li><\/ul><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-12\" href=\"https:\/\/www.figpii.com\/blog\/quality-assurance-for-ab-testing-best-practices\/#Design_details_that_went_unnoticed\" >Design details that went unnoticed!<\/a><ul class='ez-toc-list-level-5' ><li class='ez-toc-heading-level-5'><ul class='ez-toc-list-level-5' ><li class='ez-toc-heading-level-5'><ul class='ez-toc-list-level-5' ><li class='ez-toc-heading-level-5'><a class=\"ez-toc-link ez-toc-heading-13\" href=\"https:\/\/www.figpii.com\/blog\/quality-assurance-for-ab-testing-best-practices\/#1_Page_Ruler_Redux_Free\" >1. Page Ruler Redux (Free)<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-5'><a class=\"ez-toc-link ez-toc-heading-14\" href=\"https:\/\/www.figpii.com\/blog\/quality-assurance-for-ab-testing-best-practices\/#2_Fonts_Ninja_Free\" >2. Fonts Ninja (Free)<\/a><\/li><\/ul><\/li><\/ul><\/li><\/ul><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-15\" href=\"https:\/\/www.figpii.com\/blog\/quality-assurance-for-ab-testing-best-practices\/#Should_every_item_identified_in_the_QA_process_be_fixed\" >Should every item identified in the QA process be fixed?<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-16\" href=\"https:\/\/www.figpii.com\/blog\/quality-assurance-for-ab-testing-best-practices\/#The_final_check_%E2%80%94_Who_QAs_the_QA\" >The final check \u2014 Who QA\u2019s the QA?<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-17\" href=\"https:\/\/www.figpii.com\/blog\/quality-assurance-for-ab-testing-best-practices\/#Conclusion_Its_never_enough\" >Conclusion: It&#8217;s never enough!<\/a><ul class='ez-toc-list-level-4' ><li class='ez-toc-heading-level-4'><ul class='ez-toc-list-level-4' ><li class='ez-toc-heading-level-4'><a class=\"ez-toc-link ez-toc-heading-18\" href=\"https:\/\/www.figpii.com\/blog\/quality-assurance-for-ab-testing-best-practices\/#If_a_test_is_breaking_your_website_does_it_matter_how_fast_it_was_developed\" >If a test is breaking your website, does it matter how fast it was developed?<\/a><ul class='ez-toc-list-level-5' ><li class='ez-toc-heading-level-5'><a class=\"ez-toc-link ez-toc-heading-19\" href=\"https:\/\/www.figpii.com\/blog\/quality-assurance-for-ab-testing-best-practices\/#1Dont_run_your_experiments_for_too_long\" >1.Don&#8217;t run your experiments for too long<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-5'><a class=\"ez-toc-link ez-toc-heading-20\" href=\"https:\/\/www.figpii.com\/blog\/quality-assurance-for-ab-testing-best-practices\/#2_Test_test_and_keep_on_testing\" >2. Test, test, and keep on testing<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-5'><a class=\"ez-toc-link ez-toc-heading-21\" href=\"https:\/\/www.figpii.com\/blog\/quality-assurance-for-ab-testing-best-practices\/#3_Plan_for_issues_on_live\" >3. Plan for issues on live<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-5'><a class=\"ez-toc-link ez-toc-heading-22\" href=\"https:\/\/www.figpii.com\/blog\/quality-assurance-for-ab-testing-best-practices\/#4_Be_honest_with_your_team_experimentation_is_not_a_game_of_blame\" >4. Be honest with your team; experimentation is not a game of blame<\/a><\/li><\/ul><\/li><\/ul><\/li><\/ul><\/li><\/ul><\/nav><\/div>\n\r\n\r\n\r\n\r\n<p>These can vary from tactical tests such as interface changes to strategic tests such as implementing a new checkout process on an eCommerce site or a completely new dashboard for a SaaS site.\u00a0<\/p>\r\n\r\n\r\n\r\n<p>Over the years, we have noticed that the QA process can determine the success or failure of your tests.\u00a0\u00a0<\/p>\r\n\r\n\r\n\r\n<p>Visual and functional errors in the variants will distort your test results if you don\u2019t do proper QA.<\/p>\r\n\r\n\r\n\r\n<blockquote class=\"wp-block-quote is-layout-flow wp-block-quote-is-layout-flow\">\r\n<p>QA is an area we have invested in tremendously.<br \/>You want to make sure that you\u2019re not deploying a test that is going to cause a breakdown in production. Each and every test we launch is QA\u2019ed by multiple people on our side, and before we even launch any test, we always ask our clients to review those tests before we deploy them.\u201d<\/p>\r\n<cite><strong><em>\u2014\u00a0<\/em><\/strong>Khalid Saleh<\/cite><\/blockquote>\r\n\r\n\r\n\r\n<p>It\u2019s essential to have a standardized QA process in a high testing velocity program since you will be launching dozens and dozens of tests and making constant changes on your website.<\/p>\r\n\r\n\r\n\r\n<p>&nbsp;<\/p>\r\n\r\n\r\n\r\n<div class=\"wp-block-ht-block-toc  is-style-outline htoc htoc--position-wide toc-list-style-plain\" data-htoc-state=\"expanded\"><span class=\"htoc__title\"><span class=\"ht_toc_title\">Table of Contents<\/span><\/span>\r\n<div class=\"htoc__itemswrap\">\r\n<ul class=\"ht_toc_list\">\r\n<li class=\"\"><a href=\"#htoc-well-written-documentation\">Well-written documentation<\/a>\r\n<ul class=\"ht_toc_child_list\">\r\n<li class=\"\"><a href=\"#htoc-1-what-are-your-target-pages\">1. What are your target pages?<\/a><\/li>\r\n<li class=\"\"><a href=\"#htoc-2-problem-statement-and-your-solution-to-the-problem\">2. Problem statement and your solution to the problem<\/a><\/li>\r\n<li class=\"\"><a href=\"#htoc-3-list-your-variations\">3. List your variations<\/a><\/li>\r\n<li class=\"\"><a href=\"#htoc-4-test-conditions-and-exclusions\">4. Test conditions and exclusions<\/a><\/li>\r\n<\/ul>\r\n<\/li>\r\n<li class=\"\"><a href=\"#htoc-an-adequate-design\">An Adequate design<\/a><\/li>\r\n<li class=\"\"><a href=\"#htoc-what-to-test\">What to test?<\/a>\r\n<ul class=\"ht_toc_child_list\">\r\n<li class=\"\"><a href=\"#htoc-1-is-it-even-loading\">1. Is it even loading?<\/a><\/li>\r\n<li class=\"\"><a href=\"#htoc-2-does-is-look-right\">2. Does is look right?<\/a><\/li>\r\n<li class=\"\"><a href=\"#htoc-3-the-big-f-functionality\">3. The big F; Functionality<\/a><\/li>\r\n<li class=\"\"><a href=\"#htoc-4-are-you-breaking-existing-elements\">4. Are you breaking existing elements?<\/a><\/li>\r\n<\/ul>\r\n<\/li>\r\n<li class=\"\"><a href=\"#htoc-design-details-that-went-unnoticed\">Design details that went unnoticed!<\/a>\r\n<ul class=\"ht_toc_child_list\">\r\n<li class=\"\"><a href=\"#htoc-1-page-ruler-redux-free\">1. Page Ruler Redux (Free)<\/a><\/li>\r\n<li class=\"\"><a href=\"#htoc-2-fonts-ninja-free\">2. Fonts Ninja (Free)<\/a><\/li>\r\n<\/ul>\r\n<\/li>\r\n<li class=\"\"><a href=\"#htoc-should-every-item-identified-in-the-qa-process-be-fixed\">Should every item identified in the QA process be fixed?<\/a><\/li>\r\n<li class=\"\"><a href=\"#htoc-the-final-check-who-qa-s-the-qa\">The final check \u2014 Who QA\u2019s the QA?<\/a><\/li>\r\n<li class=\"\"><a href=\"#htoc-conclusion-it-s-never-enough\">Conclusion: It&#8217;s never enough!<\/a>\r\n<ul class=\"ht_toc_child_list\">\r\n<li class=\"\"><a href=\"#h.oroqjr609woi\">If a test is breaking your website, does it matter how fast it was developed?<\/a>\r\n<ul class=\"ht_toc_child_list\">\r\n<li class=\"\"><a href=\"#h.5xxeu38obi53\">1.Don&#8217;t run your experiments for too long<\/a><\/li>\r\n<li class=\"\"><a href=\"#h.m9w02k2fjv3l\">2. Test, test, and keep on testing<\/a><\/li>\r\n<li class=\"\"><a href=\"#h.2hfyvc17zmq\">3. Plan for issues on live<\/a><\/li>\r\n<li class=\"\"><a href=\"#h.sorxvbw1mtv6\">4. Be honest with your team; experimentation is not a game of blame<\/a><\/li>\r\n<\/ul>\r\n<\/li>\r\n<\/ul>\r\n<\/li>\r\n<\/ul>\r\n<\/div>\r\n<\/div>\r\n\r\n\r\n\r\n<p>&nbsp;<\/p>\r\n\r\n\r\n\r\n<h2 class=\"wp-block-heading\" id=\"htoc-well-written-documentation\"><span class=\"ez-toc-section\" id=\"Well-written_documentation\"><\/span>Well-written documentation<span class=\"ez-toc-section-end\"><\/span><\/h2>\r\n\r\n\r\n\r\n<p>For the QA team to be effective, they must be involved in the whole Conversion Rate Optimization process.\u00a0<\/p>\r\n\r\n\r\n\r\n<p>Ideally, they&#8217;re briefed together with the development team;\u00a0<\/p>\r\n\r\n\r\n\r\n<p>of course, well-written documentation and design are an acceptable substitute.\u00a0<\/p>\r\n\r\n\r\n\r\n<p>Those initial conversations are where the team gets to understand the point of view of the CRO specialist and the concerns of the development team.<\/p>\r\n\r\n\r\n\r\n<h5 class=\"wp-block-heading\" id=\"htoc-1-what-are-your-target-pages\"><span class=\"ez-toc-section\" id=\"1_What_are_your_target_pages\"><\/span>1. What are your target pages?<span class=\"ez-toc-section-end\"><\/span><\/h5>\r\n\r\n\r\n\r\n<p>Most of the time, a test is only meant for specific pages \u2013 like certain product pages or the cart page \u2013 and that&#8217;s the first thing the team needs to know.\u00a0<\/p>\r\n\r\n\r\n\r\n<p>We need to know where to test.<\/p>\r\n\r\n\r\n\r\n<figure class=\"wp-block-table\">\r\n<table>\r\n<tbody>\r\n<tr>\r\n<td><strong>Page Type<\/strong><\/td>\r\n<td><strong>Targeting Type<\/strong><\/td>\r\n<td class=\"has-text-align-left\" data-align=\"left\"><strong>Rule<\/strong><\/td>\r\n<td><strong>Example<\/strong><\/td>\r\n<\/tr>\r\n<tr>\r\n<td>Product Page<\/td>\r\n<td>Starts with<\/td>\r\n<td class=\"has-text-align-left\" data-align=\"left\">\/product-<\/td>\r\n<td>figpii.com\/product-example<\/td>\r\n<\/tr>\r\n<tr>\r\n<td>Blog Pages<\/td>\r\n<td>Contains<\/td>\r\n<td class=\"has-text-align-left\" data-align=\"left\">\/blog<\/td>\r\n<td>figpii.com\/blog<\/td>\r\n<\/tr>\r\n<tr>\r\n<td>Pricing Page<\/td>\r\n<td>Simple Match<\/td>\r\n<td class=\"has-text-align-left\" data-align=\"left\">\/packages<\/td>\r\n<td>figpii.com\/packages<\/td>\r\n<\/tr>\r\n<\/tbody>\r\n<\/table>\r\n<\/figure>\r\n\r\n\r\n\r\n<p>&nbsp;<\/p>\r\n\r\n\r\n\r\n<h5 class=\"wp-block-heading\" id=\"htoc-2-problem-statement-and-your-solution-to-the-problem\"><span class=\"ez-toc-section\" id=\"2_Problem_statement_and_your_solution_to_the_problem\"><\/span><strong>2. Problem statement and your solution to the problem<\/strong><span class=\"ez-toc-section-end\"><\/span><\/h5>\r\n\r\n\r\n\r\n<p>Next, there needs to be a good explanation of what we are trying to solve. Are we trying to help users navigate the website more easily, find out information about the product above the fold, or something much more complicated?\u00a0<\/p>\r\n\r\n\r\n\r\n<p>For example, here&#8217;s a real-life example of a test meant to smooth the navigation across different categories:<\/p>\r\n\r\n\r\n\r\n<p>By providing a sticky menu showing all the subcategories, we can help visitors find the category they&#8217;d like to visit immediately without much scrolling.<\/p>\r\n\r\n\r\n\r\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter wp-image-144\" src=\"https:\/\/www.figpii.com\/blog\/wp-content\/uploads\/2021\/06\/image5-wpp1628797996704-1024x537.png\" alt=\"A\/B Testing two different versions of a product description page\" width=\"1024\" height=\"537\" srcset=\"https:\/\/www.figpii.com\/blog\/wp-content\/uploads\/2021\/06\/image5-wpp1628797996704-1024x537.png 1024w, https:\/\/www.figpii.com\/blog\/wp-content\/uploads\/2021\/06\/image5-wpp1628797996704-300x157.png 300w, https:\/\/www.figpii.com\/blog\/wp-content\/uploads\/2021\/06\/image5-wpp1628797996704.png 1379w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/>\r\n<figcaption>Toggling between the various categories is not very easy for the customer. By providing a sticky menu with all subcategories, we can help visitors find the categories they\u2019d like to visit immediately without much scrolling.<\/figcaption>\r\n<\/figure>\r\n\r\n\r\n\r\n<h5 class=\"wp-block-heading\" id=\"htoc-3-list-your-variations\"><span class=\"ez-toc-section\" id=\"3_List_your_variations\"><\/span>3. List your variations<span class=\"ez-toc-section-end\"><\/span><\/h5>\r\n\r\n\r\n\r\n<p>Listing your variations can help the QA team understand the differences between them.\u00a0<\/p>\r\n\r\n\r\n\r\n<p>Including a short description and design examples \u2013 highlighting the differences between variations \u2013 is a must if you have over two variations.<\/p>\r\n\r\n\r\n\r\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter wp-image-145\" src=\"https:\/\/www.figpii.com\/blog\/wp-content\/uploads\/2021\/06\/image2-wpp1628798028491-1024x711.png\" alt=\"A\/B Testing two different versions of a product description page\" width=\"1024\" height=\"711\" srcset=\"https:\/\/www.figpii.com\/blog\/wp-content\/uploads\/2021\/06\/image2-wpp1628798028491-1024x711.png 1024w, https:\/\/www.figpii.com\/blog\/wp-content\/uploads\/2021\/06\/image2-wpp1628798028491-300x208.png 300w, https:\/\/www.figpii.com\/blog\/wp-content\/uploads\/2021\/06\/image2-wpp1628798028491.png 1255w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/>\r\n<figcaption>V1: At the bottom of the page on the sticky clickable banner, the following\u00a0<strong>product pictures with text<\/strong>\u00a0will appear<br \/><br \/>V2: At the bottom of the page on the sticky clickable banner, the following\u00a0<strong>text (without product pictures)\u00a0<\/strong>will appear<\/figcaption>\r\n<\/figure>\r\n\r\n\r\n\r\n<h5 class=\"wp-block-heading\" id=\"htoc-4-test-conditions-and-exclusions\"><span class=\"ez-toc-section\" id=\"4_Test_conditions_and_exclusions\"><\/span>4. Test conditions and exclusions<span class=\"ez-toc-section-end\"><\/span><\/h5>\r\n\r\n\r\n\r\n<p>Is this a mobile or a desktop test?<\/p>\r\n\r\n\r\n\r\n<p>Are you planning on only targeting users coming from a specific source? Or perhaps users from a certain coming from a particular country?<\/p>\r\n\r\n\r\n\r\n<p>Documenting these questions with an adequate answer can help reduce back and forth between team members and managers.<\/p>\r\n\r\n\r\n\r\n<p>&nbsp;<\/p>\r\n\r\n\r\n\r\n<h2 class=\"wp-block-heading\" id=\"htoc-an-adequate-design\"><span class=\"ez-toc-section\" id=\"An_Adequate_design\"><\/span>An Adequate design<span class=\"ez-toc-section-end\"><\/span><\/h2>\r\n\r\n\r\n\r\n<p>A good design for an AB test outlines all the differences between all your variations and the original.\u00a0<\/p>\r\n\r\n\r\n\r\n<p>Websites look different on different screens, make sure to keep that in mind.<\/p>\r\n\r\n\r\n\r\n<p>Depending on your test that might mean providing the images in different resolutions, rearranging the elements to make them fit on smaller screens, etc.<\/p>\r\n\r\n\r\n\r\n<p>&nbsp;<\/p>\r\n\r\n\r\n\r\n<p>After understanding the logic and look of the test, the QA team can get to work!<\/p>\r\n\r\n\r\n\r\n<p>If the QA team consists of different members, split the work and make sure there&#8217;s an understanding of who&#8217;s doing what!<\/p>\r\n\r\n\r\n\r\n<h2 class=\"wp-block-heading\" id=\"htoc-what-to-test\"><span class=\"ez-toc-section\" id=\"What_to_test\"><\/span>What to test?<span class=\"ez-toc-section-end\"><\/span><\/h2>\r\n\r\n\r\n\r\n<h5 class=\"wp-block-heading\" id=\"htoc-1-is-it-even-loading\"><span class=\"ez-toc-section\" id=\"1_Is_it_even_loading\"><\/span>1. Is it even loading?<span class=\"ez-toc-section-end\"><\/span><\/h5>\r\n\r\n\r\n\r\n<p>This is the first thing you need to check. If the test was intended for a certain type of page, quickly check as many as you can just to make sure the test is loading on different kinds of that page type.<\/p>\r\n\r\n\r\n\r\n<h5 class=\"wp-block-heading\" id=\"htoc-2-does-is-look-right\"><span class=\"ez-toc-section\" id=\"2_Does_is_look_right\"><\/span>2. Does is look right?<span class=\"ez-toc-section-end\"><\/span><\/h5>\r\n\r\n\r\n\r\n<p>For this step, we strongly suggest having a multi-monitor setup. Open the design on one and the website on another and compare them.<\/p>\r\n\r\n\r\n\r\n<p>Check different screen sizes and browsers, for desktop tests we suggest the following screen sizes:<\/p>\r\n\r\n\r\n\r\n<ul class=\"wp-block-list\">\r\n<li>920 px<\/li>\r\n<li>1024 px (Most common on iPads which might be detected as desktop by your testing engine)<\/li>\r\n<li>1360 px (Common on older PCs)<\/li>\r\n<li>1440 px (Common on MacBooks)<\/li>\r\n<li>1920 px<\/li>\r\n<li>2048 px<\/li>\r\n<\/ul>\r\n\r\n\r\n\r\n<p>And for mobile tests check on the following screen sizes:<\/p>\r\n\r\n\r\n\r\n<ul class=\"wp-block-list\">\r\n<li>320 px<\/li>\r\n<li>375 px<\/li>\r\n<li>414 px<\/li>\r\n<li>768 px<\/li>\r\n<\/ul>\r\n\r\n\r\n\r\n<p><strong>Pro tip: <\/strong>Head over to your Google Analytics and take a look at your audience technology.<\/p>\r\n\r\n\r\n\r\n<p>There you can find the most common screen sizes among your visitors.<\/p>\r\n\r\n\r\n\r\n<p>And in terms of which browsers to test on, The most popular current browsers are Microsoft Edge, Google Chrome, Mozilla Firefox, and Apple\u2019s Safari.<\/p>\r\n\r\n\r\n\r\n<h5 class=\"wp-block-heading\" id=\"htoc-3-the-big-f-functionality\"><span class=\"ez-toc-section\" id=\"3_The_big_F_Functionality\"><\/span>3. The big F; F<strong>unctionality<\/strong><span class=\"ez-toc-section-end\"><\/span><\/h5>\r\n\r\n\r\n\r\n<p>What happens when users interact with your test?<\/p>\r\n\r\n\r\n\r\n<p>You are the user now! Put yourself in everyone&#8217;s shoes, test every element that exists on the page but especially the elements you&#8217;ve added\/changed.<\/p>\r\n\r\n\r\n\r\n<h5 class=\"wp-block-heading\" id=\"htoc-4-are-you-breaking-existing-elements\"><span class=\"ez-toc-section\" id=\"4_Are_you_breaking_existing_elements\"><\/span>4. Are you breaking existing elements?<span class=\"ez-toc-section-end\"><\/span><\/h5>\r\n\r\n\r\n\r\n<p>Your new elements might be just fine, but what about the rest of the page? Do they look like before and are they working as well?<\/p>\r\n\r\n\r\n\r\n<p>For example, your new <strong>Fast Checkout <\/strong>shouldn&#8217;t be breaking the hamburger menu! I know it sounds strange but it happens.<\/p>\r\n\r\n\r\n\r\n<h2 class=\"wp-block-heading\" id=\"htoc-design-details-that-went-unnoticed\"><span class=\"ez-toc-section\" id=\"Design_details_that_went_unnoticed\"><\/span>Design details that went unnoticed!<span class=\"ez-toc-section-end\"><\/span><\/h2>\r\n\r\n\r\n\r\n<p>Some developers don\u2019t have a designer\u2019s eye, which means many tiny details go unnoticed.\u00a0<\/p>\r\n\r\n\r\n\r\n<p>The QA team has to ensure that every detail was implemented as outlined by the test plan.<\/p>\r\n\r\n\r\n\r\n<p>This includes checking the font sizes, all the different spacings and alignments, etc.\u00a0<\/p>\r\n\r\n\r\n\r\n<p>There are 2 chrome extensions that our team absolutely loves!<\/p>\r\n\r\n\r\n\r\n<h5 class=\"wp-block-heading\" id=\"htoc-1-page-ruler-redux-free\"><span class=\"ez-toc-section\" id=\"1_Page_Ruler_Redux_Free\"><\/span>1. <a href=\"https:\/\/chrome.google.com\/webstore\/detail\/page-ruler-redux\/giejhjebcalaheckengmchjekofhhmal\">Page Ruler Redux<\/a> (Free)<span class=\"ez-toc-section-end\"><\/span><\/h5>\r\n\r\n\r\n\r\n<figure class=\"wp-block-image\"><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter\" src=\"https:\/\/lh3.googleusercontent.com\/qKCnbO8gPURCkwHp-WmYeCnWE7VZ9QPswjrk16mzAYpWmBmHgae-0gQclsAdkYUDTQ9_pZBZgVnhgRhJjLVNn8twug=w640-h400-e365-rj-sc0x00ffffff\" alt=\"Page ruler redux\" width=\"640\" height=\"400\" \/><\/figure>\r\n\r\n\r\n\r\n<p>Using this extension you can measure the spacing between different elements to make sure they are the same as outlined on the design.<\/p>\r\n\r\n\r\n\r\n<p><strong>Tip:<\/strong> Check all the elements that were affected by your test. Scroll up and down the page, if the CSS rules were not set correctly your test might be affecting elements that are not related to your test.<\/p>\r\n\r\n\r\n\r\n<h5 class=\"wp-block-heading\" id=\"htoc-2-fonts-ninja-free\"><span class=\"ez-toc-section\" id=\"2_Fonts_Ninja_Free\"><\/span>2. <a href=\"https:\/\/chrome.google.com\/webstore\/detail\/fonts-ninja\/eljapbgkmlngdpckoiiibecpemleclhh\">Fonts Ninja<\/a> (Free)<span class=\"ez-toc-section-end\"><\/span><\/h5>\r\n\r\n\r\n\r\n<figure class=\"wp-block-image\"><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter\" src=\"https:\/\/lh3.googleusercontent.com\/kg7gIfPSU65iF-P7KTXyQXvluSsJ_rIoLiP_SK58auJka8ExdjEVduixJoZ1vKXuW376jai0Eejtp1vaDOJHDTOhGgo=w640-h400-e365-rj-sc0x00ffffff\" alt=\"Fonts Ninja\" width=\"640\" height=\"400\" \/><\/figure>\r\n\r\n\r\n\r\n<p>Use Fonts Ninja to check how your test is affecting the texts on the targeted pages. Again because of the way CSS works your test might be messing your paragraphs, headlines, and more!<\/p>\r\n\r\n\r\n\r\n<h2 class=\"wp-block-heading\" id=\"htoc-should-every-item-identified-in-the-qa-process-be-fixed\"><span class=\"ez-toc-section\" id=\"Should_every_item_identified_in_the_QA_process_be_fixed\"><\/span>Should every item identified in the QA process be fixed?<span class=\"ez-toc-section-end\"><\/span><\/h2>\r\n\r\n\r\n\r\n<p>Sometimes the CRO specialist might make the call that this issue is not worth spending time on.<\/p>\r\n\r\n\r\n\r\n<p>After all, we&#8217;re testing an idea to know if it will cause an increase in the conversion rate.<\/p>\r\n\r\n\r\n\r\n<p>If the test is a winner, it should be hardcoded to the site, and if it&#8217;s a loser, it&#8217;ll just be paused and removed from the site.<\/p>\r\n\r\n\r\n\r\n<p>Minor design issues can be ignored \u2013 if they don&#8217;t break the user experience.<\/p>\r\n\r\n\r\n\r\n<p>Keep an eye on your visitors, launch session recordings for the duration of your test, and monitor how users are interacting with your variations.<\/p>\r\n\r\n\r\n\r\n<h2 class=\"wp-block-heading\" id=\"htoc-the-final-check-who-qa-s-the-qa\"><span class=\"ez-toc-section\" id=\"The_final_check_%E2%80%94_Who_QAs_the_QA\"><\/span>The final check \u2014 Who QA\u2019s the QA?<span class=\"ez-toc-section-end\"><\/span><\/h2>\r\n\r\n\r\n\r\n<p>After the QA team is done with checking an experiment, then comes the last ghoul;<\/p>\r\n\r\n\r\n\r\n<p>In our experience, the whole team should participate in the final check, including CRO specialists, Designers, Developers, and even the project manager.<\/p>\r\n\r\n\r\n\r\n<p>Each member plays a vital role in the final check. Each of them brings their experience and specialty to the process.<\/p>\r\n\r\n\r\n\r\n<blockquote class=\"wp-block-quote is-layout-flow wp-block-quote-is-layout-flow\">\r\n<p><em>Launch and validate any test in a QA or UAT environment that closely resembles your production site. Only after data is validated in a QA environment, then it is pushed to production servers.\u201d\u00a0<\/em><\/p>\r\n<cite><strong><em>\u2014\u00a0<\/em><\/strong>Khalid Saleh<\/cite><\/blockquote>\r\n\r\n\r\n\r\n<h2 class=\"wp-block-heading\" id=\"htoc-conclusion-it-s-never-enough\"><span class=\"ez-toc-section\" id=\"Conclusion_Its_never_enough\"><\/span>Conclusion: It&#8217;s never enough!<span class=\"ez-toc-section-end\"><\/span><\/h2>\r\n\r\n\r\n\r\n<h4 class=\"wp-block-heading\" id=\"h.oroqjr609woi\"><span class=\"ez-toc-section\" id=\"If_a_test_is_breaking_your_website_does_it_matter_how_fast_it_was_developed\"><\/span>If a test is breaking your website, does it matter how fast it was developed?<span class=\"ez-toc-section-end\"><\/span><\/h4>\r\n\r\n\r\n\r\n<p>No matter how much time you and your team spend QAing your experiments, problems are going to arise down the road. Let&#8217;s talk about a few solutions to reduce the number of unexpected issues after your experiment goes into productions.<\/p>\r\n\r\n\r\n\r\n<h5 class=\"wp-block-heading\" id=\"h.5xxeu38obi53\"><span class=\"ez-toc-section\" id=\"1Dont_run_your_experiments_for_too_long\"><\/span><strong>1.<a href=\"https:\/\/www.google.com\/url?q=https:\/\/www.invespcro.com\/blog\/how-long-should-you-run-an-ab-test-for\/&amp;sa=D&amp;source=editors&amp;ust=1626114852000000&amp;usg=AOvVaw2n9JCfpdP7xJvZ8lo-3Oyb\">Don&#8217;t run your experiments for too long<\/a><\/strong><span class=\"ez-toc-section-end\"><\/span><\/h5>\r\n\r\n\r\n\r\n<p>AB tests are meant to prove or disprove a hypothesis, not to make updates to your website.<\/p>\r\n\r\n\r\n\r\n<p>This is especially the case if an external team is developing the AB test.<\/p>\r\n\r\n\r\n\r\n<p>Most of the time, internal and external teams fail in communicating updates to the site&#8217;s structure, and any update to the site by the internal team will break your AB test!<\/p>\r\n\r\n\r\n\r\n<blockquote class=\"wp-block-quote is-layout-flow wp-block-quote-is-layout-flow\">\r\n<p><em>Figure out how long you need to run your test to get conclusive results, communicate well with your team to hold back any upcoming updates to the site<\/em>.<strong><em>&#8220;<\/em><\/strong><\/p>\r\n<cite><strong><em>\u2014\u00a0Hatice Kaya<\/em><\/strong><\/cite><\/blockquote>\r\n\r\n\r\n\r\n<h5 class=\"wp-block-heading\" id=\"h.m9w02k2fjv3l\"><span class=\"ez-toc-section\" id=\"2_Test_test_and_keep_on_testing\"><\/span><strong>2. Test, test, and keep on testing<\/strong><span class=\"ez-toc-section-end\"><\/span><\/h5>\r\n\r\n\r\n\r\n<p>It&#8217;s never enough. Educate your developers on how to do the initial QA, ask them to do a thorough and complete QA before they deliver.<\/p>\r\n\r\n\r\n\r\n<p>Have more than one QA team member testing your experience.<\/p>\r\n\r\n\r\n\r\n<p>Different people test differently, and that&#8217;s going to be a significant advantage for you!<\/p>\r\n\r\n\r\n\r\n<blockquote class=\"wp-block-quote is-layout-flow wp-block-quote-is-layout-flow\">\r\n<p><em>There are many risks, and one I can think about is the implementation of the test. Rolling out an experiment without doing Quality Assurance. We once ran an experiment where we changed the priority order of a list of offers for our customers. When the experiment was rolled out, the swipe to go down the page wasn&#8217;t working. There was a bug. And because of that bug, we saw a pretty big negative impact.&#8221;\u00a0<\/em><\/p>\r\n<cite><strong><em>\u2014\u00a0<a href=\"https:\/\/www.google.com\/url?q=https:\/\/www.invespcro.com\/blog\/expert-answers-to-9-tough-cro-questions\/&amp;sa=D&amp;source=editors&amp;ust=1626114852001000&amp;usg=AOvVaw01vkhW0uAx62oRHueeFofX\">Chad<\/a><\/em><\/strong><\/cite><\/blockquote>\r\n\r\n\r\n\r\n<p>Before pushing to live, make sure your CRO specialist and designer do a quick QA to make sure the experience of the test on the site is what they had initially planned.<\/p>\r\n\r\n\r\n\r\n<h5 class=\"wp-block-heading\" id=\"h.2hfyvc17zmq\"><span class=\"ez-toc-section\" id=\"3_Plan_for_issues_on_live\"><\/span><strong>3. Plan for issues on live<\/strong><span class=\"ez-toc-section-end\"><\/span><\/h5>\r\n\r\n\r\n\r\n<p>This is a must for all AB tests. When you discover that a bug within your variations slipped through all the testing (this will happen!), determine what impact the data pollution had on your test.<\/p>\r\n\r\n\r\n\r\n<p>In most cases, we have to stop the test, fix the bug, and flush the test data..<\/p>\r\n\r\n\r\n\r\n<p>It&#8217;s never a good idea to launch the test on Fridays.<\/p>\r\n\r\n\r\n\r\n<p>Leave them for the beginning of the week, Most of the time, the issues are minor and can be fixed within an hour, but if you notice the problem on Sunday (or\u00a0<strong>God forbids\u00a0<\/strong>the client to report the issue to you), you won&#8217;t be able to fix it on the spot.<\/p>\r\n\r\n\r\n\r\n<h5 class=\"wp-block-heading\" id=\"h.sorxvbw1mtv6\"><span class=\"ez-toc-section\" id=\"4_Be_honest_with_your_team_experimentation_is_not_a_game_of_blame\"><\/span><strong>4. Be honest with your team; experimentation is not a game of blame<\/strong><span class=\"ez-toc-section-end\"><\/span><\/h5>\r\n\r\n\r\n\r\n<p>There&#8217;s nothing more important than having open and respectful communication between your team members.<strong>\u00a0<\/strong>Please don&#8217;t blame your team members for the issues, things happen, and we&#8217;re all human.<\/p>\r\n","protected":false},"excerpt":{"rendered":"<p>A\/B tests are frequently used to prove or disprove hypotheses that emanate from research.\u00a0 These can vary from tactical tests such as interface changes to strategic tests such as implementing a new checkout process on an eCommerce site or a completely new dashboard for a SaaS site.\u00a0 Over the years, we have noticed that the<\/p>\n","protected":false},"author":3,"featured_media":144,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_eb_attr":"","footnotes":""},"categories":[2],"tags":[],"class_list":{"0":"post-142","1":"post","2":"type-post","3":"status-publish","4":"format-standard","5":"has-post-thumbnail","7":"category-ab-testing"},"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v25.3.1 - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>Quality Assurance for AB Testing, Best Practices. - FigPii blog<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.figpii.com\/blog\/quality-assurance-for-ab-testing-best-practices\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Quality Assurance for AB Testing, Best Practices. - FigPii blog\" \/>\n<meta property=\"og:description\" content=\"A\/B tests are frequently used to prove or disprove hypotheses that emanate from research.\u00a0 These can vary from tactical tests such as interface changes to strategic tests such as implementing a new checkout process on an eCommerce site or a completely new dashboard for a SaaS site.\u00a0 Over the years, we have noticed that the\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.figpii.com\/blog\/quality-assurance-for-ab-testing-best-practices\/\" \/>\n<meta property=\"og:site_name\" content=\"FigPii blog\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/figpii.aii\/\" \/>\n<meta property=\"article:published_time\" content=\"2021-07-14T14:24:24+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2024-06-17T01:10:16+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/www.figpii.com\/blog\/wp-content\/uploads\/2021\/06\/image5-wpp1628797996704.png\" \/>\n\t<meta property=\"og:image:width\" content=\"1379\" \/>\n\t<meta property=\"og:image:height\" content=\"723\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/png\" \/>\n<meta name=\"author\" content=\"Reza\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@reza_jalaei_\" \/>\n<meta name=\"twitter:site\" content=\"@figpii\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Reza\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"10 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/www.figpii.com\/blog\/quality-assurance-for-ab-testing-best-practices\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/www.figpii.com\/blog\/quality-assurance-for-ab-testing-best-practices\/\"},\"author\":{\"name\":\"Reza\",\"@id\":\"https:\/\/www.figpii.com\/blog\/#\/schema\/person\/c93c764d4e661184693d704990fed3e9\"},\"headline\":\"Quality Assurance for AB Testing, Best Practices.\",\"datePublished\":\"2021-07-14T14:24:24+00:00\",\"dateModified\":\"2024-06-17T01:10:16+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/www.figpii.com\/blog\/quality-assurance-for-ab-testing-best-practices\/\"},\"wordCount\":1960,\"publisher\":{\"@id\":\"https:\/\/www.figpii.com\/blog\/#organization\"},\"image\":{\"@id\":\"https:\/\/www.figpii.com\/blog\/quality-assurance-for-ab-testing-best-practices\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/www.figpii.com\/blog\/wp-content\/uploads\/2021\/06\/image5-wpp1628797996704.png\",\"articleSection\":[\"AB testing\"],\"inLanguage\":\"en-US\"},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/www.figpii.com\/blog\/quality-assurance-for-ab-testing-best-practices\/\",\"url\":\"https:\/\/www.figpii.com\/blog\/quality-assurance-for-ab-testing-best-practices\/\",\"name\":\"Quality Assurance for AB Testing, Best Practices. - FigPii blog\",\"isPartOf\":{\"@id\":\"https:\/\/www.figpii.com\/blog\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/www.figpii.com\/blog\/quality-assurance-for-ab-testing-best-practices\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/www.figpii.com\/blog\/quality-assurance-for-ab-testing-best-practices\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/www.figpii.com\/blog\/wp-content\/uploads\/2021\/06\/image5-wpp1628797996704.png\",\"datePublished\":\"2021-07-14T14:24:24+00:00\",\"dateModified\":\"2024-06-17T01:10:16+00:00\",\"breadcrumb\":{\"@id\":\"https:\/\/www.figpii.com\/blog\/quality-assurance-for-ab-testing-best-practices\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/www.figpii.com\/blog\/quality-assurance-for-ab-testing-best-practices\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.figpii.com\/blog\/quality-assurance-for-ab-testing-best-practices\/#primaryimage\",\"url\":\"https:\/\/www.figpii.com\/blog\/wp-content\/uploads\/2021\/06\/image5-wpp1628797996704.png\",\"contentUrl\":\"https:\/\/www.figpii.com\/blog\/wp-content\/uploads\/2021\/06\/image5-wpp1628797996704.png\",\"width\":1379,\"height\":723},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/www.figpii.com\/blog\/quality-assurance-for-ab-testing-best-practices\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/www.figpii.com\/blog\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Quality Assurance for AB Testing, Best Practices.\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/www.figpii.com\/blog\/#website\",\"url\":\"https:\/\/www.figpii.com\/blog\/\",\"name\":\"FigPii blog\",\"description\":\"Let it marinate\",\"publisher\":{\"@id\":\"https:\/\/www.figpii.com\/blog\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/www.figpii.com\/blog\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/www.figpii.com\/blog\/#organization\",\"name\":\"FigPii\",\"url\":\"https:\/\/www.figpii.com\/blog\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.figpii.com\/blog\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/www.figpii.com\/blog\/wp-content\/uploads\/2023\/09\/figpii-logo-purple.png\",\"contentUrl\":\"https:\/\/www.figpii.com\/blog\/wp-content\/uploads\/2023\/09\/figpii-logo-purple.png\",\"width\":104,\"height\":40,\"caption\":\"FigPii\"},\"image\":{\"@id\":\"https:\/\/www.figpii.com\/blog\/#\/schema\/logo\/image\/\"},\"sameAs\":[\"https:\/\/www.facebook.com\/figpii.aii\/\",\"https:\/\/x.com\/figpii\"]},{\"@type\":\"Person\",\"@id\":\"https:\/\/www.figpii.com\/blog\/#\/schema\/person\/c93c764d4e661184693d704990fed3e9\",\"name\":\"Reza\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.figpii.com\/blog\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/19f8902ea60a3af8c5edaf5994222ecfb88a64ddef769f26eaf7799ea4ce508a?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/19f8902ea60a3af8c5edaf5994222ecfb88a64ddef769f26eaf7799ea4ce508a?s=96&d=mm&r=g\",\"caption\":\"Reza\"},\"description\":\"Might sound like a clich\u00e9 but you can be sure that I live for my work as the head of customer success and product owner at FigPii.\",\"sameAs\":[\"https:\/\/x.com\/reza_jalaei_\"],\"url\":\"https:\/\/www.figpii.com\/blog\/author\/reza\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Quality Assurance for AB Testing, Best Practices. - FigPii blog","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.figpii.com\/blog\/quality-assurance-for-ab-testing-best-practices\/","og_locale":"en_US","og_type":"article","og_title":"Quality Assurance for AB Testing, Best Practices. - FigPii blog","og_description":"A\/B tests are frequently used to prove or disprove hypotheses that emanate from research.\u00a0 These can vary from tactical tests such as interface changes to strategic tests such as implementing a new checkout process on an eCommerce site or a completely new dashboard for a SaaS site.\u00a0 Over the years, we have noticed that the","og_url":"https:\/\/www.figpii.com\/blog\/quality-assurance-for-ab-testing-best-practices\/","og_site_name":"FigPii blog","article_publisher":"https:\/\/www.facebook.com\/figpii.aii\/","article_published_time":"2021-07-14T14:24:24+00:00","article_modified_time":"2024-06-17T01:10:16+00:00","og_image":[{"width":1379,"height":723,"url":"https:\/\/www.figpii.com\/blog\/wp-content\/uploads\/2021\/06\/image5-wpp1628797996704.png","type":"image\/png"}],"author":"Reza","twitter_card":"summary_large_image","twitter_creator":"@reza_jalaei_","twitter_site":"@figpii","twitter_misc":{"Written by":"Reza","Est. reading time":"10 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/www.figpii.com\/blog\/quality-assurance-for-ab-testing-best-practices\/#article","isPartOf":{"@id":"https:\/\/www.figpii.com\/blog\/quality-assurance-for-ab-testing-best-practices\/"},"author":{"name":"Reza","@id":"https:\/\/www.figpii.com\/blog\/#\/schema\/person\/c93c764d4e661184693d704990fed3e9"},"headline":"Quality Assurance for AB Testing, Best Practices.","datePublished":"2021-07-14T14:24:24+00:00","dateModified":"2024-06-17T01:10:16+00:00","mainEntityOfPage":{"@id":"https:\/\/www.figpii.com\/blog\/quality-assurance-for-ab-testing-best-practices\/"},"wordCount":1960,"publisher":{"@id":"https:\/\/www.figpii.com\/blog\/#organization"},"image":{"@id":"https:\/\/www.figpii.com\/blog\/quality-assurance-for-ab-testing-best-practices\/#primaryimage"},"thumbnailUrl":"https:\/\/www.figpii.com\/blog\/wp-content\/uploads\/2021\/06\/image5-wpp1628797996704.png","articleSection":["AB testing"],"inLanguage":"en-US"},{"@type":"WebPage","@id":"https:\/\/www.figpii.com\/blog\/quality-assurance-for-ab-testing-best-practices\/","url":"https:\/\/www.figpii.com\/blog\/quality-assurance-for-ab-testing-best-practices\/","name":"Quality Assurance for AB Testing, Best Practices. - FigPii blog","isPartOf":{"@id":"https:\/\/www.figpii.com\/blog\/#website"},"primaryImageOfPage":{"@id":"https:\/\/www.figpii.com\/blog\/quality-assurance-for-ab-testing-best-practices\/#primaryimage"},"image":{"@id":"https:\/\/www.figpii.com\/blog\/quality-assurance-for-ab-testing-best-practices\/#primaryimage"},"thumbnailUrl":"https:\/\/www.figpii.com\/blog\/wp-content\/uploads\/2021\/06\/image5-wpp1628797996704.png","datePublished":"2021-07-14T14:24:24+00:00","dateModified":"2024-06-17T01:10:16+00:00","breadcrumb":{"@id":"https:\/\/www.figpii.com\/blog\/quality-assurance-for-ab-testing-best-practices\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.figpii.com\/blog\/quality-assurance-for-ab-testing-best-practices\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.figpii.com\/blog\/quality-assurance-for-ab-testing-best-practices\/#primaryimage","url":"https:\/\/www.figpii.com\/blog\/wp-content\/uploads\/2021\/06\/image5-wpp1628797996704.png","contentUrl":"https:\/\/www.figpii.com\/blog\/wp-content\/uploads\/2021\/06\/image5-wpp1628797996704.png","width":1379,"height":723},{"@type":"BreadcrumbList","@id":"https:\/\/www.figpii.com\/blog\/quality-assurance-for-ab-testing-best-practices\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.figpii.com\/blog\/"},{"@type":"ListItem","position":2,"name":"Quality Assurance for AB Testing, Best Practices."}]},{"@type":"WebSite","@id":"https:\/\/www.figpii.com\/blog\/#website","url":"https:\/\/www.figpii.com\/blog\/","name":"FigPii blog","description":"Let it marinate","publisher":{"@id":"https:\/\/www.figpii.com\/blog\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.figpii.com\/blog\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/www.figpii.com\/blog\/#organization","name":"FigPii","url":"https:\/\/www.figpii.com\/blog\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.figpii.com\/blog\/#\/schema\/logo\/image\/","url":"https:\/\/www.figpii.com\/blog\/wp-content\/uploads\/2023\/09\/figpii-logo-purple.png","contentUrl":"https:\/\/www.figpii.com\/blog\/wp-content\/uploads\/2023\/09\/figpii-logo-purple.png","width":104,"height":40,"caption":"FigPii"},"image":{"@id":"https:\/\/www.figpii.com\/blog\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/www.facebook.com\/figpii.aii\/","https:\/\/x.com\/figpii"]},{"@type":"Person","@id":"https:\/\/www.figpii.com\/blog\/#\/schema\/person\/c93c764d4e661184693d704990fed3e9","name":"Reza","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.figpii.com\/blog\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/19f8902ea60a3af8c5edaf5994222ecfb88a64ddef769f26eaf7799ea4ce508a?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/19f8902ea60a3af8c5edaf5994222ecfb88a64ddef769f26eaf7799ea4ce508a?s=96&d=mm&r=g","caption":"Reza"},"description":"Might sound like a clich\u00e9 but you can be sure that I live for my work as the head of customer success and product owner at FigPii.","sameAs":["https:\/\/x.com\/reza_jalaei_"],"url":"https:\/\/www.figpii.com\/blog\/author\/reza\/"}]}},"_links":{"self":[{"href":"https:\/\/www.figpii.com\/blog\/wp-json\/wp\/v2\/posts\/142","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.figpii.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.figpii.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.figpii.com\/blog\/wp-json\/wp\/v2\/users\/3"}],"replies":[{"embeddable":true,"href":"https:\/\/www.figpii.com\/blog\/wp-json\/wp\/v2\/comments?post=142"}],"version-history":[{"count":21,"href":"https:\/\/www.figpii.com\/blog\/wp-json\/wp\/v2\/posts\/142\/revisions"}],"predecessor-version":[{"id":4642,"href":"https:\/\/www.figpii.com\/blog\/wp-json\/wp\/v2\/posts\/142\/revisions\/4642"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.figpii.com\/blog\/wp-json\/wp\/v2\/media\/144"}],"wp:attachment":[{"href":"https:\/\/www.figpii.com\/blog\/wp-json\/wp\/v2\/media?parent=142"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.figpii.com\/blog\/wp-json\/wp\/v2\/categories?post=142"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.figpii.com\/blog\/wp-json\/wp\/v2\/tags?post=142"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}