Published: 04 Oct 2016
6 Best Performance Testing Practices For Black Friday
Last Updated: 05 Jan 2024
Black Friday is just around the corner and has made retailers fret about how to increase their revenue. This day is considered to be the busiest shopping day of the year, since 2005. It is that time of the year when customers are able to avail of great offers on many products. Every year, there is a significant increase in online retail sales.
1. Black Friday Facts
2. Best Performance Testing Practices For Black Friday
3. Why TestingXperts
Black Friday Facts
According to a report by IBM Watson Trend, the online sales for Black Friday increased by over 21.5% for the same day in 2014. In 2015, the average order value for mobile and desktop of the Black Friday sale was $127.84 in the U.S. and the total online sales during Black Friday and Cyber Monday hit a record of $5.81 billion.
Black Friday is the most critical day in the e-commerce industry, leaving casualties in its wake. A huge amount of traffic can increase the revenue but, when an ecommerce site is not ready to withstand the huge increase in traffic, it can result in website crashing. A website crash can have a negative impact not only on the online sale but also on customer trust and damage the brand as whole. It is extremely important for the website to be resilient to sustain a huge spike traffic.
There are less than 2 months before Black Friday so this is a perfect time for e-commerce companies to check if their website, application and infrastructure is ready for the anticipated crowd. To make sure that the website is ready, we suggest the system should be 100% performance-tuned and planned.
Here are 6 Best Performance Testing Practices For Black Friday
1. Assess baseline performance:
Review current analytics to understand the current performance of the site. What are the load during current busy day and busy hour? How is performance during busy times? Are there any bottlenecks, failures, etc.? Is performance acceptable under the current volumes? Where is the traffic coming from?
2. Evaluate your site’s performance goals from a load, stress and expected volume standpoint:
In order to determine whether your website can handle Black Friday/ Cyber Monday, you need to understand what traffic to expect. How many online shoppers do we expect on a busy hour? Do we anticipate significant spikes based on time sensitive offers or steady increases in traffic? Which transaction types do we anticipate (e.g. short visits with people taking advantage of a few offers or longer visits with browsing, searching, etc.) Based on the captured numbers, analytics data and expected results, anticipated load including the expected number of concurrent users, key transaction types, and traffic spikes can be projected.
In the event that projected performance load cannot be determined, we recommend planning for traffic of 3-4 times of the current load. To make sure that get optimal coverage of the site is considered from a transaction perspective, identify the 20% of online transactions that covers 80% of the total work load. This is usually 15-20 transactions focused on product search, shopping cart, check out, coupon codes, etc.
3. Plan all the tests:
In order to achieve your business goals, you need to determine and plan the tests that you need to run
Some of the tests that we typically plan
Load Test
In order to understand the behavior of the system under a specific load. Execute test at current load, 75% of Black Friday load, 100%, and 150%.
Stress Test
For understanding the load capacity of the system (e.g. 300% of Black Friday load)
Spike Test
For observing the system’s skills hold a heavy load that was generated abruptly.
Soak Test
For examining the ability of the system to hold a continuous load over a long period of time (e.g. 8 – 10 hours) to identify issues such as memory leaks
Isolation Test
For running repeated test and examine whether a known system error or issue that was detected previously was fixed or not
Test to the point of Failure
An organization must be prepared for the best and the worst-case scenario with large events such as Black Friday or Cyber Monday. Testing should be done to the point of failure, by running a sequence of tests while continuously increasing the load until you hit a scaling issue. This will allow the organization to know where they will have a problem in the event that marketing efforts are wildly successful.
Run the Tests
Run the performance tests as designed; ensuring that analytics are captured throughout all test. Use tools to simulate the traffic as close to what is anticipated: mix of transactions, types of shoppers, geographic model, etc.
Evaluate and remediate:
Analyze the test resulting to identify errors, bottlenecks, performance degradation. Evaluate key performance indicators such as throughput, response time, connect time and latency make it easier for evaluating the results.
Isolate the root cause of the performance issue: network, hardware, database, query, application, etc. An objective eye must be taken in the evaluation of performance test results in order to avoid internal “finger pointing” across functional teams.
After performance issues have been addressed through system tuning, fixing bugs, database tuning, query optimization, etc., recalibrate and re-run the tests to assess the improvement of your system after the changes.
Why TestingXperts
From our experience we realized most of our clients have three major concerns while planning the performance tests: evaluating the right time to conduct performance tests, evaluating the right tool and strategy to plan and execute the performance tests and evaluating the appropriate test coverage.
We at TestingXperts make sure we address our client’s concerns appropriately and plan tests at right time and at right stage. We have used wide range of performance test tools: open source tools like JMeter, Wapt and licensed tools such as VSTS, Load Runner/Storm Runner, Blaze meter, NeoLoad and many others. We make sure our clients never lose focus from their core business and always trust partners like us for keeping their system up and running in a fine-tuned environment.
For more information please do not hesitate to contact us drop us a line at
[email protected]