21 – Making it in the real world - Performance Testing

Date: 5/11/2020

Guests: None

Welcome and greetings

Recap of last episode

• In our last episode, I discussed the concepts relating to System Integration Testing. I talked about what it is, how it differs from other testing, and what you hope to gain from performing it.

Summary of this episode

• In this episode, I’m going to discuss what performance testing is and the benefits that you receive once it is completed successfully. I will also talk about the different types of performance testing and how they each tell a different story about the health and reliability of your application.

What’s in it for you?

• After listening to this episode, you will be able to discuss the various types of performance testing and the benefits of each one. You will hopefully be able to have conversations with your peers that show the value that can be realized by a successful performance test.

Episode Content

Performance testing goes by a variety of names but in all instances you are trying to determine how your application will perform under real world conditions. Nobody wants to deploy their application to their production environment only to realize too late that their web servers do not have the capacity to handle all the users that are trying to use it for some purpose. This is especially true for companies that run websites that sell products or perform banking activities.

Types of Performance Testing:

• Load Testing: How the system performs under a defined load

• Stress Testing: How the system performs when the load is increased

• Spike testing: How the system performs when the load is dramatically increased for a short duration

• Endurance testing: How the system performs when the load is maintained for longer than expected

• Scalability Testing: How the system performs when it is dynamically scaled by a pre-determined factor for transaction volume (2:1, 3:1, etc.)

• Volume testing: Another name for Load Testing that involves large sets of data.

Here are some of the activities that take place when you are planning a performance test:

• Determine test environment requirements

  • This can include servers, operating systems, the application to be tested, other applications needed to run the test, and any configurations needed to ensure the successful completion of the performance test

• Design performance testing requirements

  • what is an acceptable response time for the application under test for a given transaction type?

  • what is an acceptable level of uptime for the application and what tests will ensure that that level has an acceptable chance of being reached?

• Design the tests that will be executed

  • identify specific use cases that are in the high volume category

  • this can be accomplished by working with the development team to identify candidate use cases.

• Buy, install, configure the testing environment

  • this will include ensuring that the software required to execute the test is available, licensed, and configured prior to executing the test.

• Write you performance testing scripts per the agreed upon design

  • This can be complicated with web applications that generate lots of dynamic content.

  • The dynamic content will need to be parameterized in the scripts so that run-time errors are not encountered during the test if a non-parameterized value is encountered

• Execute the tests according to the plan

  • The performance tests are run as agreed upon.

  • Support staff should be scheduled prior to the test to ensure that any issues can be addressed in a timely manner.

  • The time it takes for a test to get to the steady state of a particular transaction volume is called the ramp up time and should be built into the testing timeframe.

  • The time it takes for all transactions to finish after the agreed upon transaction volume is reached is called the ramp down time and while not needed for most performance metrics that you will compile, it can help clean up a system if you are planning on running several performance tests consecutively.

• Accumulate, document and publish the test results

  • all results should be collated and analyzed in order to provide the most accurate recommendation possible after a test is complete.

  • The results can either clear an application for release or send it back to the software engineering team for changes to improve its performance.

Recap of this episode

• In this episode, we discussed the various types of performance testing, how they are different, and some of the benefits that come from running a successful performance test.

Like me….. Like my podcast…. Please share the link, click on the like, give us a thumbs-up, leave us a review, hit the subscribe button, and tell your friends!

Next Episode: Scott’s Spot Product Review

Where you can find us!

Direct Messages:

• @cs_everhart on Twitter

• ScottTalksTech group on Facebook

• ScottTalksTech.slack.com

Links to Podcast Providers:

• On Google Podcasts

• On Apple Podcasts

• On Spotify

• On iHeartRadio

• Pandora coming soon

• PodcastAddict – web link not available but podcast is available in the app

• On PodChaser

• On Deezer

• On ListenNotes

• On OverCast

• On CastBox

Episode #16 – Testing 1,2,3

Episode #16 – Testing 1,2,3

Date: 1/5/2020

Guests: None

Welcome and greetings

Recap of last episode

• In the last episode, we talked about the need to start your testing early in your Software Development Life-cycle (SDLC) where it is cheaper to remove defects that are found. We also discussed some of the reasoning by why you need to start early testing, some types of early testing, and costs of early testing…..or starting early.

Summary of this episode

• In this episode, I’m going to discuss some of the different types of testing and where they typically fit into your SDLC.

What’s in it for you?

• After listening to this episode, you should have a better understanding of the types of testing that are typically seen in a SDLC and what they are composed of.

Episode Content

• Requirements review
• Unit Testing – testing of a specific piece of code. This is a test for the smallest functional piece of code. Think of this as testing a specific function or method.
• Component Testing – This is used to test the complete functionality of a class. All of this testing should be stand-alone and only involve the functions and methods of this class. It can be thought of as method a calls method b and passes a defined set of data and expects method b to return a defined response (data, true/false, etc.).
• Component Integration Testing – This consists of testing how the components of your application interact with each other. It can consist of passing a specific set of data between the functions in your class to ensure that they output of the class meets the defined criteria.
• System Integration Testing – This testing is usually performed when your application is deployed to a defined testing environment. The environment should have all the infrastructure, middle-ware, and data to support your testing. This type of testing is used to evaluate how your application interacts with external entities. It could be a call to another application or website, a call to an internal company API (Application Programming Interface), or writing to an external database for logging. This type of testing can be very extensive and time-consuming if it is done manually.
• User Acceptance Testing – This testing is done after you have deployed your application to a stand-alone environment and your business users are the only ones that are allowed to test against it. They should be trying to determine if the application or changes meet the needs as outlined in the business requirements (remember when we said the easiest type of test was a requirements review).
• Regression Testing – this is a defined set of tests that are run every time that a change is made to an existing application.
• Smoke Testing – This is a sub-set of your regression test suite that can be run around a specific change to ensure that the application is performing as expected.
• Non-functional Requirements
  • Performance Testing – uses a defined load against the application (ex. 100 http sessions started per second)
  • Load Testing – tries to determine how the application will work under a higher than normal load or usage.
  • Break Testing – ramps up the load until the application breaks. This is done to determine what your real capacity is for the infrastructure that you have set up for your testing (ex. 1 web server can handle 1,000 http sessions per second). Then you can use that to extrapolate how much infrastructure that you will need for your expected production capacity.
  • Spike Testing – This testing tries to determine how your application will handle unexpected spikes in traffic. Will the application continue as normal, will it slow down, or will it stop working and crash unexpectedly.

Recap of this episode

• In this episode, I discussed various types of testing and where they are typically found in the SDLC. This is not an all encompassing list and does not dive too deeply into each type but is a general overview of how they can be used and how you might want to think about using them to testing your applications.

Shout-outs- Mike Lyles and his new book – The Drive Thru is NOT Always Faster

Like me Like my podcast – share, like, thumbs-up, review, subscribe

Next Episode: Test automation