Episode #17 – How Fast Are Your Tests? - Test Automation Concepts

Date: 1/8/2020

Guests: None

Welcome and greetings

Recap of last episode

• In the last episode, I discussed various types of testing and where they are typically found in the SDLC. This is not an all encompassing list and does not dive too deeply into each type but is a general overview of how they can be used and how you might want to think about using them to testing your applications.

Summary of this episode

• In this episode, we discuss Test Automation and how it can be used to accelerate your pipelines. We will discuss some of the methods of automating tests and some products that can help you with your automation.

What’s in it for you?

• After listening to this episode, you will be able to determine if your tests are good candidates for automation and how you can integrate test automation into your CI/CD pipelines.

Episode Content

• What is Software Test Automation?
  • It is simple the act of executing a set of test cases without any human intervention.
• Why is it important?
  • This takes the human factor out of test execution. All steps are executed in the same manner every time that the test case is run. This restricts any variation in test results from a human hand controlling the execution of the tests.
• What are some of the benefits?
  • Repeatability – Easy to duplicate your results
  • Consistency – minimal variation in test execution
  • Speed – automated scripts don’t need bio breaks
  • Reliability – no human factor
  • Schedule-ability – can be set to run automatically
• What makes a good automate-able test case?
  • Frequency of execution – provides more value quickly
  • Known inputs and outputs
  • Simple test cases
• How can we automate a test case?
  • It can be scripts that are written in house?
  • It can be done with third party tools such as Selenium, Unified Functional Test, and other automation tools such as Jenkins
• Maven can run your unit tests as one of its “goals”: clean, verify, compile, test, package, deploy
• Gradle can run your unit tests using a task of type test.
• Selenium can be used as a scripting engine to test your application
• Unified Functional Test is an example of a 3^rd party tool that creates a “script” of test steps that are then executed as part of your automation.

Recap of this episode

• In this episode, I discussed what test automation is, some of the benefits, and some types of test automation.

Like me….. Like my podcast…. Please share the link, click on the like, give us a thumbs-up, leave us a review, hit the subscribe button, and tell your friends!

Next Episode: The journey of 1000 miles starts with a single step ...... Unit Testing

Episode #12 - Ball and Chain of Custody - Legal Requirements

Last Episode recap

• In the last episode we discussed Jenkins. We talked about what it is, what it is not and how it can be configured to use in your automation efforts.

Summary of this episode

• I hope you have seen that I’m trying to do these episodes in what I think is a very specific order. We will build on each episode so that you understand how each topic discussed can be used to enhance your automation effort.
• In this episode, we will discuss maintaining a software “chain of custody”, why it is important, what happens if you don’t and how your automation can help you do it.

What's in it for you

• A better understanding of the importance of maintaining good software controls and how it keeps you and your company out of trouble.

Episode Content:

• Why does it matter?
  • In 2001, a large corporation named Enron was embroiled in a controversy involving its financial records. It was followed by other controversies with other large publicly traded corporations.
  • In July of 2002, two U.S. Congressmen, Sen. Paul S. Sarbanes and Sen. Michael G. Oxley, sponsored a bill to rein in the fraudulent financial reporting that led to these massive corruption scandals. This bill was titled the Sarbanes-Oxley Corporate Responsibility Act of 2002 (or SOX for short).
  • The bill tightened controls around how a company recorded and reported its financial Information.
  • One of its goals was to provide for structured, consistent and repeatable controls.
  • It was created as a way to for the leaders and employees of a company to create and maintain a culture of corporate financial responsibility.
  • It provided for the officers of a company to “sign” their financial statements. This means that they take personal responsibility that they are accurate and correct with no malfeasance or attempts at fraud against investors.
  • It increased the fines and sentences for individuals that attempt to commit fraud or illegal financial practices.
  • It requires a company to document its internal control processes.
  • It requires a company to to have an external auditor to certify that their internal control processes are being followed and that the financials are accurate.
  • It dictates that all off-balance sheet transactions must be reported.
  • It provides the Security and Exchange Commission more power to perform investigate any company suspected of violations.

• Who has to comply?
  • All publicly traded companies
  • All subsidiaries of foreign companies in the U.S.
  • All private companies that are preparing for an IPO
• Fines and Prison Time
  • Non-compliance could result in fines up to $1 million dollars and up to 10 years in jail
  • If a company intentionally defrauds investors, it could result in $5 million dollars in fines and up to 20 years in jail.

• When should you worry about it?
  • You shouldn’t worry about it too much but,
  • You should make sure that your control processes are well documented, your staff understands why they are in place, and any consequences that they or the company might face if they do not follow the controls.
• Where should they be implemented in your SDLC?
  • In my opinion, every step of the SDLC, for example:
    • Requirements – should be documented and changes should be tracked.
    • Development – All source code should be in an SCM system so that any changes can be tracked back to why and who made the change.
    • Build – All builds should be documented and all artifacts of the build process should be stored and versioned.
    • Testing – all test cases that are used to verify requirement functionality should be documented and any results of those test cases related to a specific build should be tagged with the build ID and archived after completion.
    • Environments – All changes to an environment used for a specific build should be tagged and documented for approval prior to implementation.
    • Deploy – The reason for the deployment, date and time, and approval should all be documented.
    • Promotion to a higher environment – all approvals should be documented and tracked for each environment promotion.

• How can your automation help you with all this documentation?
  • By using some or all of the tools that we are discussing, you can have a lot of this documentation done for you automatically by your automation.
  • For Example:
    • Requirements: Jira, Quality Center
    • Development: Github, Bitbucket, SVN, Team Foundation Server
    • Build: Maven, Gradle, Jenkins, CI/CD pipelines
    • Testing: qTest, Quality Center, Bugzilla
    • Environments: CI/CD, Jira
    • Deploy: CI/CD, Jenkins, Ansible, Terraform
    • Promotion: Jira, Remedy, ServiceNow

• Each of these tools will automatically record the changes that you make, builds that run, tests that are performed, and artifacts that are deployed.
• Using Jenkins or a similar tool, you can create a pipeline or job that can automatically create the files required by your audit partners to run their reviews.
• Usually the only changes to be reviewed after automation are any changes that are made to the automation pipeline/jobs themselves.
• Working with internal and external auditors can be a daunting task since they are trained to ask many probing questions to identify problems with your process.
• Using automation makes their and your job easier when it is time to perform these control reviews, which is usually monthly, quarterly, semi-annually or annually.

recap of this episode

• In this episode we discussed what software chain of custody is, why it is important, where it originated, how it is controlled, and how your automation can assist you in financial reporting compliance audits.

Next Episode: Artifact repositories

Episode #5 - Build Tech and Value Stream

Episode 5 - Build Tech and Value Stream
---------------------------------------
So you have your code, what do you do with it?
- Bad idea - keep it on your laptop or a network drive

What SCM tool do you use to store your source code?
- SCM - Software Configuration Management
- Also called version control
- It's where you put your code so that nothing bad happens to it (like being deleted).

What build tool you use will depend on the platform and language that you are writing your code in.
- Java, .net, C++, C#, nodeJS, etc.
- Linux, AIX, Solaris, Windows, Docker, Mainframe, etc.

First you will need to understand your build process.
- Automation enables your process to go faster and smoother but it does not replace the process.
- You need to sit down and document the entire process from Requirements definition to having an artifact that can be installed into an environment.
- Value Stream Mapping
-example:
- Document the requirement (including acceptance criteria).
- Check out code (if existing).
- Modify code to meet the acceptance requirements as written in the requirement.
- Check in / Commit your code to your SCM system.
- Update and parameter or config files as needed.
- Run automated/manual unit tests (this will be another podcast).
- Run your packaging utility (this will depend on the language that you are writing your code in).
- Ant, Maven, Gradle, Make, etc.
- Upload Artifact to server (host).
This step will be replaced when automation is written.  The artifact will be uploaded to an artifact repository (binary repository).
- Install artifact into testing environment on this host.