Web Performance

Evaluating Testing Tools vs Manual Testing

In a recently interview with John Rosky I talked about the time it takes to complete manual testing when compared with automated testing.

People want to get software products out to market faster, and if they see they can decrease the testing and development time, then they can improve quality and get the product out of the door quicker. For example, if you have five manual testers testing a large application, it would take 4 months to test an application, to test the same application with an automated testing tool, you can probably use the automated testing tool to do the testing in a week, basically you can complete a lot more testing in a shorter amount of time.

Think about the numbers involved, it's much more expensive to use manual testing than compared to automated testing.

Why Use Automated Testing Tools?

As web applications get larger and more complex, the need to use automated testing tool increases. It just gets to the point where you have to use a testing tool to make sure you have tested the complete application, and also conducted your testing in a timely manner. Here are some comments on this issue from John Rosky in my recent interview with him.

Most of my experience has been in the manual testing area; I have only used automated tools in the last two years. The reason why most people use automated tools is because they don’t want to reinvent the wheel. If a company has already put the time and effort into developing a tool, why not use that tool.

Using tools like Segue's SilkPerformer Lite or Quotium Technologies QuotiumPRO will speed the time it takes to complete testing and make it easier to conduct testing with easy to use graphical applications.

IP Spoofing lets you track Virtual Users on multiple servers

When conducting performance testing, companies with load management software for multiple servers don't know which virtual user is hitting what server. By assigning each virtual user with a unique IP address, you can track the performance of each virtual user and hence really pinpoint the bottleneck to an individual server.

Here's why, by tracking virtual users by IP addresses you can assure that load management configurations are optimized for multiple servers.

Quotium Technologies QuotiumPRO lets you create many different IP addresses. You would use QuotiumPRO's IP Generator to create virtual IP Addresses. The IP Generator gives you the ability to set the number of IP addresses and range of IP Addresses. You can also add or remove IP addresses for each network adaptor.

Another reason for spoofing IP addresses is in the case of applications that identifying users by IP address. Some applications will find access rights to a user based on the IP address.

Loadrunner - A testing tool for finding the performance bottleneck

Loadrunner is a performance testing tool used widely in the testing industry. Mercury interactive developed the product. Loadrunner offers performance monitors for just about every area of your infrastructure. A similar tool offering many performance monitors as well is QuotiumPRO from Quotium Technologies.

All tools will tell you have performance problems, the hard part is finding is the root or cause of that problem, having the proper performance monitors can save countless hours, resources, and money when trying to pinpoint the reason behind your bottlenecks.

Test Scripting Language

Automated testing tools let you build testing scripts to automate the process of running a load test for a web application. Those testing scripts are built in a scripting language. One such scripting language would be Script Control Language found in QuotiumPRO, the performance testing product from Quotium Technologies of Beverly, Massachusetts. Similar to Pearl or .php SCL is an easy to use scripting language.

Users can use the record feature in QuotiumPRO to develop the script and then customize that script.

Web stress tool, do I need to buy?

If you are thinking of load testing your web application, you will want to both load test the application and stress test the application. With load testing, putting the application through its paces under an estimated or expected number of users hitting the different parts of the web application at different speeds and times. You will also want to stress the application, where you would have a usually high number of customers taking the same action at the same time.

Most automated load testing tools charge per virtual user, you are probably going to want to continue load testing of your web application over time, but stress testing will only need to be conducted occasionally. To save on costs, ask your load testing vendor if they have any leasing or rental options. The vendor should provide options to rent their application for a week, a few weeks or a month. I know that Mercury Interactive provides this option as well as Quotium Technologies.

Installed vs. Non-Installed Agents

If you are running a test of a complex web application environment with different servers and databases, you will want to monitor the performance of the different components of your environment. Different automated load testing tools give you the option to use installed monitoring agents or non-installed monitoring agents.

If you install a monitoring agent on a server, data can be collected directly on the server and sent to the machine collecting the data for the test of the entire web application. While a non-installed agent will require that, you send a request to the monitored servers and then wait for the data to be sent back to the client. An installed monitoring agent has more time to gather and send data to a central machine. With a non-installed agent monitor there is a larger risk of network traffic interrupting the flow of data back and forth between the client and server.

However, most network administrators would prefer no agent was installed on one of their live production servers. If nothing is installed, a network administrator does not have to do anything or worry about the installed agent disrupting their server.

Mercury Interactive provides monitors that are not installed on a server.

Quotium Technologies provides you with the option to either use non-installed agent monitors or installed agent monitors on a windows server.

If you are running a test of a complex web application environment with different servers and databases, you will want to monitor the performance of the different components of your environment. Different automated load testing tools give you the option to use installed monitoring agents or non-installed monitoring agents on the servers.

Web Load Testing Downloads

Want to compare the major automated load testing tools on the market, click on the links below.

Quotium Technologies QuotiumPRO 3.5 Download Center

Mercury Interactive Download Center

Rational Software TestManager Download

Compuware QALoad Demo Center

RadView WebLOAD Download Request

Empirix e-TEST Download

Segue SilkPerformer Lite Load Test