Web Performance

IEEE 829 Test Plan

According to the IEEE 829 test planning process there are eighteen factors to consider in building a success testing plan

1 Test plan identifier. 2 References 3 Introduction. 4 Test items. 5 Features to be tested. 6 Features not to be tested. 7 Approach. 8 Item pass/fail criteria. 9 Suspension criteria and resumption requirements. 10 Test deliverables. 11 Testing tasks. 12 Environmental needs. 13 Responsibilities. 14 Staffing and training needs. 15 Schedule. 16 Risks and contingencies. 17 Approvals. 18 Glossary.

Star East Show 2004

The Star East Show is coming up in May, from May 17th to 21st. If you are going to attend, post a comment and maybe we can meet at the show.

Capacity Testing Definition

Capacity testing occurs when you simulate a surge in the number of users stressing a web application’s hardware infrastructure. A good example of this scenario would be a successful marketing promotion, or a successful media event or publication. Your company has to ensure their system can withstand the results of such success on your web site.

Indian Outsourcing Companies Rely On Customers To Conduct Performance Testing

Further to my recent article on jobs in the US vs. outsourcing, I chatted with a marketing professional at a leading Indian outsourcing company yesterday. Once the software requirements have been gathered from a customer, every other aspect of the development process can be outsourced overseas, except for performance testing of the application in the customer’s application environment.

Where performance testing is conducted, if the outsourcing company owns the entire development life cycle, the performance testing is conducted in country, development people are sent out or customers take care of this issue for themselves. Typically, if an application project is outsourced, the one point of opportunity for American testers is on the performance testing of the web applications in the customer’s environment.

ISV Pressures To Benchmark Application Performance

The pressure to benchmark an ISV's web application comes from two sources. One, the customer, a customer wants to know an ISVs application will work within their environment. And two, an ISVs investors or board of directors, investors want to know that the application is competitive in the marketplace.

(Click on the image.)

Web stress tool, do I need to buy?

If you are thinking of load testing your web application, you will want to both load test the application and stress test the application. With load testing, putting the application through its paces under an estimated or expected number of users hitting the different parts of the web application at different speeds and times. You will also want to stress the application, where you would have a usually high number of customers taking the same action at the same time.

Most automated load testing tools charge per virtual user, you are probably going to want to continue load testing of your web application over time, but stress testing will only need to be conducted occasionally. To save on costs, ask your load testing vendor if they have any leasing or rental options. The vendor should provide options to rent their application for a week, a few weeks or a month. I know that Mercury Interactive provides this option as well as Quotium Technologies.

Outsourcing To India Is Good For Testing Jobs In The US

Software & testing outsourcing to India has become a big issue for companies and US workers. Companies want to reduce their costs, if you were in business and told that it would cost $200,000 to develop an application in the US and $50,000 in India what decision would you make?

Well lots of companies are sending their projects overseas, but not only to India, apparently India is not even the largest location for outsourcing, Ireland and Canada are in the number one and two spots. However, India is seen as having the most potential for outsourcing of software development projects due to its lower costs of doing business, but this is balanced against higher geo-political risk, you may remember the Pakistan-India scare of a few years ago.

In a trend reminiscent of Japan’s manufacturing quality movement, Indian companies actually have a higher level of certification on quality standards than their counterparts in the US. Merrill Lynch recently reported that India has the largest number of CMM (Capability Maturity Model) Certifications for software processes.

With the downturn in the economy, many software programmers in the US are facing higher rates of unemployment. But now there are political repercussions for this software unemployment, cities and states are considering requiring vendors to certify they are using US labor and politicians across the country are developing policies that seek to combat the issue of outsourcing. Economics is a hard fact of life to legislate against. However, there is one area of the software development process where I’d argue there is a requirement that a company has employ highly skilled workers in the US, and that is load and stress testing.

Here’s why, you might outsource the development of code, but to truly test your application in the environment it is going to reside within, you have to run the load testing of your web application in the actual web site environment. That means you have to test your application on the servers, databases and infrastructure in the US. That testing has to be completed or at the very least developed in the location of the web application environment. This means companies are going to need highly skilled software testers to do planning and oversee the final load testing of an application here in the US.

Outsourcing brings big benefits such as lower costs, but there are also problems. Distance and time differences are some of those problems. A US based project manager or business product owner has to make sure they have really planned a project and maintain greater oversight over long distance projects. Critical to the success of outsourcing projects is the provision of good testing reports. This means that there is an even greater need for good software tools that provide easy to use and comprehensive reports with outsourcing.

Automated testing tools like Mercury Interactive’s LoadRunner, Quotium Technologies’ QuotiumPRO and Compuware’s QATest provide excellent reporting of load testing projects that help you identify bottlenecks quickly. Outsourcing is certainly a reality of today’s world, I’d argue that load testing your web application here in the US is also a reality, therefore software testing engineers with automated testing skills have better chance of getting employed in the US.

Background Articles:

Daily Mirror

BankTechNews

CNN Money

Boston Globe

Seattle Post-Intelligencer

EnterpriseInnovator

Web Performance Definition

If you want customers to stay or return to your web site you have to make, sure the performance of your web site is good. If customers have long wait times to use your web site applications they will get frustrated and go to your competitor. You must ensure that web pages download quickly, the web site is available to the customer, and there are few errors or bugs.

Performance can be measured either manually or automatically; it is rather difficult to co-ordinate several customers hitting your web site at the same time. My recommendation is to use an automated testing tool. You need to come up with criteria for selecting a tool, more on a tool selection criteria in another posting.

A Definition of Scalability

Scalability – As the number of server requests or users increases, a web application continues to perform well. Reasons for problems with scalability include:

· Databases that were not properly tuned.

· Coding design that hinders performance.

· Server hardware infrastructure that is unable to support increasing user load.

· Network capacity issues.

Performance scalability also depends on the original goals of the system. As each application is different, an application manager may set different business goals for the system. You might set a goal of having a system manage the load of a number of concurrent users, or set the goal of more transactions per second.

Installed vs. Non-Installed Agents

If you are running a test of a complex web application environment with different servers and databases, you will want to monitor the performance of the different components of your environment. Different automated load testing tools give you the option to use installed monitoring agents or non-installed monitoring agents.

If you install a monitoring agent on a server, data can be collected directly on the server and sent to the machine collecting the data for the test of the entire web application. While a non-installed agent will require that, you send a request to the monitored servers and then wait for the data to be sent back to the client. An installed monitoring agent has more time to gather and send data to a central machine. With a non-installed agent monitor there is a larger risk of network traffic interrupting the flow of data back and forth between the client and server.

However, most network administrators would prefer no agent was installed on one of their live production servers. If nothing is installed, a network administrator does not have to do anything or worry about the installed agent disrupting their server.

Mercury Interactive provides monitors that are not installed on a server.

Quotium Technologies provides you with the option to either use non-installed agent monitors or installed agent monitors on a windows server.

If you are running a test of a complex web application environment with different servers and databases, you will want to monitor the performance of the different components of your environment. Different automated load testing tools give you the option to use installed monitoring agents or non-installed monitoring agents on the servers.