Web Performance: The Interview with John Rosky

June 2004

Sun	Mon	Tue	Wed	Thu	Fri	Sat
		1	2	3	4	5
6	7	8	9	10	11	12
13	14	15	16	17	18	19
20	21	22	23	24	25	26
27	28	29	30

I recently met John Rosky, a QA testing professional; he agreed to answer my questions about the testing industry. John has been in QA since 1996, he has worked at Lotus development, GMIS, Micro Systems Lotus Development Corp./IBM and lastly a Portsmouth, NH-based subsidiary of Micros Systems, Inc. John has been working in the software development industry since 1987, so about 16 years.

John C: Why would you use an automated testing tool as opposed to developing in-house testing scripts?

John R: Most of my experience has been in the manual testing area; I have only used automated tools in the last two years. The reason why most people use automated tools is because they don’t want to reinvent the wheel. If a company has already put the time and effort into developing a tool, why not use that tool.

Certain tools, the windows GUI based testing, that's the easiest thing to test. A lot of the dirty work has been done for you, but if the issue is more difficult, and as the test becomes more complex, you may have to put more effort into making a good test.

John C: How have you used automated testing tool reports in the past? For what purposes did you need to develop the reports?

John R: Only used Mercury's LoadRunner product, we used it primarily for scalability. We also used the analysis tool as part of LoadRunner, which generates its own set of reports, then took data from those reports and extrapolated the data, and we used the analysis tool as part of Loadrunner.

John C: Where do you see testing tools technology heading?

John R: In the past few years, I am seeing a lot more java based testing tools. Personally, I think they could go almost in any direction, the testing tools per say are a very good thing. The tools are being developed to test more complex programming issues such as API. Now that more developers are using them, a bigger development audience has been created; tool vendors appear to be filling every void.

People want to get software products out to market faster, and if they see they can decrease the testing and development time, then they can improve quality and get the product out of the door quicker. For example, if you have five manual testers testing a large application, it would take 4 months to test an application, to test the same application with an automated testing tool, you can probably use the automated testing tool to do the testing in a week, basically you can complete a lot more testing in a shorter amount of time.

However, in my experience that's not exactly what happens every time, that’s the best scenario. Typically there is a learning curve, a cost, tools are not cheap, not a silver bullet, you have to be very realistic.

In my previous job, my manager had used WinRunner at another company, she tried for 2 years to convince management to buy the product, but the price scared them off, they wanted to know if the tool would ever give a good ROI. We were a small development shop part of a larger company and keeping within a low budget was top priority. We were already paying a developer; they thought: why not create a homegrown testing tool.

John C: What would you like to see developed by the tool vendors?

John R: I guess that I would speak specifically about Loadrunner, I thought it was very effective and useful, but it was also cumbersome to use and to learn.

I would want a simpler tool from the testing industry. Maybe I am asking for something that cannot happen, at the same time I liked how comprehensive LoadRunner was, but it was cumbersome to use initially. I guess, on a more generic sense, I would like to see a tool that does not require so much set up or potential development person intervention, a testing person who is not developer can pick up and use.

John C: We were chatting about the requirements of custom development houses and Independent Software Vendors to demonstrate software scalability to their clients in the clients’ environment. Can you recall any examples of such projects in your career and what were the pitfalls and advice you'd have for customers in handling that issue?

John R: In my experience, it was a custom application house; our customer was like an ISV. We had a centralized database server application; the clients were all over the world. The client was a huge hotel chain. The hotel chain had a distributed environment; every hotel had a server in every hotel. We were centralizing the application.

My testing involved the application and the database; we wanted to find out if we could scale the application from one hotel to 500 hotels. We tested the application to determine if it could handle so much processing, then bump up the number of hotels 20% and then bump up more. In our case, the existing hardware would scale up to 250 hotels.

The customer did not understand how challenging the project was going to be. They thought they could run a few tests, and keep bumping up the load. However, there were more problems and time involved in scaling the application than anticipated.

John C: Thanks John for answering all of my questions, I really appreciate you taking some time out to discuss these web performance issues.

Web Performance

Quotium Technologies blog all about testing web applications

Recent Posts

June 2004

Blogs

Recent Comments

The Interview with John Rosky

Comments

About

Archives

Categories

Testing Tools