Basic concept of how things work (HTTP, Network, LB, Web Server, App server, DB server, LDAP, SSL Accelerator etc.,)
Eagerness to learn things
patience
In sting (list continues based on what you do)
Purpose
Check whether it met predefined SLA
Compare 2 systems and find out which is performing better
Find bottleneck and tune to make the system to perform better
Performance Testing/Engineering is not a science, it is rather art.
Phases in Performance Testing/Engineering
- Requirement collection
- Feasibility study
- Test Plan
- Scripting
- Monitoring Design & Deployment
- Smoke Test
- Test Execution & Data Collection
- Analysis and Reporting
- Closure
- Requirement Collection
Understand want customer really wants
Tell customer what performance team do and make sure they understand what it means.
Ask customer what to do
While selecting business flow for simulation, select flow(s) which obey the following rule. “20% of total business flow carried out by 80% of users”. (Traversal document)
Understand the architecture (this has to done in detail if monitoring and tuning is involved)
Stress to customer that
a. We are not here to do function testing.
b. We don’t/can’t performance test everything.
c. We don’t/can’t simulate 100% of production transactions.
Feasibility Study
Can we do what customer wants? If not, ask customer for help. Better to say no rather than doing something wrong or giving false commitment. This will make sure we get repeated customer.
How fast can we do this task? (this helps in planning)
Does tool by native support this or do we need to build work around?
Note: In the same service based company, marketing team can’t say no most of the times (May be due to aggressive targets set by management or earn commission and much more); in that case, always insist for employee training whenever possible.
Test Plan
Form a team. Know their skills
When to do what?
Deadlines. Fix deadline based on team’s skill(not based on TL's skill).
Get approval from client
Scripting
Simulate as per traversal document
Remember we simulate only core portion; so don’t make mistake in that.
Build agenda/driver script based on mix.
We are testing something to find issues. Make sure your script doesn’t have issues.
Form a team. Know their skills
When to do what?
Deadlines. Fix deadline based on team’s skill(not based on TL's skill).
Get approval from client
Scripting
Simulate as per traversal document
Remember we simulate only core portion; so don’t make mistake in that.
Build agenda/driver script based on mix.
We are testing something to find issues. Make sure your script doesn’t have issues.
Monitoring design & Deployment
Get the list of servers that needs to be monitored (including versions)
If you haven’t worked on that specific version of servers, spend sometime in going though ‘new features’
Design & deploy the monitoring for OS(perfmon, vmstat, netstat etc.,) and then corresponding servers specific monitors(perfservlet, server-status, statspack etc.,).
Get the list of servers that needs to be monitored (including versions)
If you haven’t worked on that specific version of servers, spend sometime in going though ‘new features’
Design & deploy the monitoring for OS(perfmon, vmstat, netstat etc.,) and then corresponding servers specific monitors(perfservlet, server-status, statspack etc.,).
Smoke Test
Smoke Test is the test to make sure script and monitoring are working as required
Make sure entire team is there during smoke (scripting team and monitoring design/deployment team)
Have you detected something suspicious? Check and/or correct the same. Mistakes here will lead to failure of whole engagement.
Redo smoke test if some changes were made
Make sure enough time is allotted for smoke. Don’t do smoke just before 1 hour of test. Atleast give 1 or 2 days for smoke depends on complexity of engagement.
Make sure you have check list
Smoke Test is the test to make sure script and monitoring are working as required
Make sure entire team is there during smoke (scripting team and monitoring design/deployment team)
Have you detected something suspicious? Check and/or correct the same. Mistakes here will lead to failure of whole engagement.
Redo smoke test if some changes were made
Make sure enough time is allotted for smoke. Don’t do smoke just before 1 hour of test. Atleast give 1 or 2 days for smoke depends on complexity of engagement.
Make sure you have check list
Test Execution & Data collection
Should be only started if everything was gone fine in smoke.
Make sure check list is correctly followed.
After test execution, collect simulation tool logs, monitoring logs, configuration files, log files and whatever is required by analysis team.
Most importantly timesync information of all servers.
Should be only started if everything was gone fine in smoke.
Make sure check list is correctly followed.
After test execution, collect simulation tool logs, monitoring logs, configuration files, log files and whatever is required by analysis team.
Most importantly timesync information of all servers.
Analysis and Reporting
Nothing wrong in Newbie doing analysis. But?? (read the last point)
Reporting should focus and answer for every question in requirement document. Nothing wrong in providing value add. Also nothing wrong in missing value add. But make sure customer gets what he wants. There is no point is giving value addition without addressing customers basic requirements.
Recommendation should have substantiation or else say best practice.
Review the report. After customer is expecting only this from us.
Closure
Anything gone wrong? Why? Make sure that it is not getting repeated in future project
Appreciate who ever deserves. After all PE guys doesn’t only work for money.
Appreciation must be given for fresher who did excellent job in scripting inspite of this being his/her first project.
Nothing wrong in Newbie doing analysis. But?? (read the last point)
Reporting should focus and answer for every question in requirement document. Nothing wrong in providing value add. Also nothing wrong in missing value add. But make sure customer gets what he wants. There is no point is giving value addition without addressing customers basic requirements.
Recommendation should have substantiation or else say best practice.
Review the report. After customer is expecting only this from us.
Closure
Anything gone wrong? Why? Make sure that it is not getting repeated in future project
Appreciate who ever deserves. After all PE guys doesn’t only work for money.
Appreciation must be given for fresher who did excellent job in scripting inspite of this being his/her first project.
0 comments:
Post a Comment