A Context for Software Testing - Lecture Slides | CPSC 6179, Study notes of Computer Science

Download A Context for Software Testing - Lecture Slides | CPSC 6179 and more Study notes Computer Science in PDF only on Docsity! CPSC 6179 Lecture 13 Slide 1 of 12 slides A Context for Software Testing Everybody agrees that software testing is necessary, but 1) Who does the testing? 2) How is the testing to be done? 3) How should automatic test tools be applied? 4) How should code be written to facilitate testing? CPSC 6179 Lecture 13 Slide 2 of 12 slides Who Does the Testing? Unit testing by the developer is necessary, but 1. The code developer might not fully understand the system requirements. 2. The code developer might not have a grasp of system–level issues. 3. The code developer might design the tests so that the code will pass the tests. Independent testers bring value to the process. Bottom Line: Testing must be done by both the code developers and independent testers. Managers must avoid conflicts and “bad feelings” between the two groups. The code developers must see the value of independent testing. CPSC 6179 Lecture 13 Slide 5 of 12 slides Requirements for Automatic Testing 1. Must be able to recreate scenarios that lead to errors. 2. Must be able to execute a large number of tests, each in the same manner every time the test suite is run. 3. Must be able to schedule the tests to be run at a fixed time. Avoid midnight: Is it 12:00 AM or 12:00 PM? I favor 12:15 AM – unambiguous. 4. Must be able to record all results and other artifacts of the test. Bad (actual) example: an automated test program for a graphics engine. The tests were automatic but had to be observed by a human to determine that the graphics were OK. 5. Must have a dedicated error log and a dedicated results log. The test programs must use these. CPSC 6179 Lecture 13 Slide 6 of 12 slides Coding for Testability One good design is based on the “client server model”. The client part handles user interface issues. The server part handles all computation. This allows automatic testing of the “worker part” of the code by direct calls to functions in an API. EXAMPLE: Handling Mouse Clicks The client software displays some object to be clicked. The client software processes the click event, determines if the mouse has moved far enough for the click to be significant (shaky hands), and then passes the click type and mouse coordinates to the server software. We can write a dummy server to verify proper mouse handling call the server software directly, bypassing the GUI. CPSC 6179 Lecture 13 Slide 7 of 12 slides Sample Code (Written in VB Style) Assume a text box called “Text1” with an associated command button, called “DoIt”. We examine the event code for the command button. The event code DoIt_Click is part of the form that handles the GUI. Subroutine DoIt_Click Answer = VB.Trim$(Text1.Text) ‘ Dump leading & trailing spaces If ( VB.Len(Answer) > 0) Then ‘ Demand a non-empty string IsOK = Server.CheckString(Answer) If IsOK Then Server.Process(Answer) We can call these two server functions directly from the Automatic Test software, thus verifying the functionality apart from that of the GUI. CPSC 6179 Lecture 13 Slide 10 of 12 slides And Now for a “War Story” This actually happened to your instructor while he was working at Intergraph Corporation in Huntsville, AL Scenario: Using a script-based language to test a GUI by moving the mouse over the interface and emulating mouse clicks. What Happened: The mouse “escaped” from the GUI window, It landed on the MS-Windows taskbar It then opened my e-mail and opened a message. CPSC 6179 Lecture 13 Slide 11 of 12 slides Negative Productivity Issues As with any new tool, the process will initially be less productive as the tool is learned and adapted to local use. It is safer to run automated testing in parallel with the existing manual testing for the first few projects. It is also less productive. No automated test tool will live up to its advertising hype. Automated testing should be viewed as complementing and enhancing manual testing not replacing it. Initially this is less productive. QUESTION: Should the primary goal be to reduce testing effort or to improve testing quality? (You can tell I have never had to manage a budget.) CPSC 6179 Lecture 13 Slide 12 of 12 slides The TMM (Testing Maturity Model) Developed by the Illinois Institute of Technology. Based on the CMM. Level 1: Initial This level is called “accidental automation”. Testing is ad–hoc or experimental. Level 2: Phase Definition Testing is becoming a planned activity. Level 3: Integration Testing is planned, well defined, and well managed. Level 4: Management and Measurement Defect tracking is added to the process and practiced. Level 5: Optimization, Defect Prevention and Quality Control Testing is driven by statistical sampling and measure of confidence levels. Testing is continuously improved.