Feature: Page (1) of 2 - 04/20/06 Email this story to a friend. email article Print this page (Article printing at MyDmn.com).print page facebook

When Do You Quit Security Testing and Ship the App?

Are we done yet? No. Of course not. By Esther Schindler
When Do You Quit Security Testing and Ship the App?Many years ago, a computer consultant named Stan was given the assignment of writing a database application. Because of its business requirements, one unique feature of the application was that it was unusual to delete a record; it could happen, but deletions were rare. So, when the user marked a record for deletion, Stan popped up a dialog box saying, "Are you sure?" If the user replied in the affirmative, another said, "Are you really sure?" Yes? The application asked, "Would you bet your job on it?"

Stan had to change the code, because -- even when deleting a record was the right action -- no employee would ever bet his job on it.

Companies who develop new applications are in a similar position. Stan added so much security that the application no longer achieved its goal. Nowadays, developers need to provide an adequate amount of protection from hackers, without going overboard (and missing their deliverables). In other words, how does a security tester know she's done?

With functional testing, a product spec enumerates what the application should do. If the app does that, you're finished. With performance testing, you set metrics for acceptable speed or other criteria; when the app matches those, you can dust off your hands and go out for a beer. (Please invite me. I like good beer.) But with security testing, you only know about the holes you plugged. Anybody who really really wants to get in will put more energy into the break-in than you can into testing for vulnerabilities. There is no real benchmark to certify that your product is fully secure.

So, how do you define your suite of security tests to know that the job is done, and that you can go out for that beer?

Let's Start With the End: You're Never Done
The French poet, Paul Valery, said, "A poem is never finished, only abandoned." The same could be said for any kind of quality testing, but it's particularly true of security testing. According to Richard Moore, a software engineer in Manchester, England, the way to know you're done is, "Generally it's when the product has been end of lifed. Seriously." As Moore points out, "New attacks are being developed all the time, so there's no point at which you can say you're finished. Add to this the fact that you invariably get new features added as a product develops, and you see that security testing is an ongoing process."

In some situations, you can release the software based on time or budget constraints, and update it in point releases. (Assuming that Management doesn't put you on another project "temporarily," and you never get back to this one to plug the holes.) Recommends Sol, a developer in the northeast U.S., "Find as many items [as possible] and get them fixed by a cut-off or Go Live date. Even so, once your customers start using them in the many different environments, that is when your bugs will be discovered." Expect to release a few quick 1.0.1 or 1.0.9 versions.

Deal with the resources you're given and test as far as you can in the time allotted, Sol says. "The closer to the go live date, the more QA and management must be in communication with each other to determine what bugs are 'show stoppers' and get them fixed before the go live date is reached."

Add-more-later is a possibility if you're on staff, but is much less feasible if you're a consultant or other outsider. If you have less control over the schedule, Arturo Busleiman, a security consultant in Argentina, recommends, "You start defining the scope, then you do it. If you see that some aspect of the system still gives you doubts, you talk to the person who hired you, and ask for a subproject relating [to] those issues."

Page: 1 2 Next Page

Related Keywords:security, application development, programming, qa, quality assurance, design


Our Privacy Policy --- @ Copyright, 2015 Digital Media Online, All Rights Reserved