Creativity and Consistency in Security Testing

Most penetration testing teams face a common problem: how do we balance the need for creativity in security testing, with the need for a standardized test that does not vary from person to person? How do we ensure that the results of our test are equally good whether it’s done by Tom, Dick or Harry?

At Paladion, we try to solve the consistency problem in four ways:

• All of us use the same, detailed checklist.
• All of us undergo similar training.
• We change teams to see more ways of thinking.
• We do quality checks of all our tests.

We then encourage our testers to go outside the checklist whenever required, to find as many holes as they can. And they frequently do. Surprisingly, creativity thrives amidst plans and checklists.

Abraham Maslow studied this creative process at the workplace. Security testers will relate to his analysis:

The creative person is able to be flexible; he can change course as the situation changes (which it always does); he can give up his plans, he can continuously and flexibly adapt to the law of the changing situation and to the changing authority of the facts.

This means that he does not need a fixed and unchanging future. For the creative person who is able to improvise, plans are no more than heuristic scaffoldings and can be cast aside easily without regret and anxiety. He tends not to feel irritated when plans change or schedules change or the future changes. On the contrary, my impression is that he is sometimes apt to show increased interest, alertness and engagement with the problem.

* Addition to the Notes on the Creative Person, Maslow on Management, pages 229 - 230.

Two instances when our teams faced such unexpected situations come to mind:

1. An online banking application we were testing used a Java applet to encrypt all communication between the client and the server. The testers could not meaningfully modify the traffic after intercepting it, and that’s an essential requirement for most tests.
2. An online trading application required client-side SSL certificates to authenticate the user before giving any access to the application. And the testers were not given valid certificates to find holes (that was part of the requirement).

Faced with such difficult odds, the testers were challenged well beyond our traditional checklist. And rarely do you see teams so driven to break the impasse. Maslow’s observation “increased interest, alertness and engagement with the problem” is exactly what we see at such times.

Plynt provides penetration testing and code review services to clients worldwide. If you are interested, please contact us for a quote. We’ll get back to you within one working day.

Add yours.closed for this post.