Are you an “old timer” in the computer industry? I’m writing a story about the things programmers (and QA people) had to do to test their software. It’s meant to be a nostalgic piece that’ll remind people about old methods — for good or ill.

For example, there was a point where the only way to insert a breakpoint in the code was to insert “printfs” that said “I got to this place in the code!” And all testing was manual testing. Nothing was automated. If you wanted a bug tracking system, you built your own.

So tell me your stories. Tell me what you had to do to test software, way back when, and compare it to today. What tools did you use -- or build? Is there anything you miss? Anything that makes you especially glad that the past is past?

C’mon, you know you wanted a “remember when”!

Input wanted for an article.

Let's say that your company is considering the purchase of an expensive new application to help in the company's software development. The demo looks great, and the feature list makes it sound perfect for your needs. So your Management arranges for a proof of concept license to find out if the software is worth the hefty investment. The boss comes to you to ask you to be in charge of the PoC project.

I'm aiming to write an article to help developers, devops, and testers determine if a given vendor's application meets the company's needs. The only assumption I'm making is that the software is expensive; if it's cheap, the easy answer is, "Buy a copy for a small team and see what they think." And I'm thinking in terms of development software rather than enterprise tools (e.g. cloud-based backup) though I suspect many of the practices are similar.

Aside: Note that this project is beyond "Decide if we need such a thing." In this scenario, everyone agrees that purchasing a tool is a good idea, and they agree on the baseline requirements. The issue is whether this is the right software for the job.

So, how do you go about it? I'm sure that it's more than "Get a copy and poke at it randomly." How did (or would) you go about designing a PoC project? If you've been involved in such a project in the past (particularly if the purchase wasn't ideal), what advice could someone have given you to help you make a better choice? I want to create a useful guide that applies to any "enterprise-class" purchase.

For example: Do you recommend that the PoC period be based on time (N months) or workload (N transactions)? How do you decide who should be on the PoC team? What's involved in putting together a comprehensive list of requirements (e.g. integrates with OurFavoredDatabase, meets performance goals of X), creating a test suite that exercises what the software dev product does, and evaluating the results? ...and what am I not thinking of, that I should?