Input wanted for an article. Let's say that your company is considering the purchase of an expensive new application to help in the company's software development. The demo looks great, and the...
Input wanted for an article.
Let's say that your company is considering the purchase of an expensive new application to help in the company's software development. The demo looks great, and the feature list makes it sound perfect for your needs. So your Management arranges for a proof of concept license to find out if the software is worth the hefty investment. The boss comes to you to ask you to be in charge of the PoC project.
I'm aiming to write an article to help developers, devops, and testers determine if a given vendor's application meets the company's needs. The only assumption I'm making is that the software is expensive; if it's cheap, the easy answer is, "Buy a copy for a small team and see what they think." And I'm thinking in terms of development software rather than enterprise tools (e.g. cloud-based backup) though I suspect many of the practices are similar.
Aside: Note that this project is beyond "Decide if we need such a thing." In this scenario, everyone agrees that purchasing a tool is a good idea, and they agree on the baseline requirements. The issue is whether this is the right software for the job.
So, how do you go about it? I'm sure that it's more than "Get a copy and poke at it randomly." How did (or would) you go about designing a PoC project? If you've been involved in such a project in the past (particularly if the purchase wasn't ideal), what advice could someone have given you to help you make a better choice? I want to create a useful guide that applies to any "enterprise-class" purchase.
For example: Do you recommend that the PoC period be based on time (N months) or workload (N transactions)? How do you decide who should be on the PoC team? What's involved in putting together a comprehensive list of requirements (e.g. integrates with OurFavoredDatabase, meets performance goals of X), creating a test suite that exercises what the software dev product does, and evaluating the results? ...and what am I not thinking of, that I should?