Hi all,

I'm looking for some advice regarding how to set up a basic CI regression / testing suite. This isn't my full time job, but a side project my group at work wants to spin up to... shall we say, give us a more real time monitoring of functionality and performance regressions coming out of the underlying software stack development (long story).

As none of us are particularly automation experts, I was looking for some advice from my fellow Tilderinos. Please forgive me if any of the below is obvious and/or silly.

A few basic requirements I had in mind:

1. Can handle different execution environments: essentially different versions of the software stack, both in docker form and (eventually) via lmod or some other module file approach (e.g., TCL), and sensible handling of a node list.

2. Related to one, supports using the products of builds as execution environments. Ideally we'd like to have a build step compile the stack and install it to a NFS from which we can load it as a module.

3. Simple to add tests. Again, this isn't our full time job -- we mostly want to add a quick bash script / makefile / source code or the like to the tests when we run into an issue and forgot about it.

4. Related. We should be able to store the entire thing as a git repo. I have seen this to some extent with Travis, but my experience with Jenkins was... sub-par (is there a history? Changelog? Any way at all of backing up the test config?).

5. Some sort of post-processing capabilities. At a glance we need to be able to see the top line performance numbers for 20-30 apps over the different build environment. Bonus points if there's a graph showing performance vs build version or the like, but honestly a CSV log file is good enough.

6. Whatever CI software we get has to be able to run this locally. Lots of these are internal only numbers / codes. FOSS prefered.

7. A webui for scheduling runs / visualizing results would be nice, but again this could be a bash script and none of us would bat an eye.

Any thoughts would be greatly appreciated. Thanks!

What have you been listening to this week? You don't need to do a 6000 word review if you don't want to, but please write something! If you've just picked up some music, please update on that as well, we'd love to see your hauls :)

Feel free to give recs or discuss anything about each others' listening habits.

You can make a chart if you use last.fm:

http://www.tapmusic.net/lastfm/

Remember that linking directly to your image will update with your future listening, make sure to reupload to somewhere like imgur if you'd like it to remain what you have at the time of posting.

This topic is part of a weekly series. It is meant to be a place for users to discuss their weekend.

If you have any plans, things you want to get done, things you have done, things you haven't done, or even if you just want to talk about how you're doing this weekend, this is a place for casual discussion about those things.

A list of all previous topics in this series can be found here.

So, what (or how) are you doing this weekend?