Craig Berntson wrote Software Gardening: Harvesting (Continuous Delivery Techniques) on dotnetcurry a few days ago. It landed on The Morning Brew, so I gave it a read. The entire piece is tailored to TeamCity, which I wouldn’t have noted at all, except he goes to some trouble during the opening to explain that TeamCity is best-of-breed, and while other tools exist, like Jenkins, they aren’t as good. STRIKE ONE, BABY! To his credit he honestly explains that TeamCity is $2000 to use in any meaningful way, but instead of just stopping there he goes on to explain to anyone thinking “hey, Jenkins is great too–and it’s free!” that Jenkins specifically is not really free. Because of all the time you’ll waste dealing with XML and manually configuring stuff. STRIKE TWO!
I’ve used a lot of CI products. The next one that comes out-of-the-box with no manual configuration required and no XML to screw with wins my eternal allegiance. It will also be the first one I’ve ever seen. We recently reviewed the latest TeamCity at work (we use Jenkins for CI right now) and had to screw around with not just a bunch of XML files but also some awesome giant Java properties files to get LDAP and some other things working.
Jenkins required us to install the AD plugin and change a radio button to “Use Active Directory” and it just worked. I guess that means that free TeamCity isn’t really free. Or that Jenkins is more free? Or maybe just that we didn’t know shit about TeamCity and using new OTS software is hard.
In all seriousness, there is basically no TeamCity community except for the poorly attended TeamCity issue tracker on JetBrains’ own site, so getting help for TeamCity is a crapshoot, whereas there are >500 active Jenkins developers just on GitHub and 10s of 1000s of Jenkins users around the world. The power of the community is pretty awesome, which you get to experience whenever you need help. Not calculating that into whatever XML editing or manual config you have to do is a pretty huge disservice, but that probably speaks more to Craig’s lack of experience with Jenkins (his book ignored Hudson completely when it was published in 2011, even though people like me had been using it for years to do .NET CI).
What follows is the reworking of a comment I left on the article that I wanted to preserve to reference in the future when I’m inevitably arguing with some purists about how .NET CI can only work in the one specific way they know how to do it:
It was an excellent CI concept overview, but I have three critical notes:
#1 Telling .NET CI novices “Never install Visual Studio on your build server” with zero explanation of your *preference* is unreasonable in the extreme. Not having the VS build chain on the server is asking for trouble in many, many scenarios. The blogosphere is alight with very smart people complaining about how MS needs to make this process both easier and possible because these aren’t just edge cases where having VS is needed. The easiest way to get the .NET CI novice moving in the right direction is to tell them to install VS on the build server, analyze their dependencies, and maybe later suggest doing it over again without VS as a varsity-level project.
Assembling your own build chain is not trivial unless you’re building the barest of vanilla code, and ends up scaring most people away from CI altogether after their first few problems where the code builds fine on their developer machine but won’t build at all on their homemade CI toolchain. Did MS sucker you into using MSTest? Good luck getting that working without VS installed. If you need anything later than 2010 support it’s all but impossible. Working on a project type that’s using a non-standard runtime like VSTO? Need to build a VS-only project type like an installer? Microsoft Build Tools laughs in your face when you try that. Just install VS locally and don’t ask rookies to waste days or weeks building or rebuilding everything to accommodate your *preference*.
#2 FXCop works great on all .NET versions unless you stubbornly insist on running the 6 year old free version. If, however, you install VS on the build server (and fix the codeanalysispath and fxcopdir env vars if you’re upgrading), you can run e.g. \Microsoft Visual Studio 12.0\Team Tools\Static Analysis Tools\FxCop\fxcopcmd.exe against any .NET code and it works great.
Also, it doesn’t take much time to run FXCop–maybe the ancient free version is really slow, but fxcopcmd.exe from VS2013 is nice and speedy. On our longest full site builds FXCop accounts for <1% of build time and pays huge dividends with invaluable code quality feedback. Just let a few CA2000s slip into production in the wrong loop and see how everything comes to a mysterious grinding halt until you hand-debug it. Good luck!
#3 Craig’s Team City bias is completely unwarranted at this advanced stage of CI tool awesomeness. At least he had the presence of mind to compare it seriously to Jenkins instead of only mentioning CruiseControl and TFS like in his book, but literally everything he described Team City doing can be done on Jenkins with similar UI effort and *zero* use of XML. I nearly stopped reading after he complained that “always free” Jenkins requires you to mess with a lot of XML, because in the *next paragraph* he displayed a giant MSBuild XML file and explained its innards in great detail, proving handily that his target audience is more than capable of dealing with it–should it even come up (spoiler: it won’t).
Jenkins has >1000 plugins for talking to a huge array of tools and services and scales out really well with no use for a database. I noticed that TeamCity has serious plugin support since the last time I looked and appears to have about 50 plugins–14 of which they’ll actually let you see the source code to! Based on how closed off it is, the only thing Team City seems specifically optimized for is generating purchase orders to JetBrains for support and features.
After using Hudson for years to do .NET CI at several companies, I read Craig’s book when it came out because the Jenkins/Hudson fork had just occurred and I wanted to see what else was out there in case Oracle had truly ruined it for good. Because of Craig, I ended up seriously reviewing Team City, Cruise Control, and TFS. I loved his book, but I ended up hating TC and CC since both had hellish never-ending wizard-driven UIs and extremely limited integration with other tools and systems, comparatively speaking. I came to the same conclusion about TFS that he did in the article (their horrible UI in 2010 didn’t help their case, either). I ended up following the Hudson community to Jenkins and I’m proud to say I’ve left a trail of happy and capable CI users in my wake at various companies where I’ve setup operations.
As a Jenkins user, I’ve been able to look at the source code for both the core product and the numerous plugins I rely on for fetching, building, testing, and analyzing .NET and JS code (we use a lot of stuff–FxCop, Checkmarx, dotCover, NCover, MSTest, NUnit, JSHint, Jasmine, blanket, etc.). Even though I’m not a professional Java developer I’ve been able to troubleshoot and help fix my own product/plugin bugs over the years due the way core code, issues, and plugins are managed by the community, whereas I have issues and feature requests sitting on the JetBrains tracker for TC, R#, and dotCover that are years old with zero responses.
I really think it’s time he considers more acceptance criteria than whatever XML problem made him hate Jenkins/Hudson however long ago and give Jenkins a more serious look for .NET CI. It takes about 5 minutes and 20 clicks to install and integrate with Active Directory. Building solutions from the SCM of your choice and testing, covering, and analyzing .NET and JS code isn’t much farther in the distance. There are also several great plugins that help you generate workflows and automate deployments for continuous delivery, too.