‘Continuous Integration in .NET’ coauthor pens intro to CI, has more opinions than advice

Craig Berntson wrote Software Gardening: Harvesting (Continuous Delivery Techniques) on dotnetcurry a few days ago. It landed on The Morning Brew, so I gave it a read. The entire piece is tailored to TeamCity, which I wouldn’t have noted at all, except he goes to some trouble during the opening to explain that TeamCity is best-of-breed, and while other tools exist, like Jenkins, they aren’t as good. STRIKE ONE, BABY! To his credit he honestly explains that TeamCity is $2000 to use in any meaningful way, but instead of just stopping there he goes on to explain to anyone thinking “hey, Jenkins is great too–and it’s free!” that Jenkins specifically is not really free. Because of all the time you’ll waste dealing with XML and manually configuring stuff. STRIKE TWO!

I’ve used a lot of CI products. The next one that comes out-of-the-box with no manual configuration required and no XML to screw with wins my eternal allegiance. It will also be the first one I’ve ever seen. We recently reviewed the latest TeamCity at work (we use Jenkins for CI right now) and had to screw around with not just a bunch of XML files but also some awesome giant Java properties files to get LDAP and some other things working.

Jenkins required us to install the AD plugin and change a radio button to “Use Active Directory” and it just worked. I guess that means that free TeamCity isn’t really free. Or that Jenkins is more free? Or maybe just that we didn’t know shit about TeamCity and using new OTS software is hard.

In all seriousness, there is basically no TeamCity community except for the poorly attended TeamCity issue tracker on JetBrains’ own site, so getting help for TeamCity is a crapshoot, whereas there are >500 active Jenkins developers just on GitHub and 10s of 1000s of Jenkins users around the world. The power of the community is pretty awesome, which you get to experience whenever you need help. Not calculating that into whatever XML editing or manual config you have to do is a pretty huge disservice, but that probably speaks more to Craig’s lack of experience with Jenkins (his book ignored Hudson completely when it was published in 2011, even though people like me had been using it for years to do .NET CI).

What follows is the reworking of a comment I left on the article that I wanted to preserve to reference in the future when I’m inevitably arguing with some purists about how .NET CI can only work in the one specific way they know how to do it:

It was an excellent CI concept overview, but I have three critical notes:

#1 Telling .NET CI novices “Never install Visual Studio on your build server” with zero explanation of your *preference* is unreasonable in the extreme. Not having the VS build chain on the server is asking for trouble in many, many scenarios. The blogosphere is alight with very smart people complaining about how MS needs to make this process both easier and possible because these aren’t just edge cases where having VS is needed. The easiest way to get the .NET CI novice moving in the right direction is to tell them to install VS on the build server, analyze their dependencies, and maybe later suggest doing it over again without VS as a varsity-level project.

Assembling your own build chain is not trivial unless you’re building the barest of vanilla code, and ends up scaring most people away from CI altogether after their first few problems where the code builds fine on their developer machine but won’t build at all on their homemade CI toolchain. Did MS sucker you into using MSTest? Good luck getting that working without VS installed. If you need anything later than 2010 support it’s all but impossible. Working on a project type that’s using a non-standard runtime like VSTO? Need to build a VS-only project type like an installer? Microsoft Build Tools laughs in your face when you try that. Just install VS locally and don’t ask rookies to waste days or weeks building or rebuilding everything to accommodate your *preference*.

#2 FXCop works great on all .NET versions unless you stubbornly insist on running the 6 year old free version. If, however, you install VS on the build server (and fix the codeanalysispath and fxcopdir env vars if you’re upgrading), you can run e.g. \Microsoft Visual Studio 12.0\Team Tools\Static Analysis Tools\FxCop\fxcopcmd.exe against any .NET code and it works great.

Also, it doesn’t take much time to run FXCop–maybe the ancient free version is really slow, but fxcopcmd.exe from VS2013 is nice and speedy. On our longest full site builds FXCop accounts for <1% of build time and pays huge dividends with invaluable code quality feedback. Just let a few CA2000s slip into production in the wrong loop and see how everything comes to a mysterious grinding halt until you hand-debug it. Good luck!

#3 Craig’s Team City bias is completely unwarranted at this advanced stage of CI tool awesomeness. At least he had the presence of mind to compare it seriously to Jenkins instead of only mentioning CruiseControl and TFS like in his book, but literally everything he described Team City doing can be done on Jenkins with similar UI effort and *zero* use of XML. I nearly stopped reading after he complained that “always free” Jenkins requires you to mess with a lot of XML, because in the *next paragraph* he displayed a giant MSBuild XML file and explained its innards in great detail, proving handily that his target audience is more than capable of dealing with it–should it even come up (spoiler: it won’t).

Jenkins has >1000 plugins for talking to a huge array of tools and services and scales out really well with no use for a database. I noticed that TeamCity has serious plugin support since the last time I looked and appears to have about 50 plugins–14 of which they’ll actually let you see the source code to! Based on how closed off it is, the only thing Team City seems specifically optimized for is generating purchase orders to JetBrains for support and features.

After using Hudson for years to do .NET CI at several companies, I read Craig’s book when it came out because the Jenkins/Hudson fork had just occurred and I wanted to see what else was out there in case Oracle had truly ruined it for good. Because of Craig, I ended up seriously reviewing Team City, Cruise Control, and TFS. I loved his book, but I ended up hating TC and CC since both had hellish never-ending wizard-driven UIs and extremely limited integration with other tools and systems, comparatively speaking. I came to the same conclusion about TFS that he did in the article (their horrible UI in 2010 didn’t help their case, either). I ended up following the Hudson community to Jenkins and I’m proud to say I’ve left a trail of happy and capable CI users in my wake at various companies where I’ve setup operations.

As a Jenkins user, I’ve been able to look at the source code for both the core product and the numerous plugins I rely on for fetching, building, testing, and analyzing .NET and JS code (we use a lot of stuff–FxCop, Checkmarx, dotCover, NCover, MSTest, NUnit, JSHint, Jasmine, blanket, etc.). Even though I’m not a professional Java developer I’ve been able to troubleshoot and help fix my own product/plugin bugs over the years due the way core code, issues, and plugins are managed by the community, whereas I have issues and feature requests sitting on the JetBrains tracker for TC, R#, and dotCover that are years old with zero responses.

I really think it’s time he considers more acceptance criteria than whatever XML problem made him hate Jenkins/Hudson however long ago and give Jenkins a more serious look for .NET CI. It takes about 5 minutes and 20 clicks to install and integrate with Active Directory. Building solutions from the SCM of your choice and testing, covering, and analyzing .NET and JS code isn’t much farther in the distance. There are also several great plugins that help you generate workflows and automate deployments for continuous delivery, too.

Posted in Uncategorized | Leave a comment

Finding a TFS 2012 generic work item URL is way harder than is reasonable

You’d think it would be easy to find out how to get a direct link to any work item in a collection in TFS, but it’s not. In 2010 there was an URL format that worked, but they ditched that when 2012 came out and didn’t really document any replacement. When you search for work items in 2012, you get a link that includes the project name like


So it’s only accurate for work items in that project. Not really suitable for a generic link to include in a CI report or to tokenize work item references in external systems (like SmartBear’s Collaborator). Argh!!

Bart Wullems to the rescue! Bart wrote up an article that got me on the right track. In VS2012, Team Explorer contained a context menu item called “Copy Full Path”. This is gone from VS2013, much to my confusion and dismay, but I had a machine with VS2012 still pointed at our TFS and I was able to grab the URL as in Bart’s article. The 2012 generic URL format is:


You can enumerate your collections from the TFS API to find the GUID if you don’t have a copy of VS2012 lying around, or you can do what most people do after using TFS for a few years and just buy a better issue tracker that doesn’t make your life a living hell. I’ve heard rumblings that business users are looking at JIRA, which I’ve used in the past and enjoyed very much, so I’ll be encouraging them any way I can. Personally I think FogBugz is the greatest tracker ever created, but apparently my vote doesn’t count since I’m just a developer 😉

Posted in Code Review, Continuous Integration | Leave a comment

RegExr is awesome

Yes, there are a ton of regex tools out there, but RegExr easily stands out. It’s realtime checker is really slick and the app is loaded with helpful docs and references. Thank you RegExr!!

Posted in Coding | Leave a comment

Offered without comment


Posted in Uncategorized | Leave a comment

Testing software is hard?

Yeah, it’s hard. As much handwaving as there is in this article about code coverage (by a developer who works for a code coverage company, so get your grains of salt ready) about how simple Tetris is–it can be implemented in a single line of BBC Basic after all, and everyone knows “lines of BBC Basic” is becoming the hot new unusual unit of measurement–his article is actually about the many difficult-to-produce corner cases from the user interface’s perspective in an actually useful implementation of the game.

There are two primary paths people consider to gather code coverage data. On my current team at work, we typically use automated unit tests. Our coverage tool invokes the unit test console runner with arguments about which DLLs contain tests and what code to ignore, and we publish the resulting HTML reports and trend numbers on our CI server. The linked article is taking another popular approach: have actual humans run the application while the coverage tool watches and reports on what code they hit and what code they missed.

I’m not a huge fan of human-only testing–even when they are following a written script closely they might miss something or “do it wrong”, as we developers like to say about users, and even when they make videos of the entire testing session that just creates another problem in the form of huge volumes of material to review to find issues. That said, users or automated UI tests definitely answers some interesting questions about the code that unit tests cannot, like “is any of our code unreachable from the UI?”

For example, when there are known corner cases where code is extremely difficult to reach deterministically from the UI (as in the article), that would be great to know so you can do something like create a UI shortcut to manually invoke it, or otherwise increase your coverage with specific automated unit or integration tests.

All that said, I have to hand it to the author for making code coverage actually fun and engaging–this is a huge challenge by itself, so kudos to him!

Posted in Coding, Continuous Integration | Tagged , | Leave a comment

Wow, really great RegEx support coming with ReSharper 9

I’m a diehard ReSharper fan. I’ve had my own license for many years, and as far as VS has come over that time with refactorings and ease of navigation I still prefer to have the power of R# at my fingertips.

ReSharper 9 brings all new Regular Expression support: error and expression highlighting, four types of code completion, escape character help, and a very nice validation utility that lets you experiment with your expression on test input text against the .NET or JS regex rules, similar to the test capabilities in something like RegEx Buddy (which is awesome, btw), but built into VS.

DISCLAIMER: THIS IS NOT AN ENDORSEMENT OF REGEX! As the saying goes, ‘Some people, when confronted with a problem, think “I know, I’ll use regular expressions.” Now they have two problems.’ I fully subscribe to this, but I have to maintain a non-trivial amount of code with some hairy regexes and it will be great not to have a run a bunch of utilities or scripts to test something out quickly.

Posted in Coding | Tagged | Leave a comment

Corporate Social Networks Swallow Your Creativity

I’ve actually been writing an enormous volume of technical information this year, but mostly at work on a crappy enterprise social network that’s not even seen by that many people. They finally upgraded it to a barely tolerable level today (you couldn’t really post pictures on it plus 1,000,001 other terrible basic problems) and I now realize that even as crappy as it was I’ve been using it too much. I should be writing here, or basically anywhere except there.

I’ll keep trying. I have to get better at code formatting-fu because every time I want to post something here I feel like I have to review a bunch of thousand year old FAQs about how to post code snippets and it feels SO PAINFUL. Ugh. Anyhow…onward.

Posted in Uncategorized | Leave a comment