Monthly Archives: September 2014

Neves’s Laws of Testing

I got into a heated argument over testing practices at work the other day. That led me to think about the subject quite a bit, and have come up with a few laws of testing based on my experience:

  1. Only mock what’s needed to make the test deterministic and fast; otherwise call the real code
  2. A flaky test is worse than no test at all
  3. Good tests verify behaviour; bad tests verify an implementation

The worst thing about law number 3 is that it’s just stating the commonly known knowledge that coupling is bad and should de reduced, but for some reason some people don’t realise they’re increasing it when they write tests for the implementation.

Law number 1 needs some explaining. Mostly I’m only interested in mocking/stubbing/etc. functions that make a unit test no be “pure”. Usual suspects are file system operations, networking, databases, etc. I’ve seen a few examples that don’t fit the “traditional” examples but make good canditates. One that particularly opened my eyes was using a mock log object to state that error messages were logged when calling a function with certain arguments. Another situation is that in some embedded contexts it might be hard to even get a file to compile outside of a specialised environment. Mocking can make sense there too. Basically, whatever makes testing difficult should be mocked. Or to paraphrase Einstein, “All code should be mocked as little as possible, but not less”.

What are your testing laws? How do you disagree with mine?



Emacs as my IDE: homecoming

Once upon time I wrote code using Turbo Pascal and Borland C. Then I discovered protected mode and heard of a compiler called DJGPP that could emit 32-bit code that could access more than 640k of memory. I had to have it. Unfortunately I was used to my text-based IDEs and had no idea how to use a compiler from the command-line. RHIDE and its near perfect imitation of the Borland interface won me over.

At some point that I don’t quite remember (probably when I went to CERN) I started using Linux and Emacs. Emacs was weird. None of the keystroke conventions of other programs worked. It did its own thing. It took a while but I got used to it and even modified my .emacs file, although never so much as to be in a foreign place on a new installation. I’d only really copy elisp from the internet. I ended up using Emacs to write pretty much any code, except for Java. I wrote that in Eclipse.

When I joined Cisco, everyone in my team was using Eclipse in a Linux VM running on Window to write C and C++. I joined the team and started resenting Eclipse more and more each day. At one point I managed to configure a Windows XP VM running Visual Studio that used up less RAM than Eclipse on Linux! But I had to give it to the IDE: it made many things easier when working on a shared codebase. I longed for Emacs but I didn’t want to give up the IDE features that I now relied on, in order of importance:

  1. Jump to definition.
  2. Autocompletion.
  3. On-the-fly syntax checking
  4. Finding a file in the project
  5. Macro expansion
  6. Rename refactoring

If only I could find a way of having this in Emacs… I knew there must be a way, but I also knew it wasn’t going to be easy. So I procrastinated for years. Until last year.

Last year I went into a mad flurry of optimising my productivity. I tried different Linux distros, desktop environments, the works. I changed a lot of the way that I work. My Linux distro went from Kubuntu to Arch Linux. My shell, from bash to zsh. My favourite language was C++, now it’s D. And my “IDE” is now Emacs.

I use auto-complete-clang and flycheck for items 2 and 3 above, and until a few days ago, cscope for item 1. But there were problems since the compiler flags used to build the project must be known for all of this to actually work. But my build system knows the compiler flags, so how do I get it to pass those flags on?

Well, this is an Emacs post so the answer is of course, to write a new elisp package which I called cmake-ide, because that’s what I always use for C and C++ development. But cscope was still annoying me with its fake matches. What if I could use clang for not only autocompletion and on-the-fly highlighting but also for finding definitions? Turns out I can with rtags! I integrated this into cmake-ide and now life is good.

I’m never leaving Emacs now. I have everything I had with Eclipse, and the features I missed even work better than they did there. I version control my .emacs.d directory. I write unit tests in elisp to make sure that my packages don’t break. And I’m more productive for it. Life is good.