Tag Archives: build systems

2019/04/03
14 Comments
Programming

Build systems are stupid

Big complicated projects nearly always need a build system. They might be generating code. They might be compiling different branches depending on the platform or a build-time configuration option. They might turn on/off unity builds for C++.  For most simple projects however, build systems are stupid.

Let’s say you have a single C file. We need to tell the compiler the name of the file we want to compile, and the name of the resulting binary unless we’re happy with it being called a.out, which is unlikely:

gcc -Wall -Wextra -Werror -o foo foo.c

Maybe it’s the first commit in brand new repository and all you have is foo.c in there. Why am I telling the compiler what to build? What else would it build??

$ ls
foo.c
$ gcc
gcc: fatal error: no input files
compilation terminated.

“Real projects don’t look like that!”. Ok:

$ ls
build include src 
$ ls src
foo.c main.c
$ ls include
foo.h

I doubt any humans who have ever written C before would be confused as to how this project is supposed to be compiled. And yet, this is probably the simplest way I can come up with to tell the computer to do it properly:

CFLAGS ?= -Iinclude -Wall -Wextra -Werror -g

SRCS := $(shell find src -name *.c)
HDRS := $(shell find include -name *.h)
OBJS := $(subst src,build,$(subst .c,.o,$(SRCS)))

build/foo: $(OBJS)
	gcc -o $@ $^

build/%.o: src/%.c $(HDRS)
	gcc $(CFLAGS) -o $@ -c $<

Is there a better way than $(subst)? Probably. I don’t care. I was also too lazy to figure out dependencies so all source files depend on all headers. That’s a lot of lines of Makefile and all I wanted to specify were that source files are in src, headers are in include, binaries go inbuild, and that the compiler flags are -Wall -Wextra -Werror  -g. The first three are obvious from the existence and names of those directories, so there are 8 lines of code here to tell a computer what the compiler flags are. Robots: 1 Humans: 0.

CMake isn’t that much better:

cmake_minimum_required (VERSION 2.8.11)
project (foo)

file(GLOB SRC_FILES ${PROJECT_SOURCE_DIR}/src/*.c)
add_executable(foo ${SRC_FILES})
target_include_directories(foo PRIVATE ${PROJECT_SOURCE_DIR}/include)

Robots: 2 Humans: 0.

I’ll probably get told off for using file(GLOB) because what I really should be doing is manually listing source files because reasons. Why would I have files in src if I don’t want them to be compiled? In what universe does it make sense for a person whose job it is to tell computers what to do so that they don’t have to do it themselves to tell a build system which files to build? In my opinion every new language should default to doing what Pony does. Running the compiler just builds everything in the current directory tree.

I also think that new languages should ship with a compiler-based build system, i.e. the compiler is the build system. The reason for that is that as a developer, I want to rebuild the minimum possible amount of code that is required after my edits to run my tests. The CMake solution above fails at this: the compiler calculates dependencies and automatically feeds it into the build system, but those dependencies are too coarse.

If I touch any header, all source files that #include it will be recompiled. This happens even if it’s an additive change, which by definition can’t affect any existing source files.  If I change a function’s signature that is only used in foo.c, well tough: I’ll have to wait until bar.c, baz.c and quux.c all get recompiled for the temerity of depending on different functions from the same header.

This problem is bad enough in C and C++ but actually gets worse for languages with modules: now every file is a “header”. Changes to the implementation trigger recompilations, because our build systems are too stupid to know any better. I can’t see how a good build system can be an external tool unless the compiler is a library, and even then it would be doing the same work twice since dependency calculations are better done while a module/translation unit is being compiled.

Taking it a step further, the whole concept of caring about files and modules is probably outdated anyway. Why not a Smalltalk-like environment? You make changes to the source and your “image” gets rebuilt in the background, one part of the AST at a time, automagically resolving and rebuilding dependencies (at the function level!).

Computers and build systems: what have you done for me lately?

Advertisement
Tagged , , , , ,
2017/10/16
2 Comments
Programming

Why I find developing on/for Windows exasperating

I ran DOS on my first PC. The natural progession unfolded, with me then running Windows 95, Windows 98, and Windows XP after that (Windows ME, like the Matrix sequels, was a collective bad dream that didn’t really happen). I used Borland’s IDE to write C code, then RHIDE with DJGPP since I couldn’t even imagine using a compiler from the command-line. I say that because I wasn’t “brought up” using *nix at all, and my only exposure was at university. These days however, I do nearly all of my development on Linux. Why? I find it to be a much, much better experience.

Somewhat unfortunately for me, my current job requires me to do Windows development. And every time I boot into Windows or have to fix Windows-specific problems, it makes me want to cry. Why? Let me name some of the reasons why.

Speed, or the lack thereof. I haven’t done a thorough scientific analysis on this, because I don’t think it’d be worth my while to do so. It seems clear to me that NTFS is very very slow. Doing anything on it, from running CMake to compiling to linking, seems to take forever. To the point that it makes me actively wonder how anyone manages to get anything done on Windows. I can rebuild the reference D compiler on my laptop in about 1.6s after modifying one file. On Windows the same build, on the same machine, takes ~1 minute. Given that I find 1.6s infuriatingly slow, you can imagine what sorts of dark swear words I reserve for waiting for a whole minute while what would have been considered a supercomputer a few years ago decides to go get anything done.

Dependencies. Unlike *nix, there is no standard path(s) to look up libraries. Granted, even different Linux distros use different conventions and paths from each other, but libraries are usually installed with a package manager anyway so mostly you don’t care.  And if you did, your linker would find them anyway without the need for extra flags. Need to link to, say, nanomsg on Windows? Good luck with that. Ah, but there’s vcpkg, I hear you say. Apparently Visual Studio auto-magically finds the libraries that vcpkg “installs”. Job done if you’re clicking a button in an IDE, not so much if you’re using a real build system running in CI. It _could_ be just as easy as adding a flag to your linker, but, alas, the .lib files don’t all end up in the same directory. vcpkg allows me to download libraries without having to write Powershell, but then actually linking is, for lack of a better word, “fun”. On Linux? pacman -S nanomsg; ninja

Batch files and/or powershell. I personally find bash horrible to write code in, but then I do Windows work and remember there’s worse. So much worse. Sigh.

Bash. I’ll explain. Git bash is amazing, I remember a time before that existed (I tried, unsucessfully, to compile bash from source for Windows with at least 3 different implementations back in the day). So why am I complaining? First of all, because I use zsh and haven’t seen an easy way to do that yet on Windows. Secondly, because building on Windows from the command-line often requires cmd.exe. Building C++ code? I’m not going to write my own bash version of vcvarsall.bat just to do that. Commands have a habit of spitting out error messages with backslashes (cos, duh, Windows), and good luck copying and pasting that into your bash shell.

Tooling. Want to create a zip? You’ll have to download and install a 3rd party tool. Oh, but the binary doesn’t get added to the PATH, so you’ll have to write out the full path in your batch file and pray one of your machines doesn’t install it to a different location.

Things are better than they used to be on Windows. We now have the Linux subsystem, git bash, and alternatives to the horrible built-in terminal emulator. To me, it just makes things less bad, and the moment I’m back on Arch Linux it feels like coming home from a not particularly good holiday.

Tagged , , ,
2015/02/20
3 Comments
Uncategorized

Haskell actually does change the way you think

Last year I started trying to learn Haskell. There have been many ups and downs, but my only Haskell project so far is on hold while I work on other things. I’m not sure yet if I’d choose to use Haskell in production. The problems I had (and the time it’s taken so far) writing a simple server make me think twice, but that’s a story for another blog post.

The thing is, the whole reason I decided to learn Haskell were the many reports that it made me you think differently. As much as I like D, learning it was easy and essentially I’m using it as a better C++. There are things I routinely do in D that I wouldn’t have thought of or bother in C++ because they’re easier. But it’s not really changed my brain.

I didn’t think Haskell had either, until I started thinking of solutions to problems I was having in D in Haskell ways. I’m currently working on a build system, and since the configuration language is D, it has to be compiled. So I have interesting problems to solve with regards to what runs when: compile-time or run-time. Next thing I know I’m thinking of lazy evaluation, thunks, and the IO monad. Some things aren’t possible to be evaluated at compile-time in D. So I replaced a value with a function that when run (i.e. at run-time) would produce that value. And (modulo current CTFE limitations)… it works! I’m even thinking of making a wrapper type that composes nicely… (sound familiar?)

So, thanks Haskell. You made my head hurt more than anything I’ve tried learning since Physics, but apparently you’ve made me a better programmer.

Tagged , , , , , ,