Category Archives: Programming

Type inference debate: a C++ culture phenomenon?

I read two C++ subreddit threads today on using the auto keyword. They’re both questions: the first one asks why certain people seem to dislike using type inference, while the second asks about what commonly taught guidelines should be considered bad practice. A few replies there mention auto.

This confuses me for more than one reason. The first is that I’m a big proponent of type inference. Having to work to tell a computer what it already knows is one of my pet peeves. I also believe that wanting to know the exact type of a variable is a, for lack of a better term, “development smell”, especially in typed languages with generics. I think that the possible operations on a type are what matters, and figuring out the exact type if needed is a tooling problem that is solved almost everywhere. If you don’t at least have “jump to definition” in your development environment, you should.

The other main source of confusion for me is why this debate seems restricted to the C++ community, at least according to my anectodal evidence. As far as I’m aware of, type inference is taken to be a good thing almost universally in Go, D, Rust, Haskell, et al. I don’t have statistics on this at all, but from what I’ve seen so far it seems to be the case. Why is that?

One reason might be change. As I learned reading Peopleware, humans aren’t too fond of it. C++ lacked type inference until C++11, and whoever learned it then might not like using auto now; it’s “weird”, so it must be bad. I’ve heard C programmers who “like” looping over a collection using an integer index and didn’t understand when I commented on their Python code asking them to please not do that. “Like” here means “this is what I’m used to”. Berating people for disliking change is definitely not the way forward, however. One must take their feelings into account if one wants to effect change. Another lesson learned from Peopleware: the problem is hardly ever technical.

Otherwise, I’m not sure. I’m constantly surprised by what other people find to be more or less readable when it comes to code. There are vocal opponents to operator overloading, for instance. I have no idea why.

What do you think? Where do you stand on auto?

Advertisements
Tagged , , , , , , , , ,

Boden, Flutter, React Native?

A while back I had an idea for a mobile app that I still might write. The main problems were finding the time and analysis paralysis: what do I write the project in?

I’ve done some Android development before using Qt, and it was a hassle. I’ve been told that JNI was made painful on purpose so that nobody would try to do it and I can see why. At least at the time the Android NDK had modern compilers compared to what my team was allowed to use for the rest of the codebase and it was my first taste of C++11 in production.

As much as I dislike the Apple ecosystem, it would be foolish to say the least to ignore iOS. Writing Java for the Android UI I can deal with (but wouldn’t be happy about), learning Objective C or Swift? That I’m not sure of. In any case, I don’t want to write UI code twice.

Since then, I’ve been meaning to look at Flutter, which seems to avoid most of the problems I would have to face otherwise. But that means learning Dart, which would be a significant sidetrack. Yesterday however, I learned on cppcast that there’s a C++ alternative called boden. On the one hand, I don’t have to learn a new language. On the other hand I have to write C++, with all the swordfights and seg faults that entails. Which leads me all the way back to analysis paralysis.

There’s also React Native and Xamarin, and probably a few other alternatives, but they all for reasons I can’t quite explain give me a bit of an allergic reaction.

I guess I’m going to have to bite the bullet and try one of these frameworks. Or all of them. Now to think of a tiny but not useless test project…

Build systems are stupid

Big complicated projects nearly always need a build system. They might be generating code. They might be compiling different branches depending on the platform or a build-time configuration option. They might turn on/off unity builds for C++.  For most simple projects however, build systems are stupid.

Let’s say you have a single C file. We need to tell the compiler the name of the file we want to compile, and the name of the resulting binary unless we’re happy with it being called a.out, which is unlikely:

gcc -Wall -Wextra -Werror -o foo foo.c

Maybe it’s the first commit in brand new repository and all you have is foo.c in there. Why am I telling the compiler what to build? What else would it build??

$ ls
foo.c
$ gcc
gcc: fatal error: no input files
compilation terminated.

“Real projects don’t look like that!”. Ok:

$ ls
build include src 
$ ls src
foo.c main.c
$ ls include
foo.h

I doubt any humans who have ever written C before would be confused as to how this project is supposed to be compiled. And yet, this is probably the simplest way I can come up with to tell the computer to do it properly:

CFLAGS ?= -Iinclude -Wall -Wextra -Werror -g

SRCS := $(shell find src -name *.c)
HDRS := $(shell find include -name *.h)
OBJS := $(subst src,build,$(subst .c,.o,$(SRCS)))

build/foo: $(OBJS)
	gcc -o $@ $^

build/%.o: src/%.c $(HDRS)
	gcc $(CFLAGS) -o $@ -c $<

Is there a better way than $(subst)? Probably. I don’t care. I was also too lazy to figure out dependencies so all source files depend on all headers. That’s a lot of lines of Makefile and all I wanted to specify were that source files are in src, headers are in include, binaries go inbuild, and that the compiler flags are -Wall -Wextra -Werror  -g. The first three are obvious from the existence and names of those directories, so there are 8 lines of code here to tell a computer what the compiler flags are. Robots: 1 Humans: 0.

CMake isn’t that much better:

cmake_minimum_required (VERSION 2.8.11)
project (foo)

file(GLOB SRC_FILES ${PROJECT_SOURCE_DIR}/src/*.c)
add_executable(foo ${SRC_FILES})
target_include_directories(foo PRIVATE ${PROJECT_SOURCE_DIR}/include)

Robots: 2 Humans: 0.

I’ll probably get told off for using file(GLOB) because what I really should be doing is manually listing source files because reasons. Why would I have files in src if I don’t want them to be compiled? In what universe does it make sense for a person whose job it is to tell computers what to do so that they don’t have to do it themselves to tell a build system which files to build? In my opinion every new language should default to doing what Pony does. Running the compiler just builds everything in the current directory tree.

I also think that new languages should ship with a compiler-based build system, i.e. the compiler is the build system. The reason for that is that as a developer, I want to rebuild the minimum possible amount of code that is required after my edits to run my tests. The CMake solution above fails at this: the compiler calculates dependencies and automatically feeds it into the build system, but those dependencies are too coarse.

If I touch any header, all source files that #include it will be recompiled. This happens even if it’s an additive change, which by definition can’t affect any existing source files.  If I change a function’s signature that is only used in foo.c, well tough: I’ll have to wait until bar.c, baz.c and quux.c all get recompiled for the temerity of depending on different functions from the same header.

This problem is bad enough in C and C++ but actually gets worse for languages with modules: now every file is a “header”. Changes to the implementation trigger recompilations, because our build systems are too stupid to know any better. I can’t see how a good build system can be an external tool unless the compiler is a library, and even then it would be doing the same work twice since dependency calculations are better done while a module/translation unit is being compiled.

Taking it a step further, the whole concept of caring about files and modules is probably outdated anyway. Why not a Smalltalk-like environment? You make changes to the source and your “image” gets rebuilt in the background, one part of the AST at a time, automagically resolving and rebuilding dependencies (at the function level!).

Computers and build systems: what have you done for me lately?

Tagged , , , , ,

My DConf2019 talk

I’ll be speaking at DConf this year. I haven’t yet decided how to structure the talk yet, but I’d like to take a page out of Dan Sak’s excellent CppCon 2016 talk that covered technical and psychological aspects of language adoption. In his case he was addressing a C++ audience on the difficulty of convincing C programmers to adopt C++ (an admirable goal!). In mine, it’s how do I out-C++ C++?

The genesis for my talk was my decision a few years ago to write tests for a legacy C codebase. I wanted to use D, but even as a fan of the language I chose to write the tests in C++ instead. And if even I wouldn’t have picked D for that task, who would?

I still think it was the right decision to make. The first step in”importing” the legacy project into a C++ test looks like this:

extern "C" {
    #include "legacy.h"
}

There is no second step. From here one #includes a test library such as catch2 and starts writing tests for the C functions in the API.

As far as I know, every language under the sun can interface with C. There’s usually (always?) some way to do FFI, and examples invariably show how to call a function that takes 2 integers, maybe even a const char* parameter. Unfortunately, that’s not what real codebases need to do. They need to call a function that takes a pointer to a structure that’s defined in a different header, which has 3 pointers to structures that are themselves defined in different headers, which…

You get the idea. Writing the definitions by hand error-prone, boring, and time-consuming. Even if correct, nobody seems to talk about the elephant in the C API room: macros. Not once have I encountered a non-trivial C API that doesn’t require the C preprocessor to use properly. C++ users are at an advantage here: just use the macros. Everyone else has to get by with ad-hoc solutions. And if the function-like macro implementations change, the code will break.

I think that the only way to make it so that C++ isn’t the obvious and/or only choice is that a challenger allows one to write `#include “legacy.h”` and have it work just as easily as it does natively. That’s why I started the dpp project, and I’m looking forward to talking about the technical challenges I’ve encountered on my journey so far.

See you at DConf!

Tagged , , , , , , ,

When you can’t (and shouldn’t) unit test

I’m a unit test aficionado, and, as such, have attempted to unit test what really shouldn’t be. It’s common to get excited by a new hammer and then seeing nails everywhere, and unit testing can get out of hand (cough! mocks! cough!).

I still believe that the best tests are free from side-effects, deterministic and fast. What’s important to me isn’t whether or not this fits someone’s definition of what a unit test is, but that these attributes enable the absence of slow and/or flaky tests. There is however another class of tests that are the bane of my existence: brittle tests. These are the ones that break when you change the production code despite your app/library still working as intended. Sometimes, insisting on unit tests means they break for no reason.

Let’s say we’re writing a new build system. Let’s also say that said build system works like CMake does and spits out build files for other build systems such as ninja or make. Our unit test fan comes along and writes a test like this:

assert make_output == "all: foo\nfoo: foo.c\n\tgcc -o foo foo.c"

I believe this to be a bad test, and the reason why is that it’s checking the implementation instead of the behaviour of the production code. Consider what happens when the implementation is changed without affecting behaviour:

all: foo\nfoo: foo.c\n\tgcc -o $@ $<

The behaviour is the same as before: any time `foo.c` is changed, `foo` will get recompiled. The implementation not only isn’t the same, it’s arguably better now, and yet the assertion in the test would fail. I think we can all agree that the ROI for this test is negative if this is all it takes to break it.

The orthodox unit test approach to situations like these is to mock the service in question, except most people don’t get the memo that you should only mock code you own. We don’t control GNU make, so we shouldn’t be doing that. It’s impossible to copy make exactly in a mock/stub/etc. and it’s foolish to even try. We (mostly) don’t care about the string that our code outputs, we care that make interprets that string with the correct semantics.

My conclusion is that I shouldn’t even try to write unit tests for code like this. Integration tests exist for a reason.

Tagged , , , , ,

Issues DIP1000 can’t (yet) catch

D’s DIP1000 is its attempt to avoid memory safety issues in the language. It introduces the notion of scoped pointers and in principle (modulo bugs) prevents one from writing memory unsafe code.

I’ve been using it wherever I can since it was implemented, and it’s what makes it possible for me to have written a library allowing for fearless concurrency. I’ve also written a D version of C++’s std::vector that leverages it, which I thought was safe but turns out to have had a gaping hole.

I did wonder if Rust’s more complicated system would have advantages over DIP1000, and now it seems that the answer to that is yes. As the the linked github issue above shows,  there are ways of creating dangling pointers in the same stack frame:

void main() @safe {
    import automem.vector;

    auto vec1 = vector(1, 2, 3);
    int[] slice1 = vec1[];
    vec1.reserve(4096);  // invalidates slice1 here
}

Fortunately this is caught by ASAN if slice1 is used later. I’ve worked around the issue for now by returning a custom range object from the vector that has a pointer back to the container – that way it’s impossible to “invalidate iterators” in C++-speak. It probably costs more in performance though due to chasing an extra pointer. I haven’t measured to confirm the practical implications.

In Rust, the problem just can’t occur. This fails to compile (as it should):

use vec;

fn main() {
    let mut v = vec![1, 2, 3, 4, 5];
    let s = v.as_slice();
    println!("org v: {:?}\norg s: {:?}", v, s);
    v.resize(15, 42);
    println!("res: v: {:?}\norg s: {:?}", v, s);
}

I wonder if D can learn/borrow/steal an idea or two about only having one mutable borrow at a time. Food for thought.

Tagged , , , ,

The joys of translating C++’s std::function to D

I wrote a program to translate C headers to D. Translating C was actually more challenging than I thought; I even got to learn things I didn’t know about the language even though I’ve known it for 24 years. The problems that I encountered were all minor though, and to the extent of my knowledge have all been resolved (modulo bugs).

C++ is a much larger language, so the effort should be considerably more. I didn’t expect it to be as hard as it’s been however, and in this blog I want to talk about how “interesting” it was to translate C++11’s std::function by hand.

The first issue for most languages would be that it relies on template specialisation:

template<typename>
class function;  // doesn't have a definition anywhere

template<typename R, typename... Args>
class function<R(Args...)> { /* ... */ }

This is a way of constraining the std::function template to only accept function types. Perhaps surprisingly to some, the C++ syntax for the type of a function that takes two ints and returns a double is double(int, int). I doubt most people see this outside of C++ standard library templates. If it’s still confusing, think of double(int, int) as the type that is obtained by deferencing a pointer of type double(*)(int, int).

D is, as far as I know, the only other language other than C++ to support partial template specialisation. There are however two immediate problems:

  • function is a keyword in D
  • There is no D syntax for a function type

I can mitigate the name issue by calling the symbol function_ instead; however, this will affect name mangling, meaning nothing will actually link. D does have pragma(mangle) to tell the compiler how to mangle symbols, but std::function is a template; it doesn’t have any mangling until it’s instantiated. Let’s worry about that later and call the template function_ for now.

The second issue can be worked around:

// C++: `using funPtr = double(*)(int, int);`
alias funPtr = double function(int, int);
// C++: `using funType = double(int, int);`
alias funType = typeof(*funPtr.init);

As in C++, the function type is the type one gets from deferencing a function pointer. Unlike C++, currently there’s no syntax to write it directly. First attempt:

// helper to automate getting an alias to a function type
template FunctionType(R, Args...) {
    alias ptr = R function(Args);
    alias FunctionType = typeof(*ptr.init);
}

struct function_(T);
struct function_(T: FunctionType!(R, Args), R, Args...) { }

This doesn’t work, probably due to this bug preventing the helper template FunctionType from working as intended. Let’s forget the template constraint:

extern(C++, "std") {
    struct function_(T) {
        import std.traits: ReturnType, Parameters;
        alias R = ReturnType!T;
        alias Args = Parameters!T;
        // In C++: `R operator()(Args) const`;
        R opCall(Args) const;
    }
}

void main() {
    alias funPtr = double function(double);
    alias funType = typeof(*funPtr.init);
    function_!funType f;
    double result = f(3.3);
}

This compiles but it doesn’t link: there’s an undefined reference to std::function_::operator()(double) const. Looking at the symbols in the object files using nm, we see that g++ emitted _ZNKSt8functionIFddEEclEd but dmd is trying to link to _ZNKSt9function_IFddEEclEd. As expected, name mangling issues related to renaming the symbol.

We could manually add a pragma(mangle) to tell D how to mangle the operator for the double(double) template instantiation, but that solution doesn’t scale. CTFE (constexpr if you speak C++ but not D) to the rescue!

// snip - as before
pragma(mangle, opCall.mangleof.fixMangling)
R opCall(Args) const;

// (elsewhere at file scope)
string fixMangling(string str) {
    import std.array: replace;
    return str.replace("9function_", "8function");
}

What’s going on here is an abuse of D’s compile-time power. The .mangleof property is a compile-time string that tells us how a symbol is going to be mangled. We pass this string to the fixMangling function which is evaluated at compile-time and fed back to the compiler telling it what symbol name to actually use. Notice that function_ is still a template, meaning .mangleof has a different value for each instantiation. It’s… almost magical. Hacky, but magical.

The final code compiles and links. Actually creating a valid std::function<double(double)> from D code is left as an exercise to the reader.

 

Tagged , , , , , , ,

Comparing Pythagorean triples in C++, D, and Rust

EDIT: As pointed out in the Rust reddit thread, the Rust version can be modified to run faster due to a suble difference between ..=z and ..(z+1). I’ve updated the measurements with rustc0 being the former and rustc1 being the latter. I’ve also had to change some of the conclusions.

You may have recently encountered and/or read this blog post criticising a possible C++20 implementation of Pythagorean triples using ranges. In it the author benchmarks different implemetations of the problem, comparing readability, compile times, run times and binary sizes. My main language these days is D, and given that D also has ranges (and right now, as opposed to a future version of the language), I almost immediately reached for my keyboard. By that time there were already some D and Rust versions floating about as a result of the reddit thread, so fortunately for lazy me “all” I had to next was to benchmark the lot of them. All the code for what I’m about to talk about can be found on github.

As fancy/readable/extensible as ranges/coroutines/etc. may be, let’s get a baseline by comparing the simplest possible implementation using for loops and printf. I copied the “simplest.cpp” example from the blog post then translated it to D and Rust. To make sure I didn’t make any mistakes in the manual translation, I compared the output to the canonical C++ implementation (output.txt in the github repo). It’s easy to run fast if the output is wrong, after all. For those not familiar with D, dmd is the reference compiler (compiles fast, generates sub-optimal code compared to modern backends) while ldc is the LLVM based D compiler (same frontend, uses LLVM for the backend). Using ldc also makes for a more fair comparison with clang and rustc due to all three using the same backend (albeit possibly different versions of it).

All times reported will be in milliseconds as reported by running commands with time on my Arch Linux Dell XPS15. Compile times were measured by excluding the linker, the commands being clang++ -c src.cpp, dmd -c src.d, ldc -c src.d, rustc --emit=obj src.rs for each compiler (obviously for unoptimised builds). The original number of iterations was 100, but the runtimes were so short as to be practically useless for comparisons, so I increased that to 1000. The times presented are a result of running each command 10 times and taking the mininum, except for the clang unoptmised build of C++ ranges because, as a wise lady on the internet once said, ain’t nobody got time for that (you’ll see why soon). Optimised builds were done with -O2 for clang and ldc,  -O -inline for dmd and -C opt-level=2 for rustc. The compiler versions were clang 7.0.1, dmd 2.083.1, ldc 1.13.0 and rustc 1.31.1. The results for the simple, readable, non-extensible version of the problem (simple.cpp, simple.d, simple.rs in the repo):

Simple          CT (ms)  RT (ms)

clang debug      59       599
clang release    69       154
dmd debug        11       369
dmd release      11       153
ldc debug        31       599
ldc release      38       153
rustc0 debug    100      8445
rustc0 release  103       349
rustc1 debug             6650
rustc1 release            217

C++ run times are the same as D when optimised, with compile times being a lot shorter for D. Rust stands out here for both being extremely slow without optimisations, compiling even slower than C++, and generating code that takes around 50% longer to run even with optimisations turned on! I didn’t expect that at all.

The simple implementation couples the generation of the triples with what’s meant to be done with them. One option not discussed in the original blog to solve that is to pass in a lambda or other callable to the code instead of hardcoding the printf. To me this is the simplest way to solve this, but according to one redditor there are composability issues that may arise. This might or might not be important depending on the application though. One can also compose functions in a pipeline and pass that in, so I’m not sure what the problem is. In any case, I wrote 3 implementations and timed them (lambda.cpp, lambda.d, lambda.rs):

Lambda          CT (ms)  RT (ms)

clang debug      188       597
clang release    203       154
dmd debug         33       368
dmd release       37       154
ldc debug         59       580
ldc release       79       154
rustc0 debug     111      9252
rustc0 release   134       352
rustc1 debug              6811
rustc1 release             154

The first thing to notice is, except for Rust (which was slow anyway), compile times have risen by about a factor of 3: there’s a cost to being generic. Run times seem unaffected except that the unoptimised Rust build got slightly slower. I’m glad that my intuition seems to have been right on how to extend the original example: keep the for loops, pass a lambda, profit. Sure, the compile-times went up but in the context of a larger project this probably wouldn’t matter that much. Even for C++, 200ms is… ok. Performance-wise, it’s now a 3-way tie between the languages, and no runtime penalty compared to the non-generic version. Nice!

Next up, the code that motivated all of this: ranges. I didn’t write any of it: the D version was from this forum post, the C++ code is in the blog (using the “available now” ranges v3 library), and the Rust code was on reddit. Results:

Range           CT (ms)   RT (ms)

clang debug     4198     126230
clang release   4436        294
dmd debug         90      12734
dmd release      106       7755
ldc debug        158      15579
ldc release      324       1045
rustc0 debug     128      11018
rustc0 release   180        422
rustc1 debug               8469
rustc1 release              168

I have to hand it to rustc – whatever you throw at it the compile times seem to be flat. After modifying the code as mentioned in the edit at the beginning, it’s now the fastest out of the 3!

dmd compile times are still very fast, but the generated code isn’t. ldc does better at optimising, but the runtimes are pretty bad for both of them. I wonder what changes would have to be made to the frontend to generate the same code as for loops.

C++? I only ran the debug version once. I’m just not going to wait for that long, and besides, whatever systematic errors are in the measurement process don’t really matter when it takes over 2 minutes. Compile times are abysmal, the only solace being that optimising the code only takes 5% longer time than to not bother at all.

In my opinion, none of the versions are readable. Rust at least manages to be nearly as fast as the previous two versions, with C++ taking twice as long as it used to for the same task. D didn’t fare well at all where performance is concerned. It’s possible to get the ldc optimised version down to ~700ms by eliminating bounds checking, but even then it wouldn’t seem to be worth the bother.

My conclusion? Don’t bother with the range versions. Just… don’t. Nigh unreadable code that compiles and runs slowly.

Finally, I tried the D generator version on reddit. There’s a Rust version on Hacker News as well, but it involves using rust nightly and enabling features, and I can’t be bothered figuring out how to do any of that. If any Rustacean wants to submit something that works with a build script, please open a PR on the repo.

Generator      CT (ms)  RT (ms)

dmd debug      208      381
dmd release    222      220
ldc debug      261      603
ldc release    335      224

Compile times aren’t great (the worst so far for D), but the runtimes are quite acceptable. I had a sneaky suspicion that the runtimes here are slower due to startup code for std.concurrency so I increased N to 5000 and compared this version to the simplest possible one at the top of this post:

N=5k                    RT (ms)
dmd relase simple       8819
dmd release generator   8875

As expected, the difference vanished.

Final conclusions: for me,  passing a lambda is the way to go, or generators if you have them (C++ doesn’t have coroutines yet and the Rust version mentioned above needs features that are apparently only available in the nightly builds). I like ranges in both their C++ and D incarnations, but sometimes they just aren’t worth it. Lambdas/generators FTW!

Tagged , , , , ,

Improvements I’d like to see in D

D, as any language that stands on the shoulder of giants, was conceived to not repeat the errors of the past, and I think it’s done an admirable job at that. However, and perfectly predictably, it made a few of its own. Sometimes, similar to the ones it avoided! In my opinion, some of them are:

No UFCS chain for templates.

UFCS is a great feature and was under discussion to be added to C++, but as of the C++17 standard it hasn’t yet. It’s syntatic sugar for treating a free function as a member function if the first parameter is of that type, i.e. allows one to write obj.func(3) instead of func(obj, 3). Why is that important? Algorithm chains. Consider:

range.map!fun.filter!gun.join

Instead of:

join(filter!gun(map!fun(range)));

It’s much more readable and flows more naturally. This is clearly a win, and yet I’m talking about it in a blog post about D’s mistakes. Why? Because templates have no such thing. There are compile-time “type-level” versions of both map and filter in D’s standard library called staticMap and Filter (the names could be more consistent). But they don’t chain:

alias memberNames = AliasSeq!(__traits(allMembers, T));
alias Member(string name) = Alias!(__traits(getMember, T, name));
alias members = staticMap!(Member, memberNames);
alias memberFunctions = Filter!(isSomeFunction, members);

One has to name all of these intermediate steps even though none of them deserve to be named, just to avoid writing a russian doll parenthesis unreadable nightmare. Imagine if instead it were:

alias memberFunctions = __traits(allMembers, T)
    .staticMap!Member
    .Filter!(isSomeFunction);

One can dream. Which leads me to:

No template lambdas.

In the hypothetical template UFCS chain above, I wrote staticMap!Member, where the Member definition is as in the example before it. However: why do I need to name it either? In “regular” code we can use lambdas to avoid naming functions. Why can’t I do the same thing for templates?

alias memberFunctions = __traits(allMembers, T)
    .staticMap!(name => Alias!(__traits(getMember, T, name)))
    .Filter!isSomeFunction

Eponymous templates

Bear with me: I think the idea behind eponymous templates is great, is just that the execution isn’t, and I’ll explain why by comparing it to something D got right: constructors and destructors. In C++ and Java, these special member functions take the name of the class, which makes refactoring quite a bit annoying. D did away with that by naming them this and ~this, which makes the class name irrelevant. The way to do eponymous templates right is to (obviously renaming the feature) follow D’s own lead here and use either this or This to refer to itself. I’ve lost count of how many times I’ve introduced a bug due to template renaming that was hard to find.

@property getters

What are these for given the optional parentheses for functions with no parameters?

inout

Template this can accomplish the same thing and is more useful anyway.

Returning a reference

It’s practically pointless. Variables can’t be ref, so assigning it to a function that returns ref copies the value (which is probably not what the user expected), so one might as well return a pointer. The exception would be UFCS chains, but before DIP1016 is accepted and implemented, that doesn’t work either.

Wrong defaults

I think there’s a general consensus that @safe, pure and immutable should be default. Which leads me to:

Attribute soup

I’ve lost count now of how many times I’ve had to write @safe @nogc pure nothrow const scope return. Really.

Tagged ,

Unit Testing? Do As I Say, Don’t Do As I Do

I’m a firm believer in unit testing. I’ve done more tech talks on the subject than I’d care to count, and always tell audiences the same thing: prefer unit tests, here’s a picture of the testing pyramid, keep unit tests pure (no side-effects), avoid end-to-end tests (they’re flaky, people will stop paying attention to red builds since all builds will be red). I tell them about adapters, ports and hexagonal architecture. But when it comes to using libclang to parse and translate C and C++ headers, I end up punting and writing a lot of integration tests instead. Hmm.

I know why people write tests with side-effects, and why they end up writing integration and end-to-end ones instead of the nice pure unit test happy place I advocate. It’s easier. There’s less thinking involved. A lot less. However, taking the easy path has always come back to bite me. Those kinds of tests take longer. They higher up the test pyramid you go, the flakier they get. TCP ports stay open longer than a tester would like, for instance. The network goes down. All sorts of things.

I understand why I wrote integration tests instead of unit tests when interfacing with libclang too. Like it is for everyone else, it was just easier. I failed to come up with a plan to unit test what I was doing. It didn’t help that I’d never used libclang and had no idea what the API looked like or what it allowed me to do. It also doesn’t help that libclang doesn’t have an option to take a string to the code to parse and instead takes a file name, but I can work around that.

Because of this, the dpp codebase currently suffers from that lack of separation of concerns. Code that translates C/C++ to D is now intimately tied to libclang and its quirks. If I ever try to use something other than libclang, I won’t be able to. All of the bad things I caution everybody else about? I guaranteed they happened in one of my newest projects.

Before the code collapses under its own complexity, I’ve decided to do what I should’ve done all along and am rewriting dpp so it uses layers to get away from the libclang mess. I’m still figuring it all out, but the main idea is to have a transformation layer between libclang and my code that takes its data types and converts them to a new set of AST types that are my own. From then on it should be trivial to unit test the translation of those AST types that represent C or C++ code into D. Funnily enough, the fact that I wrote so many integration tests will keep me honest since all of those old tests will still have to pass. I’m not sure how I feel about that.

I might do another blog post covering how I ended up porting a codebase with pretty much only integration tests to the unit variety. It might be of interest to anyone maintaining a legacy codebase (i.e. all of us).

Tagged , , , , , ,