Okay, so this is getting really long, I'll put the ask up front: I have a strategy, I think it is reasonable. Now is a point where I can easily change things, and it won't be so easily later. So I'm looking to see if anyone has trod this road before and can recommend any of:

1. a different build system that will be easier to manage for this use case
2. a different strategy for using cmake that will be easier to manage
3. any gotchas I should be aware of, even if you don't have better solutions.

Background

I have a project I'm working on where the ultimate deliverable will be a hardware device with 3-4 different microcontrollers coordinating with each other and interacting with a PC-ish platform. This is a clean rewrite of a C++ codebase. Due to the microcontroller (and some of the PC APIs) being C++, the language of choice for most of it is likely to remain C/C++.

I'm succeeded in setting up a build system for embedded code. The old code was arduino, so it relies a lot on those libraries, but I've managed to set up enough custom cmake to get off of the ardunio tools altogether, even if I am borrowing their libraries and some of the "smarts" built into the system about setting build flags, etc. So far, I have a dockerized toolchain (cmake + make + gcc-arm-none-eabi) that can successfully build ARM binaries for the target platform.

The thing that I'm up against now is that I'd like to have a robust off-target unit testing infrastructure. My ideal case is that everything in the embedded system will be broken down into libraries that have clear interfaces, then to use unit tests with mocks to get high coverage of test cases. I'll still need some HIL tests, but because those are harder to set up and run, I want to use those for integration and validation.

In terms of OSes available, we're mostly working on Windows systems using WSL for linux. I'd like things to be as linux-based as possible to support CI on github, etc.

Goals and Cmake limitations

I started out using cmake because I hate it least of the tools I've used, and I am at least pretty far up the learning curve with it. But a limitation I'm hitting is that you can't do a mixed compile with two different toolchains in one build. The reasons why cmake has this limitation seem reasonable to me, even if it is annoying. You can easily change the toolchain that your code is built with, but that seems to be largely targeted at cross-compiling the same binaries for different systems. What I want to do is:

• build my code libraries with embedded settings for linking to the embedded binaries and build those embedded binaries (the end product)
• build my code libraries with linux-ish tools and link them against unit tests to have a nice CI test process
• (eventually) also be able to build windows binaries for the PC components -- when I get to that point, I'd like to get away from the MSVC compilers, but will use them if I have to

Current strategy

My current plan is to configure a library build like this (pseudocode):

add_library(mylib sources)
if (BUILD_TYPE STREQUAL BUILD_TYPE_EMBEDDED)
<embedded config>
elseif (BUILD_TYPE STREQUAL BUILD_TYPE_LINUX)
<linux config, if any>
endif()

#unit tests are built for each library
if (BUILD_TYPE STREQUAL BUILD_TYPE_LINUX)
add_executable(mylib_test sources test_sources)
target_link_libraries(mylib gtest etc.)
endif()

For the rollup binaries, I make the whole target conditional

if (BUILD_TYPE STREQUAL BUILD_TYPE_EMBEDDED)
add_executable(myembedap sources)
target_link_libraries(mylib)
endif()

Then the build script (outside cmake) is something like

cd build/embedded
cmake <path to src> <set embedded toolchain> -DBUILD_TYPE=embedded
make
cd ../../build/linux
cmake <path to src> -DBUILD_TYPE=linux
make

Things I like about this strategy:

• It's relatively simple to do all the builds or just one of the builds (that control would go in the shell script)
• I have one source tree for the whole build
• It lets configuration be near code
• It lets tests be near code.
• I think it's extensible to cover the PC component builds in the future

Things that worry me:

• It feels like a hack
• Support for off-target tests feels like it should be solved problem and I'm worried I'm missing something

Thanks for reading. If you made it this far, you have my gratitude. Here's a video with funny out of office messages that I enjoyed.