Virtualised Build & Test Environments for Embedded Software
The Problem
I'm currently employed as an embedded software engineer, which is neat. There are all kinds of interesting challenges involved in embedded, but also several not-so-neat challenges! At work we often joke about embedded software being for masochists. We say this because in general software there are a great number of broadly-solved (or largely minimised) problems that, for one reason or another, seem to still exist in embedded software.
The particular problem I wanted to muse on today is consistent tooling.
Our team uses a wide variety of different hardware platforms both for different projects and, sometimes, within the same project. As a consequence we use many different tool chains and development environments with varying capabilities.
This tends to mean developers start getting familiar with one tool set when suddenly the project is over and everything changes, starting at square 1. Other consequences are developers struggling to recreate successful builds performed by their colleagues of the same code base, or programmers that fear replacing an aged development machine because of the very real possibility that they will never recreate their working build environment.
This tends to mean developers start getting familiar with one tool set when suddenly the project is over and everything changes, starting at square 1. Other consequences are developers struggling to recreate successful builds performed by their colleagues of the same code base, or programmers that fear replacing an aged development machine because of the very real possibility that they will never recreate their working build environment.
You can see in the broader software world that solutions to this are becoming increasingly commonplace. Things like Vagrant, which allows you to easily create virtual machines to act as your build environments repeatably, and docker which allows you to easily create well defined deployment environments for your applications.
From where I stand, however, I cannot see any equivalent attempts to improve life in the embedded world!
The Idea?
Recently, while listening to colleagues wrestling to recreate their build environment on another developers machine, I started thinking about ways of solving these problems.
It would be excellent to simply virtualise the build environment using something like Vagrant, bake our tool chain into a virtual machine image and voila. The problem is that with embedded software, building a binary is only half the battle. You now must contend with both flashing the binary onto some hardware, as well as debugging the program by viewing registers and setting breakpoints, etc. Typically this is done via JTAG interfaces.
Wouldn't it be nice, I thought, if the Vagrant image could also be configured to act as a JTAG "tunnel", connecting to your hardware to perform flashing & debugging operations on behalf of an IDE which understood the tunnel's API?
This way a single IDE could be understood and invested in, whilst allowing you to build and debug on any platform for which a Vagrant machine could exist that meets the above requirements!
It would be excellent to simply virtualise the build environment using something like Vagrant, bake our tool chain into a virtual machine image and voila. The problem is that with embedded software, building a binary is only half the battle. You now must contend with both flashing the binary onto some hardware, as well as debugging the program by viewing registers and setting breakpoints, etc. Typically this is done via JTAG interfaces.
Wouldn't it be nice, I thought, if the Vagrant image could also be configured to act as a JTAG "tunnel", connecting to your hardware to perform flashing & debugging operations on behalf of an IDE which understood the tunnel's API?
This way a single IDE could be understood and invested in, whilst allowing you to build and debug on any platform for which a Vagrant machine could exist that meets the above requirements!
Surely someone has done this?
From what I can gather, some IDEs support invoking builds on remote machines (usually via SSH) as a way of quickly implementing cross-platform compilation. So you could see this being a way of getting an IDE to drive a compiler in a Vagrant image for you.
When it comes to debugging, there is the OpenOCD project which implements the gdbserver protocol. It is easy to imagine setting up OpenOCD on a Vagrant machine such that it awaits remote GDB instructions from your IDE.
I haven't looked, but I would be surprised if many (any?) embedded IDEs supported the remote build via SSH functionality, or remote GDB debugging. I'd be even more surprised if they could perform remote GDB debugging and usefully allow for register & memory displays and editing!
So, in conclusion:
- Is there a better way of achieving the same result? Probably.
- Would this actually work? Maybe?
- Has anyone tried? I don't think so?
I'd be interested to know what solutions exist to this problem, particularly in embedded software, if anyone knows! My favourite outcome is that somebody replies saying "You idiot! Just use XYZ!" and my problems are solved!