Thanks for sharing your thoughts! I may have been a bit dramatic when talking about a new layer, I just observe that we constantly see/create new tools to simplifying current processes with new abstractions (which require new knowledges) to perform operations we are already doing, but without good practices.

I wonder if the issue is not more with the way we work, instead of the tools we use (tools have an impact on the performance, but the problem might be more at the culture / practices level most of the time).

We got good usability by shifting things towards the development environment after adding complexity remotely then and running the abstractions more "naked" on bare metal.

The complexity also feels reduced because the remote tax is not to be paid and speeds are much faster.

This is ca. over the last 5-6 years for something similar to how you describe Dagger, some were voicing the tendency towards the local build earlier and providing the tooling.

For the cultural / practice level we consider fast feedback cycles und "single click" solutions one of the best guidelines. The single click must not be taken word-by-word, e.g. it's fine to integrate a first fast build (or fast feedback build or fast forgiving build; FFB) on the staging area after the user has submitted the commit message. If the build fails, the message is tagged in the subject by the step and the exit status (e.g. [make: 2] original subject ) and the log is inserted into the message.

As local builds have the sheer performance, increments, shares and caches (100 ms up to ~1 sec are the norm); and can be forgiving (commit and push is not blocked), the fastest feedback is available showing the amend message. Amending the commit re-runs and replaces or removes the build log and voids the exit status tagging.

It is often already transparently compatible with IDE integrations that can also run "their checks" on commit and reflect the (pushed) commit message in a popup, (again) providing fast feedback showing at least the subject beginning.

In general the same build manager should be in use on the remote and all the procedures so that they are easy to create, run and maintain from the local environment which we see leading/parenting. Running them only remote robs the most of the remote builds other benefits (depends on project, if the development environment is "lost", there is only win in the remote build).

This is also why we consider the guideline to look for fast feedback as a good one, it is a process in the maintenance lifecycle.

@GuyFalourd You're probably right about the about the culture/practices part. I think that sometimes, the "oh, shiny new stuff"-factor of new tools might be a heavy contributor to that. While I think it's very important in our profession as software engineers to keep an open mind and see what the recent developments are, often changing for the newest hype tools is probably detrimental to progress. Most of the times, the learning curve for the new abstractions you've mentioned is steeper than anticipated. Thus, most of the times, it pays off to master your current tools rather than jump to the next one. Saves you overhead required for switching to different tools and learning them.

I've probably been guilty of jumping to shiny new stuff more often than I'd like to admit...