It seems that some change came in mid-June that has reduced the
max device count we can get with this job by aprox 50%

The claim is strictly coming from looking at the graphs we get in jenkins:

magnesium

aluminium

The problem could be anywhere, including some change in the basic infra
environment. One clue is that both jobs seemed to have dropped in the same
time frame.