____ _
| __ )| | ___ __ _
| _ \| |/ _ \ / _` |
| |_) | | (_) | (_| |
|____/|_|\___/ \__, |
|___/
_ _
_/_| |__ ___ _ __| |_ _ _/_
/ _ \ '_ \ / _ \ '__| | | | |/ _ \
__/ |_) | __/ | | | |_| | __/
\___|_.__/ \___|_| |_|\__,_|\___|
< Previous post
|
Next post >
[2026-04-30][Off-topic]
The paper have been refused from EUROPAR2026. Here I summarize the different reasons or the
reviewers. The paper can be downloaded here.
I think the biggest problem of the article right now, is being caught between two stools: It
fails at providing clear details on DAHL's implementation, while also failing at showing interesting
results. Experiments being CPU only is obviously a weakness, which is not working in our favor.
Comparing DAHL directly to PyTorch is somewhat wobbly, thus it has been criticized by most of
reviewers. If we want to defend "heterogeneity adaptiveness", the comparison should be done on how
well the two frameworks adapts to different machines/dataset size.
- Tackling energy consumption in Machine Learning is a timely problem
- Results shows significant pontential
- Clear and easy-to-follow structure
- Comparison with SOTA framework
- Well designed experimental settings
- Comparing training with different images sizes is very relevant
- Open Source framework
- Need and motivation of the work is well explained
- Solid experimental methodology: multiple datasets of varying sizes, different configurations,
software-based and physical energy measurements.
- Workload impact on performance is well supported
- Good connections between machine learning and HPC concepts.
I refer to speech issues as missing context elements and disclaimers to orient reader's
expectations.
- Contribution is more linked to the implementation rather than proposing new concepts
- Scope for solving energy in AI is not clearly stated
- Most of energy consumption is due to LLMs and generative AI
- In the same manner, we know that inference is also a bigger contributor to energy consumption
- We should clearly state our position while aknowledging fixing a "sub-problem" in AI
sustainability
- We can also state that such results could be clearly generalized on other types of model, but
that our PoC focuses only on CNNs
- Need a limit section
- How DAHL falls short compared to PyTorch?
- Why comparing these too is biased? Clearly not the same boilerplate
- Explicitly write what we referred as "obvious limitations"
- Don't pretend to replace PyTorch
- Introduce DAHL more as a proof-of-concept/prototype
- Discussing limits and scope honestly in the introduction would set right expectations for the
readers
- Lacking connections in the related works section: How does our work differs from them?
- Accuracy is not tackled well enough
- Reported accuracy is very low for some datasets, implying that results might not apply to
real-world use cases.
- It was not clear for readers that accuracy don't matter as we are performing same computations
anyways. Only scheduling changes.
Details concerning DAHL are not provided making the approach confusing:
- It is hard to appreciate the leveraged solutions to reduce consumption
- What the framework does is hard to understand
- Try restructuring from high-level overview, to more technical details.
- How batch parallelism works on DAHL? Figure 2. is missing introductory explanations
- Evaluation only on one CPU
- No GPU even though it's consensually the goto in the whole industry and SOTA
- Hard to compare benefits as it isn't realistic to run on CPU only, what would be the consumption
for a equivalent training on a GPU?
- Does limiting PU (here CPU cores) could be done in a same manner on GPU to limit consumption?
- No heterogeneous cluster
- No distribution
- Comparison of DAHL vs PyTorch is flawed/biased
- Could PyTorch be better tuned to achieve similar results?
- Could it use better backends?
- "DAHL seems tailored for the evaluated workload, biasing the results", this remark is partially
true. While fashion-mnist and cifar-10 are particularly bad on DAHL, I did chose big-fashion
because it worked better on it... We clearly need to address this bias.
- Results
- Runtime != energy is trivial in the SOTA, this shouldn't be the sole result. This point is
sketchy, for some rewievers it seems obvious, for others it's important highlighting.
- It should be clearer that the whole machine components are measured, and that while the machine
has a GPU, it is turned off and don't represent consumption.
- Discussions of results unclear
- Statement "DAHL average CPU usage and energy consumption are always lower than or equal to
those of Pytorch solutions." is simply false, I should rephrase this
- Why is Runtime/Energy higher on small image sizes? We explained it is because of task
granularity, but apparently it was not clear.
< Previous post
|
Next post >