RYDE-WORK/autoresearch
1
0
Fork 0
mirror of https://github.com/RYDE-WORK/autoresearch.git synced 2026-07-22 00:27:42 +08:00
Code Issues Packages Projects Releases Wiki Activity

fix NaN loss not caught by fast-fail check

Browse Source 
`train_loss_f > 100` silently passes on NaN because IEEE 754 NaN
comparisons always return False. When an agent experiment produces
NaN (e.g. from an aggressive LR change), the run wastes the full
5-minute budget instead of failing fast.

`not (x <= 100)` catches both >100 and NaN with no added complexity.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
haosenwang1018 2026-03-09 23:51:02 +08:00
parent 068d93da75
commit b5ba8ac00d
1 changed files with 2 additions and 2 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
train.py
View File

@ -565,8 +565,8 @@ while True:
train_loss_f = train_loss.item()
# Fast fail: abort if loss is exploding
if train_loss_f > 100:
# Fast fail: abort if loss is exploding or NaN
if not train_loss_f <= 100:
print("FAIL")
exit(1)