Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,13 @@ date) and use that section as the basis for the GitHub release notes.
by default, matching the other learners. (#585)

### Fixed
- `NCESTrainer.train()` created a fresh `DataLoader(num_workers=...)` worker
pool per architecture in a loop; the existing `os.cpu_count()`-based cap on
`num_workers` only engaged when `cpu_count <= num_workers`, so it silently
did nothing on machines with more cores than the default of 8, and repeated
worker-pool creation at that count could deadlock under `coverage run`
(`NCES`/`NCES2`/`ROCES` training, e.g. `test_nces_trainer.py`). Now capped
unconditionally to `min(requested, cpu_count-1, 4)`. (#610)
- `ConceptAbstractSyntaxTreeBuilder._fix_mid_tokens_errors`/`_postprocess_tail_fix`/
`_enforce` repaired malformed token sequences via `random.choice(...)`, making
parser output non-deterministic across runs on identical input; replaced with
Expand Down
8 changes: 6 additions & 2 deletions ontolearn/learners/nces.py
Original file line number Diff line number Diff line change
Expand Up @@ -431,8 +431,12 @@ def train(self, data: Iterable[List[Tuple]]=None, epochs=50, batch_size=64, max_
refinement_expressivity=0.2, refs_sample_size=50, learning_rate=1e-4, tmax=20, eta_min=1e-5,
clip_value=5.0, num_workers=8, save_model=True, storage_path=None, optimizer='Adam', record_runtime=True,
example_sizes=None, shuffle_examples=False):
if os.cpu_count() <= num_workers:
num_workers = max(0,os.cpu_count()-1)
# Cap unconditionally: on machines with many cores this previously stayed
# uncapped at the requested value, and repeatedly spawning that many
# DataLoader worker processes (one fresh pool per architecture in
# NCESTrainer.train()) can deadlock under `coverage run` regardless of
# how many cores are available (see issue #610).
num_workers = max(0, min(num_workers, (os.cpu_count() or 1) - 1, 4))
if storage_path is None:
currentDateAndTime = datetime.now()
storage_path = f'NCES-Experiment-{currentDateAndTime.strftime("%H-%M-%S")}'
Expand Down
6 changes: 5 additions & 1 deletion ontolearn/nces_trainer.py
Original file line number Diff line number Diff line change
Expand Up @@ -60,7 +60,11 @@ def __init__(self, synthesizer, epochs=300, batch_size=128, learning_rate=1e-4,
self.tmax = tmax
self.eta_min = eta_min
self.clip_value = clip_value
self.num_workers = num_workers
# Cap unconditionally, not just on low-core machines: train() spawns a
# fresh DataLoader worker pool per architecture in a loop, and doing so
# repeatedly with a large num_workers can deadlock under `coverage run`
# regardless of how many cores are available (see issue #610).
self.num_workers = max(0, min(num_workers, (os.cpu_count() or 1) - 1, 4))
self.storage_path = storage_path

@staticmethod
Expand Down
Loading