When I ran a synthesizer with a PATE-CTGAN model over a large database (not too large tbh, a little bit less than 100MB) the fit will fail and the kernel will crash.
I tried to debug the problem and figured out the model doesn't even start the fitting process but gets stuck at the _get_train_data method. I assume that there is some inefficient memory load while transforming the train data, but I didn't look too deep into this code segment :).
I ran this over a process with 400GB, so memory shouldn't be an issue, unless something really weird is happening.
The code I ran is the same code as the one in PATE-CTGAN in the smart noise README.md file.
When I ran a synthesizer with a PATE-CTGAN model over a large database (not too large tbh, a little bit less than 100MB) the fit will fail and the kernel will crash.
I tried to debug the problem and figured out the model doesn't even start the fitting process but gets stuck at the
_get_train_datamethod. I assume that there is some inefficient memory load while transforming the train data, but I didn't look too deep into this code segment :).I ran this over a process with 400GB, so memory shouldn't be an issue, unless something really weird is happening.
The code I ran is the same code as the one in PATE-CTGAN in the smart noise README.md file.