Hi, recently I am running DLRM training. I do not understnad the necessity of creating a dictionary in data_utils.py after a modulus operation. This is time-expensive. I know the dictionary is used to collect unique values and then assign correpsonding index to it. But why it is necessary to keep this process here? A simple hashing function, like Sigrid hash, or just modulus should be enough.
Hi, recently I am running DLRM training. I do not understnad the necessity of creating a dictionary in data_utils.py after a modulus operation. This is time-expensive. I know the dictionary is used to collect unique values and then assign correpsonding index to it. But why it is necessary to keep this process here? A simple hashing function, like Sigrid hash, or just modulus should be enough.