Discussion about lab3

[harshraj22]

All discussions related to lab3

[harshraj22]

Brainstorming about the project structure for lab3

We should follow the similar structure to lab2 for API design while writing classes for lab3. It is easy to extend and has similarity with already known ones like PyTorch

[harshraj22]

• The forward(...) method needs to accept the env as it needs the reward corresponding to each action from current state. These rewards can be calculated at runtime inside the forward method making it more generic, rather than pre-calculating it, and making that workable only for GridWorld kind of envs.
• If we pass the env to agent, we need to ensure that the agent does not change the internals of the env. Probably need a context manager in utils/utils.py that makes copy of env, and the agent works with this copy rather than actual env.
• Or maybe the env can be given a method to reset the state to a given state, and agent takes action and resets continiously.