Could someone help me understand this python code?

Watch
__itertools__
Badges: 11
Rep:
?
#1
Report Thread starter 1 year ago
#1
Hi, I was going through this python code. It is part (the first one, in fact) of Microsoft's Reinforcement Learning Explained Course.

The instructions are as follows:
Spoiler:
Show





Those are implementation of the several environments that we will use throughout this course. Open and examine the simple_rooms.py file.
The Environment class is provided as an interface. An environment must have some representation of the state of which the agent is interacting with. In addition, an environment must be able to reset it self and step to the next state. These are implemented in both the reset() and the step() function. The reset() function should return the initial state, while the step() function should take in an action and at the minimum, return the next state and the reward(). The actions() function maintains the information of how many type of actions in the environment. This is used in conjunction with the ActionSpace class.
Let's take a look at the SimpleRoomsEnv class, which implements the Environment class, and examine this in more details. The SimpleRoomsEnv is a simple environment of a 4x4 rooms, limited by walls. The initial state has the agent starting at the room on top left corner, with the goal to reach the room at the bottom right corner. Take some time to study the implementation of this environment. Start by examining how the states are represented in this environment. Also, look at how the SimpleRoomsEnv class implements the reset() and step() functions as these two are the ones used to interact with an agent.






If you look at the class SimpleRoomsEnv which inherits from the Environment class, you can see that __init__ method uses the __init__ method from its superclass, which is Environment; but Environment does not have an init method. So, will it look for the method in the Python's built in class called object? What does object's init method do?

Could anyone tell me what the following code does?
def __init__(self):
....super(SimpleRoomsEnv, self).__init__()

Thank You!
0
reply
Clutch A
Badges: 9
Rep:
?
#2
Report 1 year ago
#2
You are calling the method "__init__()" which resides in the parent class SimpleRoomsEnv
0
reply
__itertools__
Badges: 11
Rep:
?
#3
Report Thread starter 1 year ago
#3
(Original post by Clutch A)
You are calling the method "__init__()" which resides in the parent class SimpleRoomsEnv
Thanks for the reply.
super(SimpleRoomsEnv, self).__init__()
Doesn't that mean, call the __init__() method from the superclass of SimpleRoomsEnv and pass in self as an argument. Isn't SimpleRoomsEnv the inheriting class, not the parent class?
0
reply
winterscoming
Badges: 19
Rep:
?
#4
Report 1 year ago
#4
For single-inheritance, super(SimpleRoomsEnv, self).__init__() essentially does exactly the same thing as Environment.__init__(self).

However, for multiple inheritance it provides an interesting way of handling the "Dreaded Diamond" problem by ensuring that the __init__ methods for all superclasses are each called exactly once by eliminating duplicates, but also ensures that the order in which they're called is consistent too

The inheritance heirarchy is essentially a graph of classes - obviously there are multiple __init__ methods which need to be called, and they need to be called in a specific order known as the Method Resolution Order (MRO). Python defines an algorithm for this which is basically a depth-first search.

This solution to the diamond problem is known as 'cooperative' multiple inheritance. There are other solutions, but this one is (IMO) a much better one than you get from languages like C++


Firstly, look at what happens when you attempt to implement the dreaded diamond without using super():
https://repl.it/repls/UrbanKindheartedWorkspace

Look carefully at the child class __init__() method - Two things:
- Firstly, the need to call 'right.__init__(self)' and 'left.__init__(self)' -- this is problematic on its own because a programmer must remember to call those in the correct order to be consistent with the order in which right and left are inherited by child. (i.e. base->right->left->child)

- Secondly, the output shows that the base.__init__() method is called twice - probably not what a programmer wants to happen because it's duplication,

One thing I'd point out is that there's nothing technically "wrong" here - this is the classic way of solving the multiple inheritance diamond problem, and this is the way that C++ does things. It sucks though - you end up with the base class being inherited twice and needing to care about the inheritance order in two different places - this is is one of the reasons why multiple inheritance is a bit dirty in C++ and it just makes the code harder to work with.


Python has a far more elegant solution to this called 'cooperative multiple inheritance' which uses the built-in super() function to avoid making any direct references to the superclass, adding an extra level of indirection instead to let super() call the correct parent or sibling according to Python's "Method Resolution Order" rules (basically a depth-first algorithm which steps through the multiple inheritance 'graph').
https://repl.it/repls/EdibleLinearCgi

Two things here:
- Firstly, super() figures out the Method Resolution Order (all nicely take care of, no need to over-think it!)
- Secondly, super() eliminates the extra (usually unwanted) call out to the class called "base".


Maybe some interesting reading related to this:
https://www.python.org/download/releases/2.3/mro/
https://docs.python.org/3/library/functions.html#super
0
reply
winterscoming
Badges: 19
Rep:
?
#5
Report 1 year ago
#5
(Original post by __itertools__)
Thanks for the reply.

Doesn't that mean, call the __init__() method from the superclass of SimpleRoomsEnv and pass in self as an argument. Isn't SimpleRoomsEnv the inheriting class, not the parent class?
Yes, SimpleRoomsEnv is the derived class.

the parent class is Environment, but 'super()' figures that out for you because it knows what the inheritance hierarchy looks like, so in effect it's actually calling Environment.__init__(self)
0
reply
__itertools__
Badges: 11
Rep:
?
#6
Report Thread starter 1 year ago
#6
(Original post by winterscoming)
Yes, SimpleRoomsEnv is the derived class.

the parent class is Environment, but 'super()' figures that out for you because it knows what the inheritance hierarchy looks like, so in effect it's actually calling Environment.__init__(self)
Thank You for the reply, but Environment class does not have the init constructor, in the given code.
0
reply
winterscoming
Badges: 19
Rep:
?
#7
Report 1 year ago
#7
(Original post by __itertools__)
Thank You for the reply, but Environment class does not have the init constructor, in the given code.
There is one on the object class though, so Environment will inherit the __init__() method from that if it doesn't have its own.
0
reply
__itertools__
Badges: 11
Rep:
?
#8
Report Thread starter 1 year ago
#8
(Original post by winterscoming)
For single-inheritance, super(SimpleRoomsEnv, self).__init__() essentially does exactly the same thing as Environment.__init__(self).

However, for multiple inheritance it provides an interesting way of handling the "Dreaded Diamond" problem by ensuring that the __init__ methods for all superclasses are each called exactly once by eliminating duplicates, but also ensures that the order in which they're called is consistent too

The inheritance heirarchy is essentially a graph of classes - obviously there are multiple __init__ methods which need to be called, and they need to be called in a specific order known as the Method Resolution Order (MRO). Python defines an algorithm for this which is basically a depth-first search.

This solution to the diamond problem is known as 'cooperative' multiple inheritance. There are other solutions, but this one is (IMO) a much better one than you get from languages like C++


Firstly, look at what happens when you attempt to implement the dreaded diamond without using super():
https://repl.it/repls/UrbanKindheartedWorkspace

Look carefully at the child class __init__() method - Two things:
- Firstly, the need to call 'right.__init__(self)' and 'left.__init__(self)' -- this is problematic on its own because a programmer must remember to call those in the correct order to be consistent with the order in which right and left are inherited by child. (i.e. base->right->left->child)

- Secondly, the output shows that the base.__init__() method is called twice - probably not what a programmer wants to happen because it's duplication,

One thing I'd point out is that there's nothing technically "wrong" here - this is the classic way of solving the multiple inheritance diamond problem, and this is the way that C++ does things. It sucks though - you end up with the base class being inherited twice and needing to care about the inheritance order in two different places - this is is one of the reasons why multiple inheritance is a bit dirty in C++ and it just makes the code harder to work with.


Python has a far more elegant solution to this called 'cooperative multiple inheritance' which uses the built-in super() function to avoid making any direct references to the superclass, adding an extra level of indirection instead to let super() call the correct parent or sibling according to Python's "Method Resolution Order" rules (basically a depth-first algorithm which steps through the multiple inheritance 'graph').
https://repl.it/repls/EdibleLinearCgi

Two things here:
- Firstly, super() figures out the Method Resolution Order (all nicely take care of, no need to over-think it!)
- Secondly, super() eliminates the extra (usually unwanted) call out to the class called "base".


Maybe some interesting reading related to this:
https://www.python.org/download/releases/2.3/mro/
https://docs.python.org/3/library/functions.html#super
(Original post by winterscoming)
There is one on the object class though, so Environment will inherit the __init__() method from that if it doesn't have its own.
Thank You and PRSOM!!!
Last edited by __itertools__; 1 year ago
0
reply
winterscoming
Badges: 19
Rep:
?
#9
Report 1 year ago
#9
(Original post by __itertools__)
Thank You and PRSOM!!!
The object class is the built in class whose init method does nothing right?
Yep, that's correct - object.__init__() doesn't do anything

(Well, almost anyway. the object.__init__() can be useful when you're using Python's *args and **kwargs syntax with an __init__ method because it will helpfully throw an error to highlight programmer mistakes when using those variable argument 'lists' - but that's a bit of a special case that you're probably not going to use very often. so don't worry too much about that)
0
reply
The Topnotch
Badges: 3
Rep:
?
#10
Report 1 year ago
#10
email me the code i run it on my machine this just a simple work
0
reply
X

Quick Reply

Attached files
Write a reply...
Reply
new posts
Back
to top
Latest
My Feed

See more of what you like on
The Student Room

You can personalise what you see on TSR. Tell us a little about yourself to get started.

Personalise

Do you have the space and resources you need to succeed in home learning?

Yes I have everything I need (288)
55.81%
I don't have everything I need (228)
44.19%

Watched Threads

View All
Latest
My Feed