Recently, I am curious about this code snippet:
class Test(object):
def __init__():
pass
Why do we have to extend or inherit from object
type?
There are no differences with or without (object)
in Python 3. But
there are significant differences in Python 2. If a class inherits (either
directly or indirectly) from object
type, then it will be a new-style
class. Otherwise, it will be an old-style class. It is suggested to avoid
old-style classes in the newly written code. However, knowing the differences
between old-style classes and new-style classes is important to every Python
2 programmers. We will cover several aspects in this post.
Slots
__slots__
is a special class attribute which controls the instance
variables that can be get from or set to an instantiated object. If
__slots__
is specified in a new-style class, heap memory usages can be
reduced because __slots__
prevents the creation of the __dict__
data structure.
If a new-style class has the __slots__
class attribute and some code
gets or sets an unspecified attribute from or to the instance, then
AttributeError
will be raised:
class A(object):
__slots__ = ['x']
try:
a = A()
a.y = 10
except AttributeError:
print 'Caught expected exception'
However, __slots__
does not work with old-style classes:
class B:
__slots__ = ['x']
b = B()
b.y = 10 # No AttributeError
print 'b.y = ', b.y
print 'b.__dict__ = ', b.__dict__
In the snippet above, b.__dict__
is created regardless the
__slots__
class attribute. The code succeed setting the y
attribute to the instance of B
. And, an extra item was added to
b.__dict__
after the assignment. This indicates that we lose all of the
benefits of __slots__
.
Properties
If a class attribute of a new-style class refers to a property
instance, then the associated getter, setter, or deleter will be invoked when
some code access the corresponding instance attribute. For example:
class A(object):
def __init__(self):
self._value = 0
def get_x(self):
print 'get_x:', self._value
return self._value
def set_x(self, value):
print 'set_x:', self._value, value
self._value = value
def del_x(self):
print 'del_x'
x = property(get_x, set_x, del_x)
a = A()
# Test getter
print a.x
# Prints: get_x: 0
# Prints: 0
# Test setter
a.x = 5
# Prints: set_x: 0 5
print a._value
# Prints: 5
print a.x
# Prints: get_x: 5
# Prints: 5
# Test deleter
del a.x
# Prints: del_x
In the code snippet above, we can see that every accesses to the instance of
a.x
will be delegated to the associated methods. Furthermore, if a
property does not have a setter and some code assign a value to such attribute,
then an AttributeError
exception will be raised:
class A2(object):
def get_x(self):
return 1
x = property(get_x)
a2 = A2()
a2.x = 5
# Prints: Traceback (most recent call last):
# Prints: File "test.py", line 10, in <module>
# Prints: a2.x = 5
# Prints: AttributeError: can't set attribute
However, property works differently if is referred by a class attribute of an
old-style class. In the code snippet below, an old-style class B
has a class attribute x
which is bound to a property
instance:
class B: # B is NOT derived from `object` type
def __init__(self):
self._value = 0
def get_x(self):
print 'get_x:', self._value
return self._value
def set_x(self, value):
print 'set_x:', self._value, value
self._value = value
def del_x(self):
print 'del_x'
x = property(get_x, set_x, del_x)
b = B()
# Test getter
print b.x
# Prints: get_x: 0
# Prints: 0
So far, the output looks similar. However, the output will become different
when the code assign a value to b.x
:
# Test setter
b.x = 5 # Not calling set_x()
print b._value # b._value left unchanged
# Prints: 0
print b.x # Not calling get_x()
# Prints: 5
print b.__dict__
# Prints: {'_value': 0, 'x': 5}
In the code snippet above, the assignment statement (which assigned 5
to
b.x
) was not delegated to set_x()
, thus b._value
was
left unchanged. Besides, the assignment statement added a new attribute
x
to the instance attribute dictionary, thus b.__dict__
contains
two items: _value
and x
. Furthermore, the attribute in the
instance attribute dictionary hides the class attribute. Consequently,
accessing b.x
will not be delegated to get_x()
method after the
assignment.
In old-style classes, del
statements will only remove the attribute
from the instance attribute dictionary __dict__
. If the attribute is
not in __dict__
, then an AttributeError
will be raised. The
deleter won't be called. For example:
# Test deleter
del b.x # Remove x from b.__dict__
print b.__dict__
# Prints: {'_value': 0}
del b.x
# Prints: Traceback (most recent call last):
# Prints: File "./test.py", line 44, in <module>
# Prints: del b.x
# Prints: AttributeError: B instance has no attribute 'x'
Likewise, assigning a value to a property without a setter function will not
result in AttributeError
. It will simply add a new attribute to
instance attribute dictionary and hide the class attribute:
class B2: # B2 is NOT derived from `object` type
def get_x(self):
return 1
x = property(get_x)
b2 = B2()
b2.x = 5 # No AttributeError
print(b2.__dict__)
# Prints: {'x': 5}
In short, the setter and deleter function do not work with old-style classes and the getter function will work only if the class attribute is not hidden by the instance attribute.
Method Resolution Order (MRO)
In Python, a derived class may override a method inherited from the base class.
However, there are some differences when multiple inheritance is involved. For
example, which test()
method will be invoked when class D
inherits both class B
and C
but only class C
overrides
the test()
method?
class A(object):
def test(self):
print 'A.test()'
class B(A):
pass
class C(A):
def test(self):
print 'C.test()'
class D(B, C):
pass
d = D()
d.test()
# Prints: C.test()
If these classes are new-style classes, then C.test()
will be invoked.
The method resolution order for new-style classes was introduced in Python
2.3. Conceptually, the method resolution order introduced in Python 2.3
tries to pick the most specific method. We can inspect the order by printing
the __mro__
class attribute:
print A.__mro__
# Prints: (<class '__main__.A'>, <type 'object'>)
print B.__mro__
# Prints: (<class '__main__.B'>, <class '__main__.A'>, <type 'object'>)
print C.__mro__
# Prints: (<class '__main__.C'>, <class '__main__.A'>, <type 'object'>)
print D.__mro__
# Prints: (<class '__main__.D'>, <class '__main__.B'>,
# Prints: <class '__main__.C'>, <class '__main__.A'>, <type 'object'>)
As shown in the output, Python run-time will look for the test()
method
from class D
, B
, C
:, and A
(in order) when we
call d.test()
.
The resolution order is different if these classes are old-style classes. For example:
class E: # E is NOT derived from `object` type.
def test(self):
print 'E.test()'
class F(E):
pass
class G(E):
def test(self):
print 'G.test()'
class H(F, G):
pass
h = H()
h.test()
# Prints: E.test()
In the code snippet above, the generic E.test()
from the base class
E
will be invoked instead of the more specific G.test()
. If a
old-style class does not override the method, then Python run-time will look
for the method from its base classes (from left to right). In this example, it
will look for test()
method in class F
first and recursively
look for test()
method in class E
, which is the base class of
F
. Since class E
defines a test()
method, the
resolution stops at class E
and class G
will not be queried.
Besides, old-style classes do not have __mro__
class attribute either:
print H.__mro__
# Prints: Traceback (most recent call last):
# Prints: File "test.py", line 21, in <module>
# Prints: print H.__mro__
# Prints: AttributeError: class H has no attribute '__mro__'
Super
super()
is a built-in function which helps us to invoke the overridden
method. For example, when class B
inherits from class A
,
B.test()
overrides A.test()
, and the implementation of B.test()
would like to call A.test()
, then it can utilize the
super()
function. The super()
function takes the current
class object and the self
reference as the argument and it will find the
overridden method to be dispatched to.
class A(object):
def test(self):
print 'A.test()'
class B(A):
def test(self):
super(B, self).test()
print 'B.test()'
b = B()
b.test()
# Prints: A.test()
# Prints: B.test()
However, super()
only works for new-style classes. It does NOT work
with old-style classes. A TypeError
exception will be raised, if an
old-style class object is passed as the first argument:
class C:
def test(self):
print 'C.test()'
class D(C):
def test(self):
super(D, self).test()
print 'D.test()'
d = D()
d.test()
# Prints: Traceback (most recent call last):
# Prints: File "./test.py", line 13, in <module>
# Prints: d.test()
# Prints: File "./test.py", line 9, in test
# Prints: super(D, self).test()
# Prints: TypeError: super() argument 1 must be type, not classobj
To call the parent methods in old-style classess, we have to explicitly get
the methods from the class attribute, and pass the self
argument
explicitly. For example:
class E:
def test(self):
print('E.test()')
class F(E):
def test(self):
E.test(self)
print('F.test()')
class G(E):
def test(self):
E.test(self)
print('G.test()')
class H(F, G):
def test(self):
F.test(self)
G.test(self)
print('H.test()')
h = H()
h.test()
# Prints: E.test()
# Prints: F.test()
# Prints: E.test()
# Prints: G.test()
# Prints: H.test()
However, this mechanism is sort of primitive. If there is a diamond shape in
the inheritance hierarcy, then some methods from the base classes will be
invoked more than once. For example, the class H
above inherits both
F
and G
and both of them will invoke E.test()
. If this
is undesired, then some extra bookkeeping code must be written.
Conclusion
In this post, we have covered four differents between new-style classes and old-style classes:
- The
__slots__
mechanism doesn't work with old-style classes - The setter and deleter of
property
doesn't work with old-style classes. - The method resolution order is slightly different.
- The
super()
function doesn't work with old-style classes.
There might still be some other nuances that are not covered by this post. However, IMO, the rule of thumb is to adopt new-style classes as soon as possible and only use new-style classes in your code because Python 3 doesn't support old-style classes.
Reference
- Guido van Rossum, Unifying types and classes in Python 2.2
- Python Wiki, New Class vs. Classic Class
- Python Wiki, New-style Classes
- The Python 2.3 Method Resolution Order