Python New Style Class

Recently, I am curious about this code snippet:

class Test(object):
    def __init__():
        pass

Why do we have to extend or inherit from object type?

There are no differences with or without (object) in Python 3. But there are significant differences in Python 2. If a class inherits (either directly or indirectly) from object type, then it will be a new-style class. Otherwise, it will be an old-style class. It is suggested to avoid old-style classes in the newly written code. However, knowing the differences between old-style classes and new-style classes is important to every Python 2 programmers. We will cover several aspects in this post.

Slots

__slots__ is a special class attribute which controls the instance variables that can be get from or set to an instantiated object. If __slots__ is specified in a new-style class, heap memory usages can be reduced because __slots__ prevents the creation of the __dict__ data structure.

If a new-style class has the __slots__ class attribute and some code gets or sets an unspecified attribute from or to the instance, then AttributeError will be raised:

class A(object):
    __slots__ = ['x']

try:
    a = A()
    a.y = 10
except AttributeError:
    print 'Caught expected exception'

However, __slots__ does not work with old-style classes:

class B:
    __slots__ = ['x']

b = B()
b.y = 10  # No AttributeError
print 'b.y = ', b.y
print 'b.__dict__ = ', b.__dict__

In the snippet above, b.__dict__ is created regardless the __slots__ class attribute. The code succeed setting the y attribute to the instance of B. And, an extra item was added to b.__dict__ after the assignment. This indicates that we lose all of the benefits of __slots__.

Properties

If a class attribute of a new-style class refers to a property instance, then the associated getter, setter, or deleter will be invoked when some code access the corresponding instance attribute. For example:

class A(object):
    def __init__(self):
        self._value = 0

    def get_x(self):
        print 'get_x:', self._value
        return self._value

    def set_x(self, value):
        print 'set_x:', self._value, value
        self._value = value

    def del_x(self):
        print 'del_x'

    x = property(get_x, set_x, del_x)

a = A()

# Test getter
print a.x
# Prints: get_x: 0
# Prints: 0

# Test setter
a.x = 5
# Prints: set_x: 0 5
print a._value
# Prints: 5

print a.x
# Prints: get_x: 5
# Prints: 5

# Test deleter
del a.x
# Prints: del_x

In the code snippet above, we can see that every accesses to the instance of a.x will be delegated to the associated methods. Furthermore, if a property does not have a setter and some code assign a value to such attribute, then an AttributeError exception will be raised:

class A2(object):
    def get_x(self):
        return 1

    x = property(get_x)

a2 = A2()
a2.x = 5
# Prints: Traceback (most recent call last):
# Prints:   File "test.py", line 10, in <module>
# Prints:    a2.x = 5
# Prints: AttributeError: can't set attribute

However, property works differently if is referred by a class attribute of an old-style class. In the code snippet below, an old-style class B has a class attribute x which is bound to a property instance:

class B:  # B is NOT derived from `object` type
    def __init__(self):
        self._value = 0

    def get_x(self):
        print 'get_x:', self._value
        return self._value

    def set_x(self, value):
        print 'set_x:', self._value, value
        self._value = value

    def del_x(self):
        print 'del_x'

    x = property(get_x, set_x, del_x)

b = B()

# Test getter
print b.x
# Prints: get_x: 0
# Prints: 0

So far, the output looks similar. However, the output will become different when the code assign a value to b.x:

# Test setter
b.x = 5  # Not calling set_x()

print b._value  # b._value left unchanged
# Prints: 0

print b.x  # Not calling get_x()
# Prints: 5

print b.__dict__
# Prints: {'_value': 0, 'x': 5}

In the code snippet above, the assignment statement (which assigned 5 to b.x) was not delegated to set_x(), thus b._value was left unchanged. Besides, the assignment statement added a new attribute x to the instance attribute dictionary, thus b.__dict__ contains two items: _value and x. Furthermore, the attribute in the instance attribute dictionary hides the class attribute. Consequently, accessing b.x will not be delegated to get_x() method after the assignment.

In old-style classes, del statements will only remove the attribute from the instance attribute dictionary __dict__. If the attribute is not in __dict__, then an AttributeError will be raised. The deleter won't be called. For example:

# Test deleter
del b.x  # Remove x from b.__dict__
print b.__dict__
# Prints: {'_value': 0}

del b.x
# Prints: Traceback (most recent call last):
# Prints:   File "./test.py", line 44, in <module>
# Prints:     del b.x
# Prints: AttributeError: B instance has no attribute 'x'

Likewise, assigning a value to a property without a setter function will not result in AttributeError. It will simply add a new attribute to instance attribute dictionary and hide the class attribute:

class B2:  # B2 is NOT derived from `object` type
    def get_x(self):
        return 1

    x = property(get_x)

b2 = B2()
b2.x = 5  # No AttributeError
print(b2.__dict__)
# Prints: {'x': 5}

In short, the setter and deleter function do not work with old-style classes and the getter function will work only if the class attribute is not hidden by the instance attribute.

Method Resolution Order (MRO)

In Python, a derived class may override a method inherited from the base class. However, there are some differences when multiple inheritance is involved. For example, which test() method will be invoked when class D inherits both class B and C but only class C overrides the test() method?

class A(object):
    def test(self):
        print 'A.test()'

class B(A):
    pass

class C(A):
    def test(self):
        print 'C.test()'

class D(B, C):
    pass

d = D()
d.test()
# Prints: C.test()

If these classes are new-style classes, then C.test() will be invoked. The method resolution order for new-style classes was introduced in Python 2.3. Conceptually, the method resolution order introduced in Python 2.3 tries to pick the most specific method. We can inspect the order by printing the __mro__ class attribute:

print A.__mro__
# Prints: (<class '__main__.A'>, <type 'object'>)
print B.__mro__
# Prints: (<class '__main__.B'>, <class '__main__.A'>, <type 'object'>)
print C.__mro__
# Prints: (<class '__main__.C'>, <class '__main__.A'>, <type 'object'>)
print D.__mro__
# Prints: (<class '__main__.D'>, <class '__main__.B'>,
# Prints: <class '__main__.C'>, <class '__main__.A'>, <type 'object'>)

As shown in the output, Python run-time will look for the test() method from class D, B, C:, and A (in order) when we call d.test().

The resolution order is different if these classes are old-style classes. For example:

class E:  # E is NOT derived from `object` type.
    def test(self):
        print 'E.test()'

class F(E):
    pass

class G(E):
    def test(self):
        print 'G.test()'

class H(F, G):
    pass

h = H()
h.test()
# Prints: E.test()

In the code snippet above, the generic E.test() from the base class E will be invoked instead of the more specific G.test(). If a old-style class does not override the method, then Python run-time will look for the method from its base classes (from left to right). In this example, it will look for test() method in class F first and recursively look for test() method in class E, which is the base class of F. Since class E defines a test() method, the resolution stops at class E and class G will not be queried.

Besides, old-style classes do not have __mro__ class attribute either:

print H.__mro__
# Prints: Traceback (most recent call last):
# Prints:   File "test.py", line 21, in <module>
# Prints:     print H.__mro__
# Prints: AttributeError: class H has no attribute '__mro__'

Super

super() is a built-in function which helps us to invoke the overridden method. For example, when class B inherits from class A, B.test() overrides A.test(), and the implementation of B.test() would like to call A.test(), then it can utilize the super() function. The super() function takes the current class object and the self reference as the argument and it will find the overridden method to be dispatched to.

class A(object):
    def test(self):
        print 'A.test()'

class B(A):
    def test(self):
        super(B, self).test()
        print 'B.test()'

b = B()
b.test()
# Prints: A.test()
# Prints: B.test()

However, super() only works for new-style classes. It does NOT work with old-style classes. A TypeError exception will be raised, if an old-style class object is passed as the first argument:

class C:
    def test(self):
        print 'C.test()'

class D(C):
    def test(self):
        super(D, self).test()
        print 'D.test()'

d = D()
d.test()
# Prints: Traceback (most recent call last):
# Prints:   File "./test.py", line 13, in <module>
# Prints:     d.test()
# Prints:   File "./test.py", line 9, in test
# Prints:     super(D, self).test()
# Prints: TypeError: super() argument 1 must be type, not classobj

To call the parent methods in old-style classess, we have to explicitly get the methods from the class attribute, and pass the self argument explicitly. For example:

class E:
    def test(self):
        print('E.test()')

class F(E):
    def test(self):
        E.test(self)
        print('F.test()')

class G(E):
    def test(self):
        E.test(self)
        print('G.test()')

class H(F, G):
    def test(self):
        F.test(self)
        G.test(self)
        print('H.test()')

h = H()
h.test()
# Prints: E.test()
# Prints: F.test()
# Prints: E.test()
# Prints: G.test()
# Prints: H.test()

However, this mechanism is sort of primitive. If there is a diamond shape in the inheritance hierarcy, then some methods from the base classes will be invoked more than once. For example, the class H above inherits both F and G and both of them will invoke E.test(). If this is undesired, then some extra bookkeeping code must be written.

Conclusion

In this post, we have covered four differents between new-style classes and old-style classes:

  1. The __slots__ mechanism doesn't work with old-style classes
  2. The setter and deleter of property doesn't work with old-style classes.
  3. The method resolution order is slightly different.
  4. The super() function doesn't work with old-style classes.

There might still be some other nuances that are not covered by this post. However, IMO, the rule of thumb is to adopt new-style classes as soon as possible and only use new-style classes in your code because Python 3 doesn't support old-style classes.