How to not shoot yourself in the foot with Python for Qt / PyQt

6 minutes read

In the last few weeks, I have been using Python and Qt, especially PyQt extensively.
During these weeks I have discovered a few ways how to shoot yourself in the foot accidentally.

In this article, you will learn which things you should watch out for when working on GUIs with Qt and Python and how to avoid the resulting problems.

The Application can't be stopped with Ctrl-C

The first thing you will notice when writing your first hello world application with PyQt is that you can't stop it anymore from the command-line with Ctrl-C.

Let's take a look at the following PyQt QML hello world application:

main.py

# -*- coding: utf-8 -*-
import sys

from PyQt5.QtGui import QGuiApplication
from PyQt5.QtQml import QQmlApplicationEngine

if __name__ == '__main__':
    app = QGuiApplication(sys.argv)

    engine = QQmlApplicationEngine()
    engine.load('./main.qml')

    sys.exit(app.exec_())

main.qml

import QtQuick 2.5
import QtQuick.Controls 2.0

ApplicationWindow {
    id: root
    visible: true

    Label {
        anchors.centerIn: parent
        text: qsTr("Hello World!")
    }
}

This code snippet works perfectly fine and when you execute it you will see the hello world window as expected.

However, the first time you try to stop the application from the console you will see that Ctrl-C has no effect and the application just keeps running.

$ python ./main.py
^C^C^C^C^C^C^C^C

Only when hitting the exit button in the application window, the application finally stops as we the expected KeyboardInterruptError:

 File "./main.py", line 13, in <module>
    sys.exit(app.exec_())
KeyboardInterrupt

Whats happening here?

Qt strongly builds on a concept called event loop. Such an event loop enables you to write parallel applications without multithreading. The concept of event loops is especially useful for applications where a long living process needs to handle interactions from a user or client. Therefore, you often will find event loops being used in GUI or web frameworks.

However, the pitfall here is that Qt is implemented in C++ and not in Python. When we execute app.exec_() we start the Qt/C++ event loop, which loops forever until it is stopped.

The problem here is that we don't have any Python events set up yet. So our event loop never churns the Python interpreter and so our signal delivered to the Python process is never processed. Therefore, our Python process never sees the signal until we hit the exit button of our Qt application window.

To circumvent this problem is very easy. We just need to set up a timer kicking off our event loop every few milliseconds.

# -*- coding: utf-8 -*-
import sys

from PyQt5.QtCore import QTimer
from PyQt5.QtGui import QGuiApplication
from PyQt5.QtQml import QQmlApplicationEngine

if __name__ == '__main__':
    app = QGuiApplication(sys.argv)

    engine = QQmlApplicationEngine()
    engine.load('./main.qml')

    timer = QTimer()
    timer.timeout.connect(lambda: None)
    timer.start(100)

    sys.exit(app.exec_())

All we added to the hello world application is the code to start to create and start a timer every 100 milliseconds. This way we can safely terminate our application with Ctrl-C from the command line.

NOTE: Don't forget to store the variable containing the timer instance somewhere or your timer instance will be garbage collected.

Your Python object lives longer than the QObject

Another "Shoot yourself in the foot" experience that you will potentially have using Python and Qt is related to memory management.

As we all know, Python supports automatic memory management, meaning a garbage collector looks for variables that aren't referenced anymore and frees memory. This usually works very well and it's probably one of the features that make Python way easier to work with than for example C++.

C++, on the other hand, allows more freedom when it comes to memory management. Freeing up resources is up to the programmer and in the responsibility of the C++ class. For this purpose, C++ objects have a destructor which is called when an object is destroyed.

However, for more complex applications memory management isn't as easy as it sounds. Who is responsible for cleaning up objects created outside of an instance and assigned to another object instance for example?

Qt and many other GUI related frameworks, therefore, come with their own memory management mechanism that is especially suitable for window-based graphical applications. The principle is pretty simple. Every object can have children and it is responsible for cleaning up its children. Let's say for example an application window has a child which is a button. When the window is closed, the window calls the deleteLater function of the button. This ensures that for example a complex GUI form is cleaned up in the right order.

Additionally, QObject's do not delete their children instantly, but instead, delegate the object deletion to the event loop. This ensures that for example objects created in other tasks are also cleaned up correctly.

To further complicate things, QtQuick also has a garbage collector similar to Python.

In most cases, you don't need to really care about Qt's memory management when working with Python and Qt. However, in some cases, it is possible that the Python object, or parts of it, lives longer than the Qt object.

Let's take a look at the following example.

long_living_object.py

# -*- coding: utf-8 -*-
import sys
import time
from threading import Timer

from PyQt5.QtCore import QTimer, QObject, pyqtSignal, pyqtProperty
from PyQt5.QtGui import QGuiApplication
from PyQt5.QtQml import QQmlApplicationEngine, qmlRegisterType


class GlobalTimer(object):
    def __init__(self, interval=1.0):
        self._registered = set()
        self._interval = interval
        self._timer = None
        self._start_timer()

    def register_callback(self, callback):
        self._registered.add(callback)

    def unregister_callback(self, callback):
        self._registered.remove(callback)

    def _start_timer(self):
        self._timer = Timer(self._interval, self._callback)
        self._timer.start()

    def _callback(self):
        for callback in self._registered:
            callback()
        self._start_timer()


Scheduler = GlobalTimer()


class Clock(QObject):

    timestampChanged = pyqtSignal(int)

    def __init__(self, parent=None):
        super(Clock, self).__init__(parent)
        self._timestamp = time.time()
        Scheduler.register_callback(self.tick)

    @pyqtProperty(int, notify=timestampChanged)
    def timestamp(self):
        return self._timestamp

    def tick(self):
        self._timestamp = time.time()
        self.timestampChanged.emit(self._timestamp)


if __name__ == '__main__':
    app = QGuiApplication(sys.argv)

    qmlRegisterType(Clock, 'mymodule', 1, 0, Clock.__name__)

    engine = QQmlApplicationEngine()
    engine.load('./long_living_object.qml')

    timer = QTimer()
    timer.timeout.connect(lambda: None)
    timer.start(100)

    sys.exit(app.exec_())

long_living_object.qml

import QtQuick 2.5
import QtQuick.Controls 2.0
import mymodule 1.0

ApplicationWindow {
    id: root
    visible: true
    width: 300
    height: 300

    Loader {
        id: clockLoader
        anchors.centerIn: parent
        active: showCheck.checked
        sourceComponent: clockComponent
    }

    Component {
        id: clockComponent
        Label {
            text: clock.timestamp

            Clock { id: clock }
        }
    }

    CheckBox {
        id: showCheck
        anchors.right: parent.right
        anchors.bottom: parent.bottom
        text: qsTr("Show clock")
        checked: true
    }
}

In this example, we have GlobalTimer class which is instantiated as global Scheduler object. Inside the Clock class we register the tick function to as a callback.

In QML we use the Clock object inside a Loader component, so we can create and destroy it on demand. The running application looks as follows:

When we uncheck the showCheck checkbox, the Loader destroys our Label and Clock components. However, our global Scheduler object still keeps ticking and we soon see the following error in our console.

Exception in thread Thread-3:
Traceback (most recent call last):
  File "/usr/lib/python3.5/threading.py", line 914, in _bootstrap_inner
    self.run()
  File "/usr/lib/python3.5/threading.py", line 1180, in run
    self.function(*self.args, **self.kwargs)
  File "/how-to-not-shoot-yourself-in-the-foot/examples/long_living_object.py", line 30, in _callback
    callback()
  File "/how-to-not-shoot-yourself-in-the-foot/examples/long_living_object.py", line 52, in tick
    self.timestampChanged.emit(self._timestamp)
RuntimeError: wrapped C/C++ object of type Clock has been deleted

It turns out our Python object outlived the QObject because the global Scheduler object still has a reference to our tick instance function.

Of course, we shouldn't design our application this way in the first place, but sometimes we depend on external libraries that we can't control. For example, a middleware library which triggers callbacks when a message arrives.

So how do we fix this?

Well, the solution sounds trivial, just unregister our callback when the Python object is destroyed.

But wait - Python classes do not have a destructor (yes, there is __del__ method, but our Python object still lives, doesn't it?).

Luckily QObject has a signal that is triggered before destruction, the destroyed signal.

Sounds easy, let's try:

...
        self._timestamp = time.time()
        Scheduler.register_callback(self.tick)

        self.destroyed.connect(self._unregister)

    def _unregister(self):
        Scheduler.unregister_callback(self.tick)

    @pyqtProperty(int, notify=timestampChanged)
        def timestamp(self):
...

Okay, you quickly will see that it doesn't work. - Another hole in the foot.

For some reason, it doesn't work to connect instance functions directly to the destroyed signal. (If you know why, please let me know.)

However, it turns out that lambdas work fine:

self.destroyed.connect(lambda: self._unregister())

And finally, our application doesn't throw errors anymore.

NOTE: Qt signals and slots are disconnected automagically on QObject destruction -> therefore, if you can use Qt signals and slots instead of callback functions.

Alternatively, we could also use a weakref in this particular example.

Conclusion

Python and Qt are a great combo. However, it is very easy to shoot yourself in the foot.

To avoid such unwanted problems remember the following:

• Always keep Python in the loop.
• Use references to Python Qt objects carefully and cleanup correctly.
• Use lambdas in QObject.destroyed.

I hope you have enjoyed reading this blog post and if so please subscribe and share it with your friends.

Your
Machine Koder