Qt

Positioning multiple windows in Qt

Submitted by mimec on 2013-05-28

It's been a while since the last post, but I've been quite busy with WebIssues. Now that the first beta of version 1.1 is released, it's time to catch up with other things. Actually I started writing this post some time ago, but then my HDD crashed, so I had to do it once again.

QWidget provides the saveGeometry/restoreGeometry pair of methods for a convenient way of storing and restoring the position of a window. This works fine when our application has just one top level window. But what if the application can open multiple windows of a given type? Consider a chat or email application, where each conversation or message can be opened in a separate window. It should be possible to store the position and size of the windows, but we also want to make sure that when the user opens multiple windows, they are displayed at slightly different positions, so that they don't cover one another entirely.

Doing this manually is not an easy task. Besides handling the maximized state of the window, restoreGeometry also ensures that the window is not displayed off screen when a display device is disconnected or its resolution has changed since the geometry was saved. Instead of implementing a custom mechanism, we can take advantage of saveGeometry/restoreGeometry, with a slight modification which makes multiple windows behave correctly.

The idea is that in addition to the geometry (which is serialized as a QByteArray) we also store a boolean flag called "offset". Whenever a new window is opened, we remember its position and set the offset to true, meaning that when another window is opened, an offset should be added to the stored position. When a window is closed, we remember its final position and set the offset to false. Another window will be then opened at the exact position of the old one. One way of doing this is handling the show and hide events:

void MyWindow::showEvent( QShowEvent* e )
{
    if ( !e->spontaneous() )
        storeGeometry( true );
}

void MyWindow::hideEvent( QHideEvent* e )
{
    if ( !e->spontaneous() )
        storeGeometry( false );
}

void MyWindow::storeGeometry( bool offset )
{
    QSettings settings;
    settings.setValue( "MyWindowGeometry", saveGeometry() );
    settings.setValue( "MyWindowOffset", offset );
}

Note that we ignore spontaneous events, which are fired when the window is minimized or restored by the user. All that's left is restoring the window's geometry. This can be done, for example, in the constructor of the window's class:

QSettings settings;
restoreGeometry( settings.value( "MyWindowGeometry" ).toByteArray() );
if ( settings.value( "MyWindowOffset" ).toBool() ) {
    QPoint position = pos() + QPoint( 40, 40 );
    QRect available = QApplication::desktop()->availableGeometry( this );
    QRect frame = frameGeometry();
    if ( position.x() + frame.width() > available.right() )
        position.rx() = available.left();
    if ( position.y() + frame.height() > available.bottom() - 20 )
        position.ry() = available.top();
    move( position );
}

We add an arbitrary offset of 40 pixels to the window's X and Y position. Then we ensure that the window doesn't span outside the right and bottom edge of the screen and otherwise move it to the left or top edge, respectively.

Note that this is not a perfect solution. If the user opens and closes the windows in certain order, they will overlap (I leave it as an excercise to the reader to come up with such scenario). But it nicely handles the two most common scenarios: reopening a closed window in the same location and ensuring that multiple windows, opened one after another, have slightly different positions.

Fuzzy optimizations in QTransform

Submitted by mimec on 2012-12-06

Before I get to the point, just a quick note: I recently released a new version of both WebIssues and Fraqtive and I'm planning to release a new version of Saladin by the end of this year, so I'm quite busy, as usual at this time of the year. Also check out the fractal animations created using the latest version of Fraqtive. There are much more impressive deep zoom animations available, but they were created using specialized tools which use high precision numbers. Fraqtive uses SSE2 to maximize the real-time experience, so it's limited to double precision, which allows to zoom the fractal about 10^13 times. But this is enough to produce some cool effects.

Now back to the main topic. For a long time there was a strange bug in Fraqtive that I thought was impossible to fix. As those who use it know, in Fraqtive it is possible to move and zoom around the fractal using the mouse, and when you release the mouse button, the new region of the fractal is recalculated. This works by converting the start and end position to a QTransform (which is basically a 3x3 matrix defining the offset, zoom and rotation) and calculating the relative transformation. It worked fine until a certain zoom level was reached; then it started producing weird results. I always blamed the limited precision of double numbers, though the effect could be observed a few orders of magnitude before reaching the maximum valid zoom.

Recently I debugged the entire code, including the calculations performed inside QTransform class and discovered that it performs some optimizations which cause the wrong results. The documentation mentions that QTransform performs some optimizations based on the type of the matrix. For example, if the transform includes translation only, the scale and rotation components are ignored. What the documentation doesn't mention, though, is that Qt uses fuzzy compare functions (such as qFuzzyCompare) to determine the type of the matrix.

The problem with qFuzzyCompare and it's undocumented qFuzzyIsNull counterpart is that they assume that only about 12 digits are significant in a double precision floating point number. This often makes sense, because limited precision can result in some subtle differences which cause the strict comparison operator to fail. We all know that 10 / 3 is 3.33333... and that value multiplied by 3 gives 9.99999.... In mathematics, this value is equal to 10, but for a computer, these numbers are not necessarily equal.

The effect of using fuzzy comparisons is that at a zoom level higher than 12, the scale factors are considered equal to 0, so the transformation is considered non-invertible, when in fact it is. Also rotation tends to be ignored at this zoom level. The solution is not to use QTransform when high precision is required or to write custom functions which do not have these side effects. See also [#QTBUG-8557] where a similar problem is discussed.

Note that fuzzy comparisons are used not only in QTransform, but also in other classes like QMatrix4x4 and QVector3D. Some time ago I came across the a similar problem with QVector3D::normalized which caused Descend to incorrectly calculate normal vectors for the surfaces. The problem is that Descend calculates three samples that are very close to one another in order to precisely determine the normal vector. In some areas of the surface it would hit the 12 significant digit limit, so I ended up writing my own version of this function which didn't use fuzzy comparisons.

Qt and ZIP files

Submitted by mimec on 2012-11-20

This is a follow up to the series of articles about serialization in Qt that I wrote a few months ago. Sometimes it's necessary to create complex files which contain lots of various information. Serializing everything into one stream of data is not always a good idea. Also sometimes it may be a good idea to compress serialized data, because it's usually quite verbose. Instead of coming up with a custom file format, the best idea is to use the solution which is implemented in may applications and document formats, including both OpenOffice and MS Office documents; that is to wrap data in a ZIP file.

There are many advantages of using the ZIP format. We can save data in multiple smaller files which are zipped together. We can include additional files and attachments, for example JPG images. Finally, we can compress selected files to make the resulting file more compact.

There are a few libraries for Qt which handle ZIP files, including OSDaB and QuaZIP. There are also the QZipReader and QZipWriter classes which are part of Qt. They are internal classes that you can find in src/gui/text subdirectory in Qt sources. There are plans to include them in official Qt API (see QTBUG-20896), but they will not make it into Qt 5.0 and it's not clear whether they will be included in later versions. However, if the license permits, you can simply copy them into your own application (but see the notes about static linking below).

A problem for Windows developers is that all these libraries and classes depend on zlib.h. This is fine on Linux, however zlib is (unfortunately) not part of Windows API. The good news is that QtCore not only includes zlib statically when built on Windows, but it also marks its functions as DLL exports. Thanks to this, the QZipReader and QZipWriter classes, which are part of QtGui, can still work on Windows. In my applications, including Descend which uses ZIP file format for its projects, I simply include the zlib.h and zconf.h files from Qt in the source package and use them when system zlib library is not available. Then I use the following simple .pri file to include either system zlib headers or the custom ones:

contains( QT_CONFIG, system-zlib ) {
  if( unix|win32-g++* ): LIBS += -lz
  else: LIBS += zdll.lib
} else {
  INCLUDEPATH += $$PWD
}

Thanks to this, #include "zlib.h" works no matter if zlib is a system library or not. When Qt is built without system-zlib (which is usually the case on Windows), it will include all the necessary exports in QtCore, so the application will link and work correctly.

Including internal Qt classes as part of the application is potentially dangerous, because there can be conflicts between the classes provided by Qt and by the application itself, especially when the application is linked with a different version of Qt. This is even more dangerous when the application is statically linked with the Qt libraries, which I usually do in Windows release builds to prevent DLL dependencies problems.

To avoid problems, I always remove the 'Q' prefix from such classes. This way they are seen as separate entities from the ones provided by Qt. However, I encountered a strange problem with Descend. In dynamically linked debug builds it worked fine, but when compiled in static release mode, it crashed when copying text to the clipboard. At first I suspected a strange bug in Qt or the compiler itself, and the problem was hard to debug because it only happened in release builds. However, after some analysis, I discovered that QPlainTextEdit copies text into the clipboard not only as plain text, but also in HTML and ODT formats. Coincidentally, creating an ODT document is exactly what Qt uses the internal QZipWriter class for...

It finally turned out that there was an innocent looking structure in qzip.cpp called FileHeader, which I slightly modified, but I didn't rename it. A plain structure would be fine, but this one had an implicit constructor and destructor, because it contained a QByteArray member. Unfortunately the linker doesn't detects such conflicts (it would not be possible anyway, because of how C++ works), but happily overrides the symbols from Qt library with identically named symbols defined in my application.

This can lead to unexpected problems not only when copying code from Qt, but also in many other situations. After all, many libraries can include a class named FileHeader. That's why static linking must be used carefully. Qt generally uses the Q prefix everywhere, even in private classes and functions, to prevent this type of problems, but this particular one didn't have it. If you ever experience mysterious crashes in release builds, this can be one of the possible reasons.

Serialization in Qt - part 4

Submitted by mimec on 2012-08-22

I the last few posts I wrote about serializing data in an extensible and effective binary format using QDataStream. So far it was focused on value types and simple structures that can be easily converted to QVariant and QVariantMap. In the last post I mentioned that creating objects dynamically based on class name requires implementing some kind of an object factory. Now let's analyse what is needed to serialize an entire hierarchy of abstract objects that refer to one another. Note that this is a very complex topic and there is no single, universal solution, so I won't provide the full code. Instead I will discuss what is necessary to craft such solution depending on the exact requirements.

Let's assume that we're serializing a project which consists of shapes of various types - circles, squares, etc. There are also some complex shapes, like groups or layers, which consist of other shapes. The first difficulty is that shape is an abstract type, so we need to store the actual class name along with the object data in order to be able to re-create the object upon deserialization. This was more or less covered in the last post.

Another difficulty is that objects refer to one another by pointers, forming a graph of relations, in which one object may be accessed from many other objects. We need to ensure that the object is only serialized and deserialized once, and all other references must be correctly maintained. There even can be cyclic dependencies; for example a parent object can have a pointer to a child, and the child can have a pointer to the parent.

For sake of simplicity I will assume that each serializable class inherits QObject; that's not really necessary, but having a single common base class makes things easier. The class should also be registered in the object factory discussed before. Finally, it should implement the following interface, which provides methods for serializing and deserializing the object:

class Serializable
{
public:
    virtual void serialize( QVariantMap& data, SerializationContext* context ) const = 0;
    virtual void deserialize( const QVariantMap& data, SerializationContext* context ) = 0;
};

The data is stored in a QVariantMap for reasons that were also discussed in one of the previous articles, so that the file format is extensible and backward compatible. The context object is responsible for performing the serialization and deserialization. We will get to it in a moment.

Note that the Serializable class could be an abstract class which inherits QObject. All concrete classes could then inherit it and implement the serialization methods. However, in this case it wouldn't be possible to add serialization capabilities to existing subclasses of QObject, for example widgets. Using a separate interface gives us more flexibility. Although multiple inheritance in C++ is a very complex subject, it's very common in most object oriented languages for a class to inherit behavior and implementation from a single base class, and implement a number of additional interfaces.

An incomplete example of a serializable class containing a pointer might look like this:

class Shape : public QObject, public Serializable
{
    Q_OBJECT
public:
    void serialize( QVariantMap& data, SerializationContext* context ) const
    {
        data[ "Name" ] << m_name;
        data[ "Other" ] = context->serialize( m_other );
    }

    void deserialize( const QVariantMap& data, SerializationContext* context )
    {
        data[ "Name" ] >> m_name;
        m_other = context->deserialize<Shape>( data[ "Other" ] );
    }

private:
    QString m_name;
    Shape* m_other;
};

The serialize method of the SerializationContext first checks if the given object was already serialized. If not, it appends it to the internal list of objects and calls the serialize method on this object to store its data in a QVariantMap. Then it returns a handle to the object, which is a QVariant. Internally it contains an integer value identifying the object in the given context.

The deserialize method checks if the object with the given handle was already deserialized. If not, it creates a new instance of the appropriate class using the object factory and calls the deserialize method. Note that the object is not necessarily a Shape; it might actually be a subclass of it like Square or Circle.

Note that the context doesn't actually read or write any data from/to a stream. Instead, it stores a list of records, which include the pointer to the object, its class name and serialized data. So the handle is simply the position of the object in the list. The entire context can be written into the stream once all objects are serialized. Conversely, when deserializing, the context is first read from the stream, and then individual objects are deserialized.

We may serialize as many objects as we need using the same context, but we need to store the handles in the stream along with the context data, because we will need them when deserializing. Alternatively, we may serialize an entire hierarchy of objects by serializing the "root" object and ensuring that all children are serialized recursively:

QDataStream stream;
SerializationContext contex;

context.serialize( root );

stream << context;

In that case we don't need to store the handle, because we know that the handle of the first serialized object is always integer zero (not to be confused with invalid variant, which represents a NULL pointer):

QDataStream stream;
SerializationContext contex;

stream >> context;

Shape* root = context.deserialize<Shape>( QVariant::fromValue<int>( 0 ) );

So what does the SerializationContext class look like? This is an incomplete definition:

class SerializationContext
{
public:
    template<typename T>
    QVariant serialize( T* ptr );

    template<typename T>
    T* deserialize( const QVariant& handle );

    friend QDataStream& operator <<( QDataStream& stream, const SerializationContext& context );
    friend QDataStream& operator >>( QDataStream& stream, SerializationContext& context );

private:
    struct Record
    {
        QObject* m_object;
        QByteArray m_type;
        QVariantMap m_data;
    };

private:
    QList<Record> m_records;
    QHash<QObject*, int> m_map;
};

The serialize and deserialize methods are discussed below. The shift operators make it possible to read and write the entire context from/to the stream. The list of records stores information about objects, including their type and data. The map is optional; it simply makes lookup slightly faster for a large number of objects.

You can notice that both the serialize and deserialize methods are templates. Why not simply cast everything to void*? Also why the record and the map stores a QObject*, instead of a void*?

This is because of how multiple inheritance works in C++. Let's assume that you have a pointer to a Shape object. When you cast it to QObject*, and to Serializable*, you will receive two different pointers, that may be different from the original one. That's because in memory, the Shape object consists of a QObject, followed by Serializable, so an offset must be added or subtracted to convert one pointer to another.

You can safely cast pointers up and down the hierarchy of classes using the static_cast operator, and the compiler will ensure behid the scenes that the pointers are adjusted accordingly. But when you cast something to void*, you lose all the information, so casting it back to some other pointer may produce wrong results!

Let's take a look at the serialize method:

template<typename T>
QVariant SerializationContext::serialize( T* ptr )
{
    if ( ptr == NULL )
        return QVariant();

    QObject* object = static_cast<QObject*>( ptr );

    QHash<QObject*, int>::iterator it = m_map.find( object );

    if ( it != m_map.end() )
        return QVariant( it.value() );

    int index = m_records.count();

    Record record;
    record.m_object = object;
    record.m_type = object->metaObject()->className();
    m_records.append( record );

    m_map.insert( object, index );

    Serializable* serializable = static_cast<Serializable*>( ptr );

    QVariantMap data;
    serializable->serialize( data, this );

    m_records[ index ].m_data = data;

    return QVariant( index );
}

Notice how the pointer is explicitly casted to QObject*, and later it's casted to Serializable*? It's not possible to cast a QObject* to Serializable*, because they are unrelated classes, and forcing the cast by using reinterpret_cast or casting to void* would certainly crash the application. This is even more apparent in the deserialize method:

template<typename T>
T* SerializationContext::deserialize( const QVariant& handle )
{
    if ( !handle.isValid() )
        return NULL;

    int index = handle.toInt();

    Record& record = m_records[ index ];

    if ( record.m_object != NULL )
        return static_cast<T*>( record.m_object );

    QObject* object = ObjectFactory::createObject( record.m_type );

    record.m_object = object;

    m_map.insert( object, index );

    T* ptr = static_cast<T*>( object );
    Serializable* serializable = static_cast<Serializable*>( ptr );

    serializable->deserialize( record.m_data, this );

    return ptr;
}

Here the QObject* is first casted "up" to the actual type, T*, and then "down" to Serializable*. It doesn't matter if T is a Shape and the object is actually a Square or Circle, because all subclasses of Shape have the same layout of base classes.

Other than that, the code is quite straightforward, though there is another gotcha when dealing with circular references between objects. The record must be appended to the list before the serialize method is called on the object. This way, if the same pointer is encountered while serializing the object, it is not serialized again, which would lead to infinite recursion.

Object factory

Submitted by mimec on 2012-07-26

Before I continue the series about data serialization in Qt, there is one important related topic than needs to be mentioned first: the ability to dynamically create an object based on a class name.

Let's assume that we want to serialize a list of shapes. The Shape is an abstract class and actual objects stored in the list have various derived classes: Rectangle, Circle, etc. During serialization, we can save the class name and the object data for each item. During deserialization, we need to be able to create an instance of an appropriate class. This is where an object factory becomes necessary. In languages like C# or Java, which support reflection, it is possible to instantiate a class given as a string with a few lines of code. But in C++ there is no such mechanism.

The simple solution is to create a single function with a large switch (or series of ifs) that creates the object of an appropriate type. It's not particularly elegant and it breaks the object oriented design, but in many cases it's acceptable. However, when you have lots of classes that are scattered through different parts of the application, it may become hard to manage. And if the application has external modules or dynamically loaded plug-ins, it becomes even more difficult.

A much more elegant solution is when the factory is abstract and it doesn't know anything about the objects it creates. Instead, classes that can be instantiated using the factory have to be registered using some type of internal map. This way, each module or plug-in can independently register its own set of classes.

Qt has two mechanism that can be useful for creating such factories. They may seem similar, but in fact there are major differences.

QMetaType
The construct() method can be used to create an instance of any built-in type, or a custom type for which the Q_DECLARE_METATYPE macro was specified. This is what a QVariant does internally to wrap an custom type. However, this mechanism is designed to be used for value types, i.e. classes that have default constructors and copy constructors. It doesn't make any sense for objects of abstract classes, which are almost always passed by pointer, and often have their copy constructors disabled.
QMetaObject
The newInstance() method can be used to create an instance of any class that inherits QObject. The only condition is that the constructor needs to be explicitly declared with the Q_INVOKABLE modifier. This works fine with polymorphic objects, because QObject is often used as a base class for various abstract hierarchies of classes (for example widgets). Note that since Qt 4 it is not possible to retrieve a QMetaObject based on the class name without some additional work.

It is very easy to create an object factory which relies on QMetaObject. An example can be seen here. However, this solution has a few disadvantages:

  • The constructor needs to be explicitly declared with Q_INVOKABLE in order to be accessible from the QMetaObject.
  • There is no compile time check whether the appropriate constructor exists and is accessible, or whether parameter types are correct. You will only get a runtime warning when you actually try to create the instance and NULL will be returned instead.
  • Subclassing QObject adds some memory footprint to each object instance. Also dynamic method calls used by QMetaObject have some overhead as run-time type checking has to be performed.

However it's not difficult to create a custom factory for classes of any type, that doesn't have these limitations. An example of a custom factory that works for any class that inherits QObject can be seen below:

class ObjectFactory
{
public:
    template<typename T>
    static void registerClass()
    {
        constructors().insert( T::staticMetaObject.className(), &constructorHelper<T> );
    }

    static QObject* createObject( const QByteArray& className, QObject* parent = NULL )
    {
        Constructor constructor = constructors().value( className );
        if ( constructor == NULL )
            return NULL;
        return (*constructor)( parent );
    }

private:
    typedef QObject* (*Constructor)( QObject* parent );

    template<typename T>
    static QObject* constructorHelper( QObject* parent )
    {
        return new T( parent );
    }

    static QHash<QByteArray, Constructor>& constructors()
    {
        static QHash<QByteArray, Constructor> instance;
        return instance;
    }
};

With this approach, there is no need to declare the constructor with Q_INVOKABLE. Also, if no appropriate constructor is found, a compile-time error will be reported in the constructorHelper() method as soon as the class is registered. This code is very easy to use:

ObjectFactory::registerClass<Foo>();

// ...

QObject* foo = ObjectFactory::createObject( "Foo" );

It is also easy to modify this code so that it works for custom abstract class hierarchies that do not inherit QObject. For example, instead of using the class name retrieved from the QMetaObject as a key, it can use a key of any type that is passed to the registerClass() method or retrieved automatically from a static class member. Also a different set of parameters can be passed to the constructor depending on the needs.