Blog

Writing

Submitted by mimec on 2013-01-12

I want to write. Well, of course, I do; but I don't mean programs and technical documentation, but novels. My New Years resolution is to finish the book that I started some time ago and get it published. Why this sudden change of mind? Just a few months ago I wanted to start a business based on WebIssues. I even managed to briefly bring the attention of the management of the company I work for to it. But their idea of investing very little in order to hopefully get some profit wouldn't make too much sense. My own vision wasn't downright rejected, but considering all the political aspects that rule a corporation like this, and my complete lack of influence on these things, I can't realistically expect that this is ever going to happen.

Obviously, making a living from writing is an even more insane idea. It's a very demanding market, and in Poland also quite a narrow one. It also requires a lot of pure luck, probably even more than running a successful business. Not to mention that writing a novel requires huge amounts of time. But the real problem is that I'm really starting to hate programming. Commercial or open source, it's tedious, repetitive and rarely creative. And writing is not a new idea. I wrote some stories as a child. In high school I started writing a book with two friends; it didn't last long, but it was a lot of fun. But this time it's different, because I already have most of the plot in my mind, so the ideas are there waiting to be put on paper.

I already mentioned the novel I'm writing once or twice, but perhaps this time I will shed some more light on it. The idea came to my mind in summer 2011, while I was reading Neal Stephenson's Snow Crash, but it was also influenced by Lev Grossman's The Magicians which I read shortly before. It's basically a cyberpunk story, taking place largely in two different virtual worlds, but it also has some elements of contemporary fantasy and techno-thriller. The main characters are a few students of a school for young hackers, which is called the Academy of Magic, because in a virtual world, the boundaries between hacking and magic are blurred for the uninitiated. As my younger brother described it, when I told him about the novel today, it's like a "rolled pancake" :). I admit that mixing genres is risky, but if I do it well, maybe something interesting will come out of this.

And by the way, yesterday was my son's first birthday :). I must publish some new photos soon because I haven't done that in a while.

Tags

Habits and standards

Submitted by mimec on 2012-12-16

Yesterday was the seventh anniversary of mimec.org, but I will not elaborate on that. It suffices to say that the last year was very different from the previous ones. My son Adam changed from a blurry ultrasonographic image to a little boy who runs around the house. There is no time for anything. I can hardly keep up with my paid job, not to mention the open source projects, but I still managed to make four minor releases of WebIssues, an one release of Saladin (with another one pending), Fraqtive and Descend.

A few days ago I finally got a new laptop. It has a 15" Full HD display, which for some reason is very rare these days, powerful CPU and GPU and plenty of RAM. Minecraft runs at about 50 FPS at full screen with far viewing distance :). The bad news, though, is that my company run out of Windows 7 licenses, and I was forced to install Windows 8. I'm not going to rant about it, becuse enough has been said about it already. After installing the English language pack and removing the metro-garbage from the start menu, I'm getting used to it without having to change my habits too much. It's just hilarious that the now so called "desktop" applications suddenly became legacy and are only temporarily supported for backward compatibility. It reminds me of how all existing applications suddenly became "unmanaged" when .NET was created, as if they were crippled in some way. Microsoft suggested that in a few years all applications would become "managed", and finally support for those "unmanaged" ones would be dropped. Of course I don't mind .NET; it's just the kind of marketing speech that makes me laugh.

But when I saw Office 2013 with the black and white UI and icons designed for displays that support only 8 colors, it actually made me a bit upset. For a long time Office was setting the user interface design standards for a lot of Windows applications, especially regarding toolbars and menus, because the default ones always had a very plain look. Obiously I also always tried to keep up with the trends. Over ten years ago, in Grape3D, I used third party menu and toolbar classes for MFC which mimicked the flat, semi-transparent highlighting style know from Office XP. Later I wrote my own set of classes which broke out of the Office trends for a while and looked more like IE 6. But soon after that Microsoft released Office 2003 with the spectacular bright blue and orange UI which automatically changed its colors to match the Windows XP theme. Whether it looked good or not, it became a long time standard. Just take a look at version 0.9 of the WebIssues Client, or the so called "modern" Qt style which I wrote in 2008, and you will know what I mean.

The so called "ribbon" introduced in Office 2007 was something that people complained and ranted about nearly as much as the Metro UI in Windows 8, but it eventually turned out to be a very good idea. It was not just a cosmetic change, but something entirely new. Currently all my programs use a similar concept, which is available as part of the XmlUi component. At the same time the bright colors were toned down and the whole thing looked equally good with classic Windows style as with Luna and Aero. But now that I'm getting more and more used to Windows 8 and Office 2013, even the soft gradients and slightly rounded corners of XmlUi are beginning to look a bit odd. So what is the next logical step? Should we, developers, all turn to creating rectangular, black and white UI? How soon will Microsoft change its mind and what will be the next "standard"? Or perhaps it's time to stop bothering?

Fuzzy optimizations in QTransform

Submitted by mimec on 2012-12-06

Before I get to the point, just a quick note: I recently released a new version of both WebIssues and Fraqtive and I'm planning to release a new version of Saladin by the end of this year, so I'm quite busy, as usual at this time of the year. Also check out the fractal animations created using the latest version of Fraqtive. There are much more impressive deep zoom animations available, but they were created using specialized tools which use high precision numbers. Fraqtive uses SSE2 to maximize the real-time experience, so it's limited to double precision, which allows to zoom the fractal about 10^13 times. But this is enough to produce some cool effects.

Now back to the main topic. For a long time there was a strange bug in Fraqtive that I thought was impossible to fix. As those who use it know, in Fraqtive it is possible to move and zoom around the fractal using the mouse, and when you release the mouse button, the new region of the fractal is recalculated. This works by converting the start and end position to a QTransform (which is basically a 3x3 matrix defining the offset, zoom and rotation) and calculating the relative transformation. It worked fine until a certain zoom level was reached; then it started producing weird results. I always blamed the limited precision of double numbers, though the effect could be observed a few orders of magnitude before reaching the maximum valid zoom.

Recently I debugged the entire code, including the calculations performed inside QTransform class and discovered that it performs some optimizations which cause the wrong results. The documentation mentions that QTransform performs some optimizations based on the type of the matrix. For example, if the transform includes translation only, the scale and rotation components are ignored. What the documentation doesn't mention, though, is that Qt uses fuzzy compare functions (such as qFuzzyCompare) to determine the type of the matrix.

The problem with qFuzzyCompare and it's undocumented qFuzzyIsNull counterpart is that they assume that only about 12 digits are significant in a double precision floating point number. This often makes sense, because limited precision can result in some subtle differences which cause the strict comparison operator to fail. We all know that 10 / 3 is 3.33333... and that value multiplied by 3 gives 9.99999.... In mathematics, this value is equal to 10, but for a computer, these numbers are not necessarily equal.

The effect of using fuzzy comparisons is that at a zoom level higher than 12, the scale factors are considered equal to 0, so the transformation is considered non-invertible, when in fact it is. Also rotation tends to be ignored at this zoom level. The solution is not to use QTransform when high precision is required or to write custom functions which do not have these side effects. See also [#QTBUG-8557] where a similar problem is discussed.

Note that fuzzy comparisons are used not only in QTransform, but also in other classes like QMatrix4x4 and QVector3D. Some time ago I came across the a similar problem with QVector3D::normalized which caused Descend to incorrectly calculate normal vectors for the surfaces. The problem is that Descend calculates three samples that are very close to one another in order to precisely determine the normal vector. In some areas of the surface it would hit the 12 significant digit limit, so I ended up writing my own version of this function which didn't use fuzzy comparisons.

Qt and ZIP files

Submitted by mimec on 2012-11-20

This is a follow up to the series of articles about serialization in Qt that I wrote a few months ago. Sometimes it's necessary to create complex files which contain lots of various information. Serializing everything into one stream of data is not always a good idea. Also sometimes it may be a good idea to compress serialized data, because it's usually quite verbose. Instead of coming up with a custom file format, the best idea is to use the solution which is implemented in may applications and document formats, including both OpenOffice and MS Office documents; that is to wrap data in a ZIP file.

There are many advantages of using the ZIP format. We can save data in multiple smaller files which are zipped together. We can include additional files and attachments, for example JPG images. Finally, we can compress selected files to make the resulting file more compact.

There are a few libraries for Qt which handle ZIP files, including OSDaB and QuaZIP. There are also the QZipReader and QZipWriter classes which are part of Qt. They are internal classes that you can find in src/gui/text subdirectory in Qt sources. There are plans to include them in official Qt API (see QTBUG-20896), but they will not make it into Qt 5.0 and it's not clear whether they will be included in later versions. However, if the license permits, you can simply copy them into your own application (but see the notes about static linking below).

A problem for Windows developers is that all these libraries and classes depend on zlib.h. This is fine on Linux, however zlib is (unfortunately) not part of Windows API. The good news is that QtCore not only includes zlib statically when built on Windows, but it also marks its functions as DLL exports. Thanks to this, the QZipReader and QZipWriter classes, which are part of QtGui, can still work on Windows. In my applications, including Descend which uses ZIP file format for its projects, I simply include the zlib.h and zconf.h files from Qt in the source package and use them when system zlib library is not available. Then I use the following simple .pri file to include either system zlib headers or the custom ones:

contains( QT_CONFIG, system-zlib ) {
  if( unix|win32-g++* ): LIBS += -lz
  else: LIBS += zdll.lib
} else {
  INCLUDEPATH += $$PWD
}

Thanks to this, #include "zlib.h" works no matter if zlib is a system library or not. When Qt is built without system-zlib (which is usually the case on Windows), it will include all the necessary exports in QtCore, so the application will link and work correctly.

Including internal Qt classes as part of the application is potentially dangerous, because there can be conflicts between the classes provided by Qt and by the application itself, especially when the application is linked with a different version of Qt. This is even more dangerous when the application is statically linked with the Qt libraries, which I usually do in Windows release builds to prevent DLL dependencies problems.

To avoid problems, I always remove the 'Q' prefix from such classes. This way they are seen as separate entities from the ones provided by Qt. However, I encountered a strange problem with Descend. In dynamically linked debug builds it worked fine, but when compiled in static release mode, it crashed when copying text to the clipboard. At first I suspected a strange bug in Qt or the compiler itself, and the problem was hard to debug because it only happened in release builds. However, after some analysis, I discovered that QPlainTextEdit copies text into the clipboard not only as plain text, but also in HTML and ODT formats. Coincidentally, creating an ODT document is exactly what Qt uses the internal QZipWriter class for...

It finally turned out that there was an innocent looking structure in qzip.cpp called FileHeader, which I slightly modified, but I didn't rename it. A plain structure would be fine, but this one had an implicit constructor and destructor, because it contained a QByteArray member. Unfortunately the linker doesn't detects such conflicts (it would not be possible anyway, because of how C++ works), but happily overrides the symbols from Qt library with identically named symbols defined in my application.

This can lead to unexpected problems not only when copying code from Qt, but also in many other situations. After all, many libraries can include a class named FileHeader. That's why static linking must be used carefully. Qt generally uses the Q prefix everywhere, even in private classes and functions, to prevent this type of problems, but this particular one didn't have it. If you ever experience mysterious crashes in release builds, this can be one of the possible reasons.

Descend 0.2 released

Submitted by mimec on 2012-10-15

The first official version of Descend was released today. You can grab it from the Descend website which was also officially launched today. It's quite a historical moment, as Descend has been waiting for about eight years to be released. You can read a short history of this program and its predecessors here. Also this is the fourth sub-website in the mimec.org family, so I decided to add some nice links on the front page.

What's coming next? As I said recently, I have plans to start working on version 1.1 of WebIssues, but before I release the first beta version, you can also expect new releases of both Saladin and... Fraqtive. Possibly by the end of this year :).

Tags