Saturday, January 15, 2011

Moving...

Trying to post C++ code on this blog is a total PITA, especially the template ridden kind that this blog is meant to specialize in. The problem being that I can't just paste C++ inside of 'pre' tags and expect it to show up right, even AFTER editing all the less than symbols (which actually takes me a while to get right because I'm constantly missing them).

I did find a way to make syntax highlighting work here, but in the process of doing that I found a simply better alternative to blogspot: wordpress. I can paste my code directly into wordpress and it shows up right. No messing around.

So, Crazy Eddie's Crazy C++ is now going to be located at http://crazycpp.wordpress.com/

New posts will be made at that location, not here. I'll set it up so you can subscribe via email if you desire.

Thursday, January 6, 2011

Quest for sane signals in Qt - step 3 (reaching the goal)

The goal for this article series was to create a method to connect any function object to a Qt signal using syntax similar to:

connect_static(object, signal, function);

Well, it turns out that this exact syntax is not going to work but the result of this article is how we can get our type checking back and connect to a Qt signal using this syntax:

connect_static(SIGINFO(object, signame, (param1,param2)), function)

Remember that 'signal' above was "signame(param1,param2)" so the main difference here is that there's an extra comma and a macro similar to "SIGNAL" in Qt. I'll explain why this is necessary.

Further, the only way I was able to come up with this syntax was to use the 'decltype' keyword that only exists in the new C++ standard. If I find a way to get rid of this requirement I'll post but for now, most of the compilers used today have versions that support this keyword and have for quite a while now. If you don't have one of these you can still use this technique, it just won't look quite as nice.

The first thing that is needed here is a component that can hold enough information for us to connect to the signal and to verify the signatures match. Because Qt casts to void* and we then cast back, the signature of the converter has to exactly match the signal.

template < typename Class, typename MemPtr >
struct siginfo
{
Class * object;
char const* signame;

typedef typename boost::function_types::parameter_types<MemPtr>::type memptr_params;
typedef typename boost::mpl::pop_front<memptr_params>::type sig_params;
typedef typename boost::mpl::push_front<sig_params,void>::type sig_types;

typedef typename boost::function_types::function_type<sig_types>::type signature;
};

All this object does is hold a pointer to the object we want to connect to, the signal name (to be fetched with Qt's SIGNAL() macro-see below) and derives the signal signature from the signature of the member pointer. We have to remove the first parameter of that signature because the member pointer "function" has the extra 'this' as its first parameter (this is how function_types defines behavior wrt member functions).

Next we need a utility function to create this object:

template < typename Class, typename MemPtr >
siginfo<Class, MemPtr> make_siginfo(Class * object, MemPtr , char const* signame)
{
siginfo<Class,MemPtr> retval = {object, signame};
return retval;
}

Nothing particularly interesting there. This function will be used by our macros because attempting to instantiate the class directly wouldn't be as easy, nor possibly even possible, otherwise.

The next thing we need is a way to get the member function from the supplied arguments. Because Qt's signals are functions (I gather there are cases when they are not, such cases won't work here) created by the MOC, we can grab the address and treat them just like normal member functions. On initial evaluation we might assume that simply doing a 'Class::member' thing is enough; it is not. Qt allows signals to be overloaded and we want to retain this ability; if we tried to just do the naive member specification we would get ambiguous reference errors if the signal is overloaded. We need to perform a cast (which is why instantiating siginfo directly in our macro may not be possible by the standard):

#define FUNPTR(Object, FunctionName, Params) \
[Object]() -> void ( std::remove_pointer<decltype(Object)>::type:: *) Params \
{ \
struct helper : std::remove_pointer<decltype(Object)>::type \
{ \
typedef std::remove_pointer<decltype(Object)>::type obj_t; \
typedef void (obj_t::*ftype) Params; \
static ftype fetch() { return & helper :: FunctionName ; } \
}; \
return static_cast<decltype(helper::fetch())>(nullptr); \
}()

The first thing you'll note when looking at this definition is that it seems absurd, and it is but it's also absolutely necessary if we want to keep type safety. We could just cast nullptr to the appropriate type, because we've constructed that type from the parameters of the macro, but then user of the finished product could attempt connecting to a signal that doesn't exist, and that's something that we wished to avoid. The other issue here is that we can't just grab the address of the signal (even though it IS a function) because the preprocessor symbol 'signals' equates to 'protected', thus all Qt signals are protected. The trickery here forces the compiler to verify that the function exists with that signature so that the user can't mess up in the manner we're trying to fix.

We're not breaking the protected interface because we don't actually return the real pointer. The effect of returning the nullptr cast to the appropriate type allows us to do the examination of that type in the functions discussed above because we can pass it off as a variable to make_siginfo. The effect of doing so from this function cases misuse to result in something like this error:

main.cpp(19): error C2298: 'return' : illegal operation on pointer to member function expression

You could improve on this by using static_assert.

The other thing to note is that this is why there has to be a comma in the signature. There's no way to turn 'f(params)' into the two bits 'f' and '(params)' and as you can see, they have to be used separately in this line. We can, on the other hand, put the two bits together to use SIGNAL to create our siginfo:

#define SIGINFO(Object, FunctionName, Params) \
make_siginfo(Object \
, FUNPTR(Object,FunctionName,Params) \
, SIGNAL(FunctionName Params))

Concatenation would be preferred to turn "fun_name" and "(par,ams)" into "fun_name(par,ams)" but one, commonly used compiler, complains about it. Luckily Qt will still respond correctly if there's the space between function name and parameter list.

Very little remains now. Remember our sig_convert class from the last article? We simply need a constructor to build it:

template < typename Signal >
struct sig_convert
{
template < SigInfo, typename Fun >
sig_convert(SigInfo info, Fun f)
// : QObject(info.object) <- what we'd do in the real version.
{
static_assert( std::is_same<Signature, typename SigInfo::signature>::value
, "Signatures do not exactly match.");

// set the fwd function and attach the signal (see pt1 and pt2)
}
};

We could just make the SigInfo the parameter for sig_convert, but doing it this way allows similar signals to share code.

The very last bit is to make 'connect_static'. Cool thing here is that it doesn't have to be a MACRO:

template < typename SigInfo, typename Function >
sig_convert<typename SigInfo::signature> connect_static(SigInfo info, Function f)
{
return sig_convert<typename SigInfo::signature>(info,f);
}

In our real version we would want to return a pointer and the constructor for sig_convert would take our signal object as its parent in order to tie its lifetime with that object. This is how usual use would warrant us to do it.

Of course, because connect_static is supplying the same signature that the sig_convert then checks against, we know it'll always pass. It might seem logical to then remove that check. I'm not going to though because I envision it being possible that I'd want to use sig_convert directly when the assumptions around connect_static are not valid for some reason.

As was mentioned earlier, these particular bits of the solution only work because and when the signal you're connecting to was made by MOC and thus exists as a member function of the class you're connecting to. If you're working with something unusual that doesn't meet this requirement you can still use the bits from pt1 and pt2 but you won't get back the type safety; you'll have to do it exactly by hand. Under such conditions it may be better to simply use the Qt slot mechanism, which does the dynamic stuff pretty well and should be safer, but you could still use these objects if you're careful.

Also, the versions in these articles have all assumed a void returning slot. The last entry of this series will discuss how to add the ability to connect to non-void signals. The issue in that case isn't going to be so much implementing the invoker, because that's easy, it will be in the analysis of a protected function when you don't have protected access.

Tuesday, January 4, 2011

Quest for sane signals in Qt - step 2 (reconstructing parameters)

The second part of this problem involves turning our void** parameter array into the correct parameters and invoking our function.

At this point we'll assume that the type information exists in the form of a function signature, which, being a type, can be analyzed through metaprogramming; a library exists to do this within boost called FunctionTypes. This assumption is safe to make since this information is going to either be provided by the user or through the function data available from the signature; this mechanism will be discussed later. For now, we'll start with a function signature and devise a way to call some function F compatible with that signature using a mechanism similar to what qt_metacall is going to be forwarding. Here is the driver code for that problem:

#include <string>
#include <iostream>
#include <boost/bind.hpp>

struct converter
{
virtual ~converter() {}

virtual void call(void ** args) const = 0;

};

void call_converter(converter const& cv)
{
int a0 = 42;
double a1 = 66.6;
std::string a3 = "Hello World!";

void* params[] = { &a0, &a1, &a3 };
cv.call(params);
}

void fun(std::string const& str)
{
std::cout << str << std::endl;
}

template < typename Signature >
struct sig_converter
{
...described below...
};

int main()
{
sig_converter<void(int,double,std::string)> cvt(boost::bind(&fun, _3));
call_converter(cvt);
}

None of that code is part of the solution, it is simply code meant to drive the experiment used to solve one part of that solution. The "converter" is the placeholder for the QObject and "call" is a placeholder for qt_metacall (though in the eventual solution it won't be virtual, but will instead call a virtual invoker). In our main function we want to make sure we can attach to our "signal", observing or ignoring whatever parameters are necessary to make the call.

The thing we need to do now, is fill the body of sig_converter. A good initial test will simply forward to a "function" (of the std or boost kind). We can later simply use this solution (since a signal IS a function) or devise a more generic one that has as little indirection overhead as possible. So, at this stage our object looks like so:

#include <boost/function.hpp>

template < typename Signature >
struct sig_converter
{
sig_converter(boost::function<Signature> f) : fwd(f) {}

void call(void ** params)
{
...
}

private:
boost::function<Signature> fwd;
};

That being all the straight-forward stuff that pretty much everyone is familiar with, it's time to start with the metaprogramming needed to turn 'params' into the right types, in the right order, and use them to invoke fwd(). The first thing that needs to happen is to turn the 'Signature' parameter into an mpl sequence that we can iterate:

typedef typename boost::function_types::parameter_types<Signature>::type params_t;

Next piece needed is an invoker that uses this sequence of types to call 'fwd'. As with usual with metaprogramming, the iteration to make this happen will be recursion:

template < Signature >
struct sig_converter
{
...
typedef typename boost::function_types::parameter_types<Signature>::type params_t;

template < typename FromIter = typename boost::begin<params_t>::type
, typename ToIter = typename boost::end <params_t>::type >
struct invoker
{
template < typename Args >
static void apply( boost::function<Signature> const& f
, void ** params
, Args const& args )
{
// the type of the current parameter.
typedef typename boost::mpl::deref<FromIter>::type arg_type;
// an iterator to the next parameter.
typedef typename boost::mpl::next<FromIter> ::type next_iter;

// get the value...
arg_type val = *reinterpret_cast<arg_type*>(*params++);

// Call the next iteration, having pushed our current parameter onto the param "stack"
invoker<next_iter, ToIter>::apply(f,params,boost::fusion::push_back(args,val));
}
};
};

// the end case, when we've reached the end of our iteration...
template < typename Signature >
template < typename IterTo >
struct sig_converter::invoker<IterTo,IterTo> // specialize for iterators being the same...
{
template < typename Args >
static void apply( boost::function<Signature> const& f
, void ** // unused - our arguments have been converted
, Args const& args)
{
// black magic created by boost to invoke a function with a 'fusion' sequence
boost::fusion::invoke(f,args);
}
};

And THAT is the worse of the black magic (minus what's UNDER these boost types). The way this works is that each version of sig_converter will instantiate it's internal 'invoker' with iterators to the sequence created from its Signature parameter. It, in turn will instantiate 'invoker' with the next iterator, and the next, and the next, until the end is reached. This last instantiation invokes the function. Along the way we're given type information for the current parameter, we convert it, and we iterate to the next parameter to pass on to the next instantiation.

So, that all being done, the last thing we need to do is finish up by filling in 'call' with one, simple line of code:

invoker<>::template apply<boost::fusion::nil>(fwd,params,boost::fusion::nil());

What we've done here is instantiate the beginning of the iteration by using the default arguments, and passing in an empty stack for the arguments to be pushed onto.

A lot of the ideas in this code come from the documentation for boost::function_types, namely the interpreter example. If you want to learn more, there's the boost documentation and the book, "C++ Template Metaprogramming" by Abrahams and and Gurtovoy. This book is an essential part for today's C++ developer's bookshelf.

Quest for sane signals in Qt - step 1 (hand coding a Q_OBJECT)

If it wasn't for the particular implementation of signals that Qt has, it would be a quite wonderful library. So much about it has been made very easy in comparison to most other UI libraries for C++. I really enjoy working with the library except for its signal architecture. Don't get me wrong, the signal/slot idea is a vast improvement over every other thing out there; what I don't like is Qt's way of doing it.

The way I see it, there's two fundamental problems with Qt's approach:

1) If I wanted Python I'd use it. C++ is a strongly typed language and this feature is extremely powerful in that it provides much opportunity to catch bugs before they ever happen. The designers of Qt apparently don't like this feature though and spent a lot of time and effort making QObjects dynamic types. You can connect slots to signals without knowing whether or not either one is actually real. The problem with this is that you don't know that you've spelled something wrong until you run the program and happen upon the problem; this can be damn difficult to debug in some cases too, which is one good reason why I'm NOT using Python.

2) Qt slots can only exist in Q_OBJECT classes. This has more implications than simply being bound to QObject because Q_OBJECT classes must be processed by the moc. The moc, in turn, implements an incomplete C++ preprocessor and is not compatible with template classes. Since only slots can be connected to signals you're stuck having to implement a hand-coded Q_OBJECT for each one. This means you can't auto-generate slot classes. Furthermore, since slots have to be complete objects and not just functors as with other C++ implementations of the signal/slot architecture, you can't connect to binds or lambda expressions.

The first of these is frustrating and slows me down, but the second is actually fairly debilitating compared to other signal/slot implementations. The really great thing about implementations like boost::signals (or signals2 - even better) is that you can connect signals to objects that don't even know anything about signals or slots. This means that your model and data classes don't have to be bound to this kind of use and you don't need to hand-code in-between objects to connect them (as you do in Qt). In fact, these mechanisms allow you to even connect to free functions, giving you even more possibilities for expression without creating new, pointless coupling.

However, like I said, the rest of the Qt library is great and since the developers actually cared about commercial development on Windows operating systems, it makes the single best C++ option that exists for UI development on that system. Even though it is "cross-platform" it is still the best for native development that you never intend to use anywhere else. The simple fact is that native options, such as MFC, are horrible to use. The GTKmm library uses something similar to boost::signals but unfortunately doesn't provide accessibility on Windows, unlike Qt. This feature is absolutely needed by anyone who intends to auto-drive their UI through testing scripts on that system since all such test suites use the accessibility framework to do their thing.

So, given that I hate the Qt signal system but need to use Qt, my quest is to find a method to sanitize Qt's signals. I don't intend to "fix" them, they'll still be there and being used, but I do intend to fix the interface to them. I want static typing back and I want to be able to connect to any function like interface, just like boost::signals. Thus the real goal here is to turn a Qt signal into a boost signal. We don't have to go the other way around because Qt slots are just functions (with some junk about them in some meta-information) so we can connect them to any boost signal freely.

It would be nice to be able to do this on the fly. So I'm looking for a use case something like so:


connect_static(qt_object, qt_signal, [](args){ ... });


Of course the problem I have at this point is that whatever does the translation MUST be a Q_OBJECT. Of course it can't be because I need to be able to generate the translation functor from the parameter description. The only ways to do that is of course to extend the moc in some way, use the preprocessor, or (my preferred choice) template meta-programming. The first is simply not practical and the later two are incompatible with the moc. Thus, what we need to do here is create a Q_OBJECT without using the moc. Today I pulled this off and it actually turned out to be a LOT easier than looking at the source code and various web sites would indicate.

The first thing I did was to examine the source code that is generated by the moc and try to figure it out. I actually made it pretty far but it was obvious that creating preprocessor macros (didn't think TMP would do it) to mimic the MOC would be damn daunting. Here's one website that describes the Qt metamodel:

Qt Internals & Reversing

I also tried to ask on stackoverflow and qtcentre for help to see if all of this was even necessary. The former generated few responses indicating the MOC and any of my ideas to do this where incompatible. The latter just generated a flame war in response to my frustration with a whole lot of very silly answers, such as telling me to put a particular class definition in "#ifdef" blocks (not a good place to go to for help, lots of really poor advice being handed out at that site). I did, however, eventually yank out a link from one individual that actually ended up holding what I believe is the key to getting this accomplished:

Dynamic Signals and Slots

That site alone does not contain exactly what I need, but pulling various bits from it and adding bits I learned from looking at what qt_metacall does (stepped all the way through it quite a few times trying to figure out why my slots where not being called) resulted in the ability to create a QObject derived class that responds to any signal attached to it and could then be extended to forward to a subclass, which would not have to be a Q_OBJECT:



struct listener2 : public QObject
{
int qt_metacall(QMetaObject::Call call, int id,void** args)
{
id = QObject::qt_metacall(call,id,args);
if (id == -1 || call != QMetaObject::InvokeMetaMethod)
return id;

assert(id == 42);

std::cout << "Funky slot!!" << std::endl;
return -1;
}

void make_connection(QObject * obj, char const* signal)
{
QByteArray sig = QMetaObject::normalizedSignature(signal);
int sigid = obj->metaObject()->indexOfSignal(sig);

QMetaObject::connect(obj, sigid, this, QObject::metaObject()->methodCount() + 42);
}
};


This object must be connected with "make_connection" rather than with connect, because otherwise the system breaks down when it tries to find the slot and doesn't, but besides that it is almost perfect. The args void* array actually points at the arguments generated by the signal and then must be reinterpret_cast to the appropriate thing (see the .moc output of any of your classes).

The key here is to find the id of the signal you want to connect to and use QMetaObject::connect instead, supplying an id you can recognize on the other side. You must add the methodCount() of your base so that it can take care of existing slots. In the qt_metacall function you first let the base attempt to take care of it. This will subtract from the input id resulting in id's that YOUR derived object is meant to take care of. The assert is unnecessary but adds some semblance of safety. The return of -1 in standard Qt speak for, "I've handled this call."

This is a big step. A subclass to this thing could take the void** data and then use TMP to generate the correct casts and invoke a boost::signal. Implementing that will be the subject of "step 2" followed by attempts to regain static typing and safety.