The problem with any study/experiment is the preconceptions inherent in its design.

McGarrigle and Donaldson showed the inherent design flaw in Piaget’s experiments re the preservation of number in the Concrete Operational stage of development by introducing ‘Naughty Teddy’ into the design and found that children had a much better grasp of things than Piaget’s original experimental design had led him to believe.

So, yes, IQ tests can be (are) flawed and …short of some discovery (that I am not expecting any time soon, if ever) that gives us the ability to state categorically and without error that we have unequivocally mapped out all aspects of intelligence for good… always will be.

It is the nature of human beings to be imperfect (indeed, by definition, to be otherwise would be to be unhuman), so our knowledge, and ways of obtaining it, will always be lacking.

Moreover, progress is not linear … and we often take several steps backwards before we take one forwards.

But, when we do take steps forwards, they can also, like the moon-landings, be a giant leap forwards … so, who knows what the Future will bring? There have certainly been some very great advances made in the field of neuro of late — many in the ‘of late’ sense that ideas I was aware of as old thirty years ago already have been presented as ‘new and exciting’ in the last ten … which exasperates me, because they’re not new … but new technologies and more refined understanding have allowed us to make serious advances in the interim and some of the new data is new and valuable.

We don’t even know what intelligence is — hence our continuing inability to create a General Artificial Intelligence of the kind that was ‘just around the corner now’ in the 1950s, 1960s, 1970s, 1980s … you get the idea. And, even allowing for the recent recognition that there may well be something to Gardner’s idea of multiple intelligences (albeit not necessarily his specific model), our measures of it (or them) will indeed be severely lacking for some time to come (if not, as I previously suggested, forever).

Granted, I agree with de Bono’s take on the matter … that modern (Western) education systems are no more than funnelling the chaotic crowd on one side of an archway and then congratulating themselves that, on the other side of it, the same people exit in single-file instead, proving no more than that, if you march people through a narrow archway, they emerge in single file and, if your measure of success is that they should do so then you will inevitably see your approach as successful. But that doesn’t invalidate the principle of looking for ways to measure inteligence (in all its forms) and refining them as time goes by and new idea, techniques and data come to light. And, for all it’s inadequacies … and for all that it may well be a self-fulfilling prophecy, as it were … in today’s world at least (in Western societies and anywhere else adhering to the ideological perspectives thereof), IQ does correlate significantly with future success in Life.

IQ may not be the only measure of intelligence, but that doesn’t mean it isn’t a measure of intelligence or that the kind of intelligence it measures is somehow invalidated by not being a form of General A̶r̶t̶i̶f̶i̶c̶i̶a̶l̶ Intelligence that encompasses all intelligence. Imperfect it may be, invalid it is not, however (not until some time in the Future, when we find that to be conclusively the case anyway)

And this is, as I have opined here, the nature of all human endeavour. So, my rebuttal of your observation was not so much specifically related to IQ per se but more to of the idea that Psychology as a discipline/field is static and does not evolve. It used to be presented in very top-down manner for a while and we were encouraged to view ‘laboratory conditions’ as the epitome of unbiased observation, but that was really a matter of the kind of thinking prevalent throughout Western societies at the time and not unique to Psychology. Post 1960s, a lot more thinking was applied to wondering about how we go about thinking and then self-referentially applied to the field itself … resulting in ethnomethodological and ethological approaches being taken as well. Furthermore, the fact that one specific approach (laboratory conditions) was shown to be inadequate on its own does not in invalidate the approach (that of isolating, as far as possible, the relevant variables from the irrelevant); ethological approaches are no less strict about that, they just recognise that physical laboratories do not necessarily return valid results any more than did Piaget’s experiments — Naughty Teddy may have highlighted a flaw in the design but did not overturn the validity of the experiment itself, simply the conditions under which it was carried out.

So … there is more than one kind of intelligence (or at least more than one aspect to it at least, not just IQ) … and there is more than one kind of approach to Psychology, is my argument — it’s top-down and bottom-up simultaneously.