Dear Editor,
The debate about the poll in the SN, (for example ‘Further questions about Bisram poll’ by Dr Anand Parsaud (13.5.08), can only be a healthy development for Guyana. Pollsters should be grilled about their organisation and methodology, particularly if the subject-matter is attitude surveys with political implications. At the same time, users should remain respectful until they have the evidence to conclude otherwise. The tenor of some of the exchanges has been distinctly offensive and unhelpful.
In, for example, the UK, the results of opinion polls conducted by such highly reputable professional organisations as the National Opinion Poll and YouGov are first published in newspapers of all political complexions. As you would expect in a medium designed primarily for lay readers, there is only a brief reference to the abstruse intricacies of methodology. A full description is lodged in the company’s websites.
Moreover, a few newspapers such as the UK Guardian commission their own opinion polls. But as the contracted organizations have a well-established professional track record in market research, their results are universally respected.
I have to agree that until there is greater transparency about Mr Bisram’s organisation and methodology, there will continue to be scepticism of the statistical results, even if his surveys are in fact completely above board.
Concerning his organisation, users are entitled to know mainly:
a) Whether his company is legally registered with the authorities;
b) What are its terms of reference;
c) Its turnover;
d) The size (but not names) of its professional and field force and the training administered;
e) Its professional activities.
On methodology, some of the key questions would seem to be:
a) The questionnaire used, question content (in terms of concepts, definitions and nomenclatures) and question order; these can seriously influence the results;
b) The data collection method – whether by personal interview or postal means; again this could affect response error, non-response bias and the validity of any imputation and weighting algorithms subsequently used;
c) The sample design and selection:
Unless random samples are used the stochastic error quoted by Mr Bisram is not valid; pollsters generally tend to use quota samples, which do allow in some exceptional circumstances for the assignment of sampling errors but not as a rule; (I am afraid that the relative error of a fixed 4% quoted is incorrect);
The spatial distribution of the sample; for example, over-representation of Georgetown could bias the results one way and over-sampling of say West Coast Demerara could bias the results in the opposite direction;
How were the within-stratum sample members (the ultimate sampling units) selected; was that left completely to the subjectivity of the data collector, in which case what training was given?
d) Editing and validation procedures and other aspects of quality control.
May I reiterate, I do not seek to impugn the reliability and the integrity of Mr Bisram’s work. I simply do not know it. Moreover, while the newspaper is an appropriate, indeed customary, forum for the release of numerical results plus some very brief, general information about reliability, it is not the place for technical details of methodology. But my main point is that unless there is greater transparency on all the aspects mentioned above, not just by Mr Bisram but any survey organisation as part of its standard dissemination practice, perhaps through the company’s website, there will continue to be searching and occasionally hostile questions.
Yours faithfully
D. Ramprakash
