There is a reason it is illegal to not answer the census, because unless everyone answers, the data is not an accurate representation of New Zealand (Stats New Zealand, 2013). Colmar Brunton get paid megabucks by the government to monitor the ‘mood’ of New Zealanders. They undertake market and social research for companies and the government to be able to get an idea of what we the masses are thinking. I am always automatically skeptical of surveys and this kind of research. It must be my science and psychology background, always checking methodologies never just enjoying the fun stats. I took an entire paper in my undergrad degree dedicated to research methods in Psychology, which basically ruined surveys forever. Disclaimer: I don’t hate surveys, they are a useful tool to judge how people are feeling. But I do object to calling a survey a representation of an entire nation because no matter how good your survey, they come with many methodological challenges related to reliability and validity (Taylor, 2017).
So here I am ruining it for all of you too. We had a look at their 2017 “better futures” report on how New Zelanders feel about our sustainable development goals. Overall it’s a pretty bland document, and the information available for their methodologies is very skimpy. Their 2017 and 2019 better futures reports only has a sample size of 1000 people surveyed online. (Colmar Brunton, 2017; Colmar Brunton, 2019) They call these 1000 people a “national representative”. But already their methodologies are removing groups of people from being represented. People who don’t have access to or can not navigate a computer are automatically excluded from being represented. Already you are cutting out the views of some of our most vulnerable members of society. Low socio-economic people who cant access the internet are not represented in this survey. You may find that people at the lowest end of our socioeconomic spectrum are far less concerned about supporting ethical brands and are more concerned with making ends meet.
Also, there are many New Zealanders who would not sit down and answer a survey from Colmar Brunton. Young people, busy people, people who don’t like surveys or don’t care, are all excluded from being represented. You might argue its their fault for not participating, and yes you would be correct. However, you can not then call it a true representation of New Zealand’s opinions. There are many more obstacles for gaining a representative sample from a survey such as language of instruction; If it’s only offered in english you exclude a lot of the immigrant population. Another obstacle is the interpretation of the question. People can read the same thing but come up with completely different interpretations of what it means causing the data collected to be not what was intended (Taylor, 2017).
The dirty secret of these types of surveys is that they don’t care if they are leaving out large groups of our population. They care most about the opinions of the people with power. The middle aged, middle to upper class. These are the people who are the most influential in terms of buying power and voting power so these are the people that represent New Zealand.
References.
Stats New Zealand. (2013). 2013 Census frequently asked questions. Retrieved from http://archive.stats.govt.nz/Census/2013-census/info-about-the-census/faqs.aspx
Taylor, J. (2017). Improving Reliability in Collection Condition Surveys by Utilizing Training and Decision Guides. Journal of the American Institute for Conservation, 56(2), 126–141.
Colmar Brunton. (2019). Better futures Report. Retrieved from. https://static.colmarbrunton.co.nz/wp-content/uploads/2019/05/Colmar-Brunton-Better-Futures-2019-MASTER-FINAL-REPORT.pdf
Colmar Brunton. (2017). Better futures Report. Retrieved from https://blackboard.aut.ac.nz/bbcswebdav/pid-4936829-dt-content-rid-10793722_3/orgs/CSOC_AE_AK3515/Colmar-Brunton-Better-Futures-Report-FINAL.pdf
Provocations Blog 
I don’t come from a particularly scientific background, but the semester of undergraduate papers in criminology was enough to teach me that surveys are treacherous!
The sample size of that study also struck me as troublesome. I didn’t know it had been online, but it usually is their online or conducted via landline- and only surveying people with a landline presents a very skewed sample of society!
This reminded me of the voting quiz programmes such as Vote Compass that the govt encourages us to use in order to make decisions about which party policies most closely align with our own views. I was at a former workplace just before the election, and when it seemed that my coworkers didn’t know who to vote for, and didn’t know how to find out, I suggested they try that. It was quiet, so everyone got out their phones and completed the survey. However, a fairly significant number of the questions were phrased in a way that either reduced their understanding, or relied on a fairly significant amount of prior knowledge. How inflation worked, what immigration would mean for them etc. One that stood out to me was a question about whether the govt should fund charter schools. No one knew. No one knew what a charter school was, or how it might effect New Zealand either way. And this was a tool supposedly designed to help in understanding. I wonder if surveys come up against the same issues- thoughtless answers, or just plain incorrect ones, because the questions don’t explain themselves, and people like to have an opinion, whether or not they fully understand the issue.
LikeLike