The Queensland Plan (QP) is being prepared as the state government’s 30 year vision for Queensland. While community engagement for the QP was extensive, there were significant flaws in the process.
The main flaw was that participants in the online survey of community opinions were self-selected. The views of such a ‘convenience sample’ are highly unlikely to represent those of the broader Queensland community.
Other significant flaws in the process were
- Selection of community delegates to the two summits by MPs rather than a more ‘arm’s length’ method
- At the Mackay summit, selection of the questions to be used subsequently in the online community survey was by majority vote rather than a more ‘consensus-seeking’ process
- A majority of Brisbane summit delegates represented sectional interests rather than being unaligned people drawn from the broader community
- The online community survey questions tended to confine discussion about the future within a particular implicit vision/values framework
- The wording of the survey questions proved to be so unclear that prompt answers were added to the survey after the submission period had started
- The dominant themes from the ‘non-representative’ online survey responses were developed into ‘focus areas and enablers’, which were subsequently endorsed by majority vote at the Brisbane summit where there was a majority of ‘sectional interest’ delegates.
The starting point for a truly deliberative and representative dialogue about Queensland’s future should be a statistically valid survey of the Queensland population to identify the community’s shared values.
Over the past year, the Queensland government has been preparing The Queensland Plan (QP)[see note 1], a ‘30 year vision’ for Queensland.
Work on community engagement for the QP began early in 2013. The stated objective at the outset was to produce a 20-30 year vision for Queensland through a ‘bottom up, grassroots’ community engagement process, with the Queensland government acting as the ‘sponsor and enabler’ of the process [note 2].
ENGAGEMENT PROCESS DESIGN AND MAIN ELEMENTS
A so-called ‘Collaborative Governance’ engagement model was adopted [note 3], incorporating four main phases:
1. Co-define: develop a small number of ‘focus questions’ to stimulate discussion; test and refine the questions with core stakeholders
2. Co-design: state MPs to select community representatives; MPs, community reps and council mayors work together at a summit to further refine questions
3. Co-determine: encourage Queenslanders to discuss and respond to the questions through both state-wide and electorate-specific processes; MPs, community reps, mayors and peak body reps produce regional and state-wide priorities
4. Co-deliver: develop actions for government, business and the community.
The following principles [note 4] were adopted for the community engagement:
1. Inclusive – encouraging all to be involved
2. Appreciative – recognise the value of previous planning efforts
3. Accessible – easy to participate (more than one way)
4. Sophisticated (or deliberative) – dialogue includes a range of perspectives
5. Influential – engagement will make a difference
6. Stretching but realistic – ambitious but achievable
7. Well informed – learning from research and others
8. Shared ownership – wide community acceptance of outcomes.
Main engagement elements
There were four main elements [note 5] of the community engagement process:
1. A one-day summit in Mackay attended by about 400 delegates including three community delegates from each of the 89 state electorates, together with 89 MPs and 40 mayors; six ‘focus questions’ were developed, with the answers to be used to guide development of the QP [note 6]
2. A consultation period from May to August 2013 for the community to submit responses to the six questions; 15,007 individual submissions were received, together with 582 group submissions and 79 electorate submissions from events held by MPs [note 7]
3. A two-day summit in Brisbane attended by about 626 delegates including 3 community delegates from each of the 89 electorates, together with 170 peak body delegates, 89 MPs, 30 local government representatives, 25 state department directors-general, and other special invitees; the summit considered a report on responses received during the consultation period and worked on the ‘focus areas and enablers’ proposed in that report [note 8]
4. A consultation period from December 2013 to March 2014 for community input on a ‘working draft’ of the QP, which includes a vision and a summary of the top 35 goals voted on at the Brisbane summit, grouped under nine ‘foundation areas’ [note 9].
WHAT GOOD COMMUNITY ENGAGEMENT LOOKS LIKE
Before critically assessing the QP community engagement, it is useful to consider what good community engagement looks like.
Collective experience with community engagement over several decades has established some level of agreement on what ‘good practice’ looks like. The main principles relevant to the QP are:
- Participants reflect a cross section of population – selection not open to manipulation
- Outcomes are focused on community, not self-interest – participants act as citizens
- Process is interactive and deliberative – in-depth discussion with participants and experts
- Decision making procedures are effective, preferably consensual
- Process is in the hands of an independent, skilled, flexible facilitator – to ensure all participants control the agenda and content [note 10].
Perth’s Dialogue with the City project is a good example of these principles in action, engaging the citizens of the Perth metropolitan area in a deliberative process about the future of the city.This project used a statically valid sample survey to gauge community values, together with media discussion of alternative futures and consensus-seeking decision processes [note 11].
While the QP community engagement process was very extensive and there was a genuine effort to involve the community, these good intentions were undermined by a number of deficiencies in the process as discussed in the following sections.
Insufficient context for community engagement
During the community engagement design phase, it was recognised that that the QP was not being prepared with a blank canvass and that the government was committed to certain directions [note 12]. However in practice, very little guidance was provided for participants on the policy directions to which the government was already committed (presumably this included the so-called ‘four pillar economy’ framework) [note 13].
In the Collaborative Governance model used for the QP, there is a ‘dilemma’ that needs to be solved and this is identified upfront [note 14]. The dilemma that the QP is addressing was ill-defined at the start of the engagement process, thus leaving the process without much clarity.
The CSIRO report Our Future World [note 15] was the main background document for the first summit (the Mackay summit) but does not focus specifically on Queensland. Circulation of specific issues papers at the start of the engagement process would have allowed participants to be better informed rather than just express preconceived opinions.
Confusion between Vision and Plan
The difference between a vision and a plan was not well articulated during the process. US urban planner and visioning specialist Steven Ames describes the difference this way:
‘Visioning is a process through which a community envisions the future it wants, and then plans how to achieve it. It brings people together to develop a shared image of “where” they want their community to be in the future. Once a community has envisioned where it wants to be, it can begin consciously to work toward that goal’ [note 16].
Visioning is fundamentally about identifying the values behind ‘action ideas’. In Portland (Oregon), the Portland 2030 visioning process involved:
‘asking people what they value most about their community, then what changes they would like to see today. After identifying immediate challenges facing the city, we asked people’s greater vision for Portland in 20 years. Finally, we asked, “How can we get there?” By taking community members through this simple exercise, we gained an understanding of where Portlanders agree and where there are tensions in public opinion’ [notes 17 & 18].
Community engagement for the QP seems to have blurred the distinction between a vision and a plan. In particular, Queenslanders were invited to contribute to the QP by answering the following six questions:
• In the context of living in the community, how do we move our focus from me to we?
• How do we create and foster an education culture that teaches skills and values to meet global challenges and optimise regional strengths?
• How do we empower and educate individuals, communities and institutions to embrace responsibility for an active and healthy lifestyle?
• How do we structure our economy to ensure our children inherit a resilient future?
• How do we strengthen our economic future and achieve sustainable landscapes?
• How do we attract and retain the brightest minds and ideas where they are most needed and capitalise on global opportunities? [note 19]
It is noteworthy that all these questions are ‘how’ questions, that is, they ask about ‘means’ (action ideas to achieve outcomes) rather than ‘ends’ (desired outcomes) or visions. This is putting the cart before the horse.
The questions in fact, have a vision ‘embedded’ in them, which is something like the following:
‘Queensland has become a very community-minded state with an education culture that teaches skill and values to meet global challenges and optimise regional strengths. Queenslanders are empowered and educated to adopt an active and healthy lifestyle. Our economy is structured to create a resilient future for our children, with a strong economy and sustainable landscapes. Queensland attracts the brightest minds and ideas where they are most needed and is able to capitalise on global opportunities’.
Significantly, this embedded vision is remarkably similar to that in the recently-released QP working draft [note 20]. This raises the question: was the community’s input after the Mackay summit really about how to implement a predetermined vision rather than what future the community wants for Queensland?
Inadequate resourcing of the process
At the outset, it was recognised that financial constraints and reduced timeframes created risks for the engagement process. A particular risk was relying so much on MPs for local community engagement within each electorate [note 21].
The Mackay summit was the first opportunity for community delegates to sit together and discuss Queensland’s future. In my view, it was overly ambitious to expect that these delegates would be able to deliberate meaningfully and then seek some level of agreement on six focus questions for subsequent community discussion – all in one day. It was also overly ambitious to expect most MPs to run an effective local engagement program without considerable help [note 22].
To be truly deliberative, summits of this type need several days for participants to understand issues, develop trust and work towards consensus [note 23].
Summit participants did not reflect a cross-section of the state population
There is merit in having a panel of selected community members discuss the state’s future. However, the outcomes of such discussions do not necessarily represent the views of the Queensland community as a whole.
Community delegates for the two summits were selected by each local MP. The option of inviting the public to apply to attend the Mackay summit was not chosen during the engagement design phase [note 24]. This resulted in those aged 35-54 and 55-74 being significantly over-represented and females under-represented [note 25].
For the Brisbane summit, community delegates were in the minority, with MPs, prominent Queenslanders, directors-general and senior public servants, and representatives from local government, sectional interest groups (so-called peak bodies), youth, and education delegates being the majority [note 26]. The demographic profile of this majority is very unlikely to reflect that of the state population as a whole [note 27]. Consequently, it cannot be claimed that the outcomes of summit discussions represent the aspirations of the Queensland community as a whole.
Engagement processes were tightly controlled by government
While the government was officially described as the ‘sponsor and enabler’ of the engagement process (implying a somewhat ‘hands-off’ approach), in practice this was a government-controlled process.
In the first session at the Mackay summit, delegates responded to the CSIRO report and produced questions and comments for further discussion. However, these questions and comments were not given any weight in subsequent sessions and had little or no influence on the summit process. Unfortunately, this first session seems to have created unrealised expectations about the extent of ‘undirected’ discussion of issues. For a number of delegates, the lack of follow on from this first session seriously damaged their trust in the summit process.
In the following session, delegates sat in their electorate groups, with their MP taking a leading role. This is contrary to the fundamental principle that facilitators should be independent. The active presence of the MP would almost certainly have been inhibiting for some participants, particularly for any state government employees [note 28].
Subsequent discussion at this summit was firmly directed towards producing six questions at day’s end rather than encouraging deep deliberation on what future the delegates wanted for Queensland [note 29].
As it turned out, the Mackay summit was the only possible opportunity for deliberation on the general question of what future delegates wanted. Discussions at the Brisbane summit had moved beyond that point, with delegates quickly directed on the first morning to the ‘focus areas and enablers’ prepared from the analysis of the community’s feedback on the six questions formulated at the Mackay summit [note 30]. Delegates were encouraged to ‘find a home’ for their issues within the focus areas/enablers framework, but the framework itself was not up for debate.
It could be argued that the program for each summit had to be tightly controlled because of the limited time available. However, this serves to highlight the inadequate resourcing provided for the project.
Online survey respondents not representative of Queensland population
The six questions formulated at the Mackay summit were administered in an online survey with respondents being self-selected. Such a ‘convenience sample’ is highly unlikely to to be representative of the population they are part of (in this case the whole of Queensland) [note 31].
Even if the demographic profile of the convenience sample is similar to that of the broader Queensland population, the survey results still cannot be generalised from the sample to the broader population.
Consequently, it cannot be claimed that the survey results represent the aspirations of Queenslanders [note 32].
Claims that 78,251 individuals are represented in public submissions have not been justified
It has been repeatedly stated that the submissions received ‘are representative of approximately 78,251 individuals’. In fact there were 15,007 individual submissions, 582 group submissions and 79 electorate submissions [note 33]. It is claimed that the 582 group submissions represent the views of 60,316 individuals, based on the self-reported answers to one of the survey questions [note 34].
A large proportion (48%) of group submissions was made by private businesses [note 35]. In calculating the number of individuals represented by the group submissions, has it been assumed that a submission from a private business represents the views of all its employees? If so, this is highly questionable.
Unsuitable survey questions
There is no indication that the survey questions were tested before being used in the online community survey. This would normally be done to ensure the questions make sense to participants and produce useable results.
As it turned out, the wording of the questions proved to be so unclear that ‘prompt answers’ were added to the survey after the submission period commenced [note 36]. After that time, all submissions (including the so-called ‘unprompted’ responses) were probably influenced by the prompt answers selected by the government and/or its consultants.
The QP seems to be on-track to achieve its objective of producing a 20-30 year vision through a ‘bottom-up, grassroots’ engagement process. However, claims that the December 2013 QP working draft ‘represents the aspirations of the community’ have not been justified [note 37].
The engagement process aimed to be ‘inclusive’ (encouraging all to be involved) [note 38] rather than ‘representative’ (actively ensuring participants reflect the diversity of demography, views and values of the Queensland community) [note 39]. Consequently, it has not been shown that participants in the two summits or the community survey on the six focus questions reflect the views of the Queensland population. Hence the results do not necessarily represent the shared aspirations of the Queensland community.
The community engagement process has been biased in a number of ways including the following:
1. Selection of community delegates to the two summits by MPs rather than by a more ‘arm’s length’ method
2. Majority of Brisbane summit delegates represented sectional interests rather than being unaligned people drawn from the broader community
3. At the Mackay summit, selection of questions to be used in the subsequent online community survey was by majority vote rather than a more ‘consensus-seeking’ process
4. The online community survey questions tended to confine discussion about the future within a particular implicit vision/values framework
5. Self-selection bias in the respondents to the community survey produced responses that do not necessarily reflect the aspirations of the whole community
6. The dominant themes from these ‘non-representative’ survey responses were then developed into ‘focus areas and enablers’ for discussion at the Brisbane summit where there was a majority of ‘sectional interest’ delegates.
The result of these biases was that the sectional-interest dominated Brisbane summit endorsed by majority vote a vision (‘focus areas’) developed by the government’s consultants using a somewhat subjective process drawing on the dominant themes in the ‘non-representative’ survey responses [note 40]. The QP working draft now in its public comment phase is based on these focus areas.
Unfortunately, the opportunity the QP provides for a truly deliberative and representative dialogue about Queensland’s future has not been taken up so far. However, it’s not too late to carry out a statistically valid survey of the whole Queensland population to identify the community’s shared values. These values should then be an important part of producing a vision for Queensland that truly reflects the shared aspirations of the community.
1 See official Queensland Plan website http://queenslandplan.qld.gov.au/
2 Right to Information disclosure log (RTID201 25 Oct 2013) page 13 http://www.premiers.qld.gov.au/right-to-info/disclosure-log/assets/disclosure-log-entry-rtid201.pdf
3 Pages 8 & 9 RTID201
4 Page 46 RTID201
5 Excluding the peak body briefing working held prior to the Mackay summit
6 Mackay summit data report http://queenslandplan.qld.gov.au/journey/mackay-summit/assets/mackay-summit-data-report.pdf
7 Community feedback report http://www.queenslandplan.qld.gov.au/journey/qld-response/assets/qldplan-report.pdf
8 Brisbane summit data report http://queenslandplan.qld.gov.au/journey/brisbane-summit/assets/brisbane-summit-data-report-october-2013.pdf and attachments http://queenslandplan.qld.gov.au/journey/brisbane-summit/assets/attachments-1-to-20.pdf
9 The Queensland Plan: a 30 year vision for Queensland -our working draft http://www.queenslandplan.qld.gov.au/journey/public-review-working-draft/assets/qld-plan-working-draft.pdf
10 Lyn Carson, Random Selection: Achieving Representation in Planning page 2 http://www.activedemocracy.net/articles/03_randomselection.pdf
11 Janette Hartz-Karp, A Case Study in Deliberative Democracy: Dialogue with the City http://www.activedemocracy.net/articles/jhk-dialogue-city.pdf
The project was designed as ‘a deliberative process of engagement’ aiming to understand ‘what a large, representative group of Perth residents would want if they were well informed and had the opportunity to deliberate; building the future plan for the city on their common views.’The aim was to be both ‘large scale and representative.’ Features of this process were: (1) A survey of a representative sample of residents to gauge community concerns and values; (2) Release of comprehensive issues papers to inform dialogue; (3) Media (print, radio and TV) discussion of issues; (4) Discussion of four different development scenarios; (5) Searching for common ground between the diverse participants (rather than ‘majority rules’).
12 Pages 19 & 46 RTID291
13 Four pillar economy http://www.premiers.qld.gov.au/about-us/what-we-do/assets/gov-statement-of-objectives.pdf
14 Page 8 RTID201
15 Our Future World report http://www.csiro.au/en/Portals/Partner/Futures/Our-Future-World-report.aspx
16 Steven Ames, What is Visioning? pages 3, 4 & 5 http://www.nh.gov/oep/resource-library/planning/documents/preparing-master-plan-visioning.pdf The four main steps in Ames’ Oregon Model of visioning are as follows: (1) Profiling the community (where are we now?) – identifying the strengths and weaknesses of the community and shared values/beliefs; (2) Analysing the trends (where are we going?) – identifying the probable (trend) future; (3) Creating the vision (where do we want to be?) – describing the preferred future (vision); (4) Developing an action plan (how do we get there?) – action plan(s).
17 Portland 2030: A Vision for the Future page 8 http://www.portlandoregon.gov/bps/article/168876 also page 67 RTID201 in support of this approach
18 An example of what Portlanders said they valued is: ‘We value taking responsibility for actions that will affect our long-term future. Sustainability means meeting the environmental, social, cultural and economic needs of the present while ensuring the similar needs of future generations. Sustainability indicates care and respect for the ecosystem as well as for the people within it’.
19 The six community survey questions http://queenslandplan.qld.gov.au/journey/qld-questions/
20 The Queensland Plan: a 30 year vision for Queensland -our working draft page iv http://www.queenslandplan.qld.gov.au/journey/public-review-working-draft/assets/qld-plan-working-draft.pdf
21 Page 25 RTID201
22 No doubt some MPs engaged quite well with their electorates.
23 Lyn Carson An inventory of democratic deliberative processes in Australia Page 2 http://www.activedemocracy.net/articles/engaging%20comm%20summary%20070115.pdf
24 Pages 97 & 100 RTID201
25 Page 138 RTID201
26 Brisbane summit data report page 2 http://queenslandplan.qld.gov.au/journey/brisbane-summit/assets/brisbane-summit-data-report-october-2013.pdf
27 The active participation by large numbers of special interest group reps, MPs, and local and state government officials detracted from the role of the summits as forums for the wider community
28 For example, the 20 delegates who were state school principals
29 Mackay summit program pages 2, 3 & 4 http://queenslandplan.qld.gov.au/journey/mackay-summit/assets/mackay-summit-data-report.pdf
30 Community engagement feedback report pages 18 & 19 http://www.queenslandplan.qld.gov.au/journey/qld-response/assets/qldplan-report.pdf
31 ‘When online surveys are accessible to anyone who visits a website, the researcher has no control over sample selection. These self-selected opinion polls result in a sample of people who decide to take the survey– not a sample of scientifically selected respondents who represent the larger population. In this situation online survey results are biased because people who just happen to visit the website, people who are persuaded with a monetary or other incentive to sign up for the survey, people who have a vested interest in the survey results and want to influence them in a certain way, and people who are driven to the site by others are included in the sample. This results in a double bias, because this distortion is in addition to the basic sample already having excluded people who do not have Internet access.’ The Fallacy of Online Surveys: No Data Are Better Than Bad Data http://www.responsivemanagement.com/news_from/2010-05-04.htm
32 To achieve this would require a statistically valid survey of the state population
33 Community engagement feedback report page 39 http://www.queenslandplan.qld.gov.au/journey/qld-response/assets/qldplan-report.pdf
34 The Colmar Brunton presentation to the Brisbane summit stated that the number 60,316 had been validated. How this validation was done does not seem to have been reported
35 Community engagement feedback report page 49 http://www.queenslandplan.qld.gov.au/journey/qld-response/assets/qldplan-report.pdf
36 Community engagement feedback report pages 33 & 34
37 The Queensland Plan: a 30 year vision for Queensland – our working draft page http://www.queenslandplan.qld.gov.au/journey/public-review-working-draft/assets/qld-plan-working-draft.pdf
38 Page 46 RTID201
39 As was done for Perth’s Dialogue with the City
40 Community engagement feedback report page 18 ‘The six questions posed to Queenslanders during the consultation period were in essence a conduit to gather insight on our State’s future needs across a range of topical areas – this would in turn allow an interrogation and exploration at a more holistic level what the ultimate focus areas are for the State. Using the insights and key themes uncovered in these questions, we could then look beyond the six questions themselves to explore wider trends and focus areas / goals for the State. Focus areas are fundamentally the goals we are aiming to achieve for our vision. Colmar Brunton has assessed the responses provided across the six questions and has identified five focus areas for Queensland.’ http://queenslandplan.qld.gov.au/visions/assets/qldplan-report.pdf