Alberta Case Study

Alberta Canadian PAME Pilot Project

1.      Background

Alberta Parks manages a system of 473 provincial sites that play a critical role in the conservation of biodiversity and other intrinsic social and cultural values. All of the sites fall under one of eight classifications: Wilderness Areas, Ecological Reserves, Heritage Rangelands, Willmore Wilderness Area, Wildland Provincial Parks, Provincial Parks, Natural Areas, and Provincial Recreation Areas. Over 250 of the sites in the parks and protected area system are helping Canada meet its commitment to achieving Canada Target 1, including meeting quantitative targets like effective management.

In June of 2016, a joint partnership between Alberta Parks, Ontario Parks and University of Northern British Columbia (UNBC) researcher, Dr. Pamela Wright, was initiated to test the evaluation of protected area management effectiveness (PAME) in a sample of protected areas. Project planning was guided by a Management Effectiveness Working Group consisting of Parks Planning staff, Dr. Pamela Wright from UNBC and Dr. Joyce Gould, Science Coordinator for Alberta Parks.

Initial scoping

A critical component of the pilot project was the initial scoping exercise led by Dr. Pam Wright to help choose evaluation tools. Scoping helped the working group identify what they wanted to achieve with the pilot project in terms of 1) Primary Objectives, 2) Reporting Outcomes, 3) Scale, 4) Frequency, 5) Level of External Involvement, 6) Building Support, and 7) Evaluation Criteria (summarized in Table 8).

 

 

 

 

 

Table 8: Scoping assessment outcomes in the alberta pilot

  Alberta
Primary Objectives 1.       Improve protected area management

2.       Strengthen legislative/policy frameworks

3.       Provide support for increased political and financial resources

4.       Increase transparency and improve accountability

5.       Achieve international alignment and recognition

Reporting outcomes ·         Combination of numeric and descriptive format

·         Site-specific scoring but able to roll-up into system wide network evaluation

·         Need to describe and rank threats, identify most important issue to focus on for each site, and understand management processes/responses.

Scale ·         Individual site-level evaluation

·         Complete as part of regional parks planning process

·         But focus on sites identified as being under high threat

Frequency ·         5 years:  sites with high use/high threat

·         10 years:  remote sites with large land bases

External Involvement ·         Complete the pilot internally with no external involvement

·         Ongoing discussion about how to involve stakeholders, First Nations/rightsholders, and other external communities in a future Parks PAME program.

Communication ·         Initial presentation to Parks Directors Committee

·         Formation of Management Effectiveness Working Group involved regional staff (planners)

·         Presentations to partners

Evaluation Criteria ·         Capacity and cost-effectiveness

·         Provide valuable information

·         Replicable across sites and times

·         Robust, valid, credible and simple

 

Planning and Adaptation

Once the scoping exercise was complete, the working group evaluated a short-list of PAME evaluation tools to see how they met identified needs. With 473 sites in the Parks system that fall under a range of classifications, sizes, remoteness and accessibility and use, the working group wanted to test tools that would be quick and easy to use while providing a detailed and fulsome evaluation. The tools needed to be accessible and applicable to frequently used sites as well as remote sites with little information available. It also needed to be a tool that could be adapted to the specific needs of Alberta Parks.

Two tools were chosen, the Management Effectiveness Tracking Tool (METT) and the Enhancing Our Heritage Tool (EOH), which were then customized and adapted to an Alberta context. The intention was to apply each tool to specific sites to help determine which tool works best on which type of site. However, after the first pilot workshop the tools were combined and the core evaluative component was based on the METT tool (renamed Enhanced METT or METT+). Significant adaptations were made:

  • The language and terminology were adjusted to suit a Canadian context (and also modified from jurisdiction to jurisdiction).
  • Where the wording on some questions was found to be consistently unclear, it was reworded or sometimes separated into ‘double barreled’ questions into unique parts where the responses could be different.
  • Questions related to certain issues across all the jurisdictions were enhanced (e.g., an enhanced examination of Management Planning).
  • Where there were jurisdictionally specific processes (e.g., examining Management Plan renewals and currency) that assessed planning adequacy more thoroughly, these completed rankings were substituted in place of METT questions.
  • All available data were used in our rankings (e.g., spatial analysis in other tools used for the PAME pilots provided information on design risks that were linked with the evaluative professional judgment questions within the METT+).
  • The tool was supplemented with extra questions on topics of specific concern that were pan-Canadian, for example, climate change, stakeholder consultation, Indigenous/rights-holder shared management.
  • Capacity-building modules, typically from the EOH toolkit, were built in where a poorer evaluation on an item or topic was linked to a more detailed worksheet from the EOH (or elsewhere). This allowed intensive work on the topic, either as part of the PAME evaluation or as a next step.
  • Additionally, a combination of tools was chosen to support or enhance the actual METT + evaluation tool. This allowed the working group, within the bounds of data available, to include a more comprehensive focus on the conservation, recreation and other social values, the threats to those values and to enhance examination of the outcomes of management.

Tools to Inform the Evaluation:  Values and Threats Assessments

An integral component of the PAME framework is establishing site context by identifying the values of the site and the threats to those values. Evaluators need to begin with a clear understanding of what values each site is supposed to be protecting. In many cases in Alberta, management plans are out of date or non-existent or the values of the sites are unknown, unclear and/or have changed. Every PAME tool includes a basic values and threats assessment component. However, after a review of these basic tools, the working group concluded that the assessments within the METT and EOH tools were not comprehensive enough to meet the identified needs. Instead, additional tools were developed and adapted internally to identify conservation and recreation values and design risks and identify and rank the threats to those values.

A Conservation Values and Risk Assessment (CVRA) tool was adapted for Alberta by Dr. Joyce Gould based on the Conservation Risk Assessment (CRA) tool initially developed by Dr. Pam Wright for BC Parks. Drafts of the adapted tool were reviewed by ecologists in each of the regions as well as to scientists within other divisions in the department for feedback. The tool captures conservation values at a variety of scales, including ecosystems, ecological communities and species for both terrestrial and aquatic systems as well as structural components and function. It includes a section on design risk to capture key components related to the most effective design. Data sources include, but are not limited to, the Alberta Conservation Information Management System, corporate datasets, monitoring information, professional opinion and traditional knowledge. The NatureServe species rank calculator adapted by Dr. Gould to function as the conservation threats assessment calculator for an entire protected area for threats stemming from both inside and outside the site. The calculator captures a standardized list of primary threats based on the Open Standards threats categories and terminology[1] developed by the IUCN-CMP. It assesses threats based on factors of severity, scope and timing.

Complementary Recreation Values and Risk Assessment tools based on the CVRA format were developed by members of the Working Group to identify and assess the main recreation values of the site. A recreation assessment was included because it is an important value in many Alberta Parks sites and plays a key role in how Albertans understand and interact with the parks and protected area system. The values assessment tool provides an overview of the key recreation values and design or use risks through the use of a set of questions based on supporting indicators for each factor. It uses data from Alberta’s draft Recreation Framework inventory, planning documents, pre-existing inventories and digital data such as the provincial Recreation and Tourism Features Inventory (RTFI) and subject matter or local area experts. The associated risk assessment tool mirrors the conservation threats calculator, but with recreation specific threats categories. The tool identifies and assesses the pressures affecting identified recreation values of the site.

Site Selection

The Working Group chose six sites across Alberta based on a range of criteria including classification, size, values, location, issues, intensity and range of use, existing and potential threats and available information to contribute to and guide management actions. Several sites were proposed to each participating region and the final site selection decision was made by each region according to their own needs. The final list of sites included three Provincial Parks, two Wildland Provincial Parks and one Provincial Recreation Area (see table below). At least one site was chosen for each Parks region (Northwest, Northeast, Central and Kananaskis) and a seventh site representing the South Region will be evaluated in Fall 2017, apart from the pilot study.

 

 

 

Site Region Rationale for Selection
Lois Hole Centennial Provincial Park Central Urban, Management Plan in development
Birch Mountains Wildland Provincial Park Northwest Large, remote, northern
Bow Valley Wildland Provincial Park Kananaskis High use/high risk
Lakeland Provincial Park and Provincial Recreation Area Northwest High use/high risk
Peter Lougheed Provincial Park Kananaskis High use/high risk
Hay-Zama Wildland Provincial Park Northeast Large, remote, northern
Castle Mountain Wildland Provincial Park South Recently established

 

2.      Implementation

The workshop process evolved throughout the duration of the pilot project, based on the experiences and feedback from participants and facilitators. Much of what was learned about the process came about as a result of being organizationally flexible and adapting the process to meet the needs and capacity of the participants. Each workshop began with a clear plan in terms of the length, schedule and preparatory instructions and materials, and each of these elements was adjusted throughout the pilot. Regional planning team leads from each region decided who was to be invited, but were encouraged to include a wide range of voices from boots-on-the-ground staff to the higher-level managers and directors.

The first workshop made it very clear that while the values and threats assessments were integral to the process, they required extra time that had not been factored in. An extra day was then added to workshops that had been originally scheduled for one day. Detailed information was initially sent out for review ahead of time, but it was quickly recognized that staff did not have the resources to devote more time to the project than what was allotted for the workshop. Instead, a brief overview of PAME and the pilot project was emailed to participants ahead of the workshop.

A challenge that was encountered in almost every workshop was the desire to rush through the values and threats assessments due to time constraints, but these components were essential to understanding what values the site was being managed for. It was also important to allow staff the opportunity to express their own frustrations with management challenges. Sites that had a high number of threats tended to elicit much discussion, so a time-saving device was to identify and prioritize the perceived top threats and focus on those first before moving on to the next tool component, and then going back to review the remaining threats if there was time at the end. Wrapping up each tool component with a “checking-in” exercise provided an opportunity to gauge how the participants felt about the tool, the process so far, what they were learning, and how the results from the tools compared to their lived experiences and expert knowledge of the site.

3.      Outcomes

Results:  How did we do?

The workshops were successful in engaging the staff and developing a deeper understanding of the site. There was a major learning curve for all participants as they learned about the tools—as they were using them for the first time. Staff described feeling exhausted but exhilarated at the end of each day.

In terms of values, sites in the Kananaskis region had high conservation values related to providing critical and essential habitat for keystone species and apex predators. High ranking of recreation values reflected the sites’ very high usage. Central and northern sites had high conservation values, as areas of low disturbance surrounded disturbed landscapes. Residential development is the predominant threat in the central region site while energy development was the most significant threat assessed in the northern sites. Additionally, northern sites with challenging accessibility tended to have low recreation values while the central and northern sites with easy access had high recreation values. The highest-ranking threats to each site are summarized in the table below.

Threat Categories LHCPP Birch BVWPP Lakeland PLPP HZ WPP
Residential & Commercial Development ü          
Energy Production & Mining   ü       ü
Transportation & Service Corridors ü          
Biological Resource Use       ü    
Human Intrusion & Disturbance ü   ü ü ü  
Natural System Modifications ü   ü ü ü  
Ecological Management / Natural Factors         ü ü
Pollution ü          
Climate Change & Severe Weather ü       ü  

 

For METT+ results, the score for each component was shown as a percentage and rated based on a predetermined scoring level standard. During each workshop, participants were given thirty minutes to complete the survey individually, which was then completed as a group. Where there were differing scores based on individual assessments, a final score was based on discussion and consensus. The comments recorded for each question are extremely valuable and used to identify issues and gaps, providing useful information for recommendations. Interestingly, the outcomes component tended to score high even where the other management elements scored low. Alberta Parks has limited and inconsistent outcomes monitoring for individual sites, and the outcomes questions were answered subjectively. This may change over time as a Parks monitoring program is developed.

 

 

 

PAME Component LHCPP BMWPP BVWPP LLPP
& PRA
PLPP HZ WPP   Scoring Level Standard
Context 56% 44% 56% 61% 56% 56%   (67%-100%) Sound Management
Planning 21% 46% 55% 44% 52% 50%   (51%-66%) Basic Management
Input 51% 45% 47% 63% 47% 45%   (34%-50%) Basic Management with Significant Deficiencies
Process 47% 31% 49% 51% 53% 60%   (0- 33%) Clearly Inadequate Management
Output 33% 64% 75% 67% 76% 75%      
Outcomes 35% 81% 75% 82% 84% 83%      

 

Key learnings

  • Understand and articulate what you want to achieve. The scoping exercise was critical in understanding the desired goals, purpose and outcomes and helped the working group to pick the appropriate tools.
  • Assess and gauge the temperature of the organization. Alberta has been seeking a way to implement a cultural shift and saw this pilot project as an opportunity to incorporate adaptive management tools into existing processes.
  • Reach out broadly within the department to build a network of resources and expertise. The pilot project process has functioned as a tool to build bridges between divisions and departments. There are a lot of data gaps, so it is important to supplement experts in to the process beyond your organization.
  • Beware your biases. One of the biggest challenges to a fulsome assessment was the lack of data available. While it is perfectly acceptable to somewhat rely on professional opinion, it is important to pair technical experts, who may be more impartial, with site experts to retain objectivity about the site.
  • A comprehensive values/risk/threats assessment component is essential to the process. The assessment component helped to clarify what the conservation and recreation values the sites were being managed for and provided an opportunity to identify and rank the threats to those values, which, in turn enhanced the management effectiveness evaluation component. It is important to allow for lots of time to develop and adapt assessment tools and not rush through the values and threats assessments component.
  • The process is about collective learning. Opportunities for a broad range of staff to gather together to discuss one site are very rare. One of the most frequent comments was that the process provided an incredible learning opportunity for the staff to develop an understanding of the site from a range of different perspectives. This ultimately changed the way they thought about how it was being managed.
  • Include a wide range of voices/perspectives/participants in the room. There is a need for both technical staff whom are not necessarily tied directly to the site but can contribute and provide less biased context to the values/threats, and staff and site managers who are familiar with the site itself.
  • Have faith in your process but be flexible. Significant changes were made to the tools, but the overall process remained the same: the values and threats assessments upfront followed by the management effectiveness evaluation. Due to the range of backgrounds and expertise of participants, it was sometimes challenging to keep participants engaged, especially during the conservation values assessment, but it was an opportunity for everyone to fundamentally understand the site’s conservation values.
  • Test the tools and the process first. When the pilot project began, it was not known what tools would work with what types of sites or how much time each component would take. With a completed a pilot project, there is a much stronger sense of how to develop and implement a PAME program.

4.      Next steps

The response to the PAME pilot project in Alberta has been overwhelmingly positive and the number one question asked at the end of each workshop is “This is great but what’s next?”. The main concern is that the project will lose momentum and be shelved. Over and over again, participants have stated that a management effectiveness program could initiate a fundamental shift in how protected areas are managed in Alberta, making Alberta Parks a more proactive, rather than reactive, organization. Participants could envision how this suite of tools could support adaptive management. The PAME pilot project has been invaluable in revealing a path forward for Alberta by showing what does and does not work and by identifying the major concerns and potential barriers to overcome for a successful program.

Toolkit Development

The Enhancing our Heritage (EOH) toolkit in its entire format (12 worksheets) was considered too time-consuming for the needs of the pilot project; however, the individual worksheets within the toolkit package are an excellent resource for focusing on specific components of management effectiveness that may be identified through the METT evaluation. Other components of the toolkit, such as the reporting tools, are being developed from scratch.

  1. Management plan survey: Tool to assess the adequacy of the management planning document. Useful for sites with out of date plans and plans under review or development.
  2. Stakeholder/Indigenous METT: Tool to help identify stakeholders and work with them to assess what their relationship is like with the site and the site values.
  3. Assessment and evaluation linking tool: Tool or process to bridge the values and threats assessments with the management effectiveness evaluation. Current practice is a discussion to summarize results from the assessments prior to completing the METT.
  4. Other values assessment: Tool to assess other social values (beyond recreation). Currently these have been identified and captured in different ways throughout the workshops.
  5. Training and tool guides: Developed for both the conservation and recreation assessment tools. The existing METT tool guide will likely need to be adapted to incorporate changes. There are ongoing discussions about whether to include a tool training component such as a video tutorial, which may be incorporated as part of facilitator training when the Alberta PAME program is developed.
  6. Output assessment tool: A tool from the EOH toolkit that assesses the achievement of annual work program targets and other output indicators for the site.
  7. Monitoring management outcomes tool: A tool to help identify whether the site is protecting its values and achieving its management objectives. Involves the development of a long-term monitoring program.
  8. Reporting tools: A range of PAME reporting tools is being considered and developed as part of the outcomes of each workshop including:
  • A PAME Workshop Report that provides a summary of all results from each of the tools and recommendations
  • Values and threats assessment summary tables within management plans
  • Summary of data and research gaps
  • “PAME Report Card” that gives a letter grade based on the overall score from the METT+ as well as the score from each component and includes summary of comments for each component.

Monitoring Pilot Project

A monitoring pilot project of parks and protected areas based on the outcomes of the PAME pilot project is currently being developed in partnership with the Environmental Monitoring and Science Division. The five-year project will design, plan and initiate a system for monitoring biodiversity and ecosystem health in Alberta’s parks relative to the broader provincial landscape. It will include outcomes on monitoring design and standard operating protocols. The project will also examine the value of Alberta’s parks as benchmarks for ecological integrity.

[1] http://cmp-openstandards.org/using-os/tools/threats-taxonomy/