MRIP Priorities
As outlined in the initial MRIP Implementation Plan and subsequent updates, the MRIP strategy is to improve our survey methods, as needed, to meet the changing needs for successful fisheries management. This includes improving the fundamental design of our surveys, current data collection tools and developing new designs to reduce bias, address gaps in information, and produce more precise, accurate statistics.
NOAA Fisheries, through MRIP, is continuously evaluating and testing survey improvements and implementing these improvements when appropriate. We’ve also designed new surveys and additional survey components will be added as necessary to provide the appropriate mix of monitoring tools needed to accurately track fishing effort and catch in each region. As results emerge indicating which combinations of methods show the most promise for specific regions, the MRIP Regional Management Teams (the Fisheries Information Networks (FINs) or their equivalents) will select the data collection design best suited to their fisheries, management needs, and other uses. The Regional Teams and MRIP will continue to evaluate the possibilities of broadening the application of those tools, consistent with available funding, to attempt to improve coverage and/or bring more resolution to spatial and temporal monitoring.
Overall Goal
To ensure that we are addressing national and regional needs, our specific priorities shift each year. However, the overarching priorities for MRIP include:
- Catalogue and review all of our survey designs to determine their accuracy and effectiveness at meeting the needs of scientists, stock assessors, managers and stakeholders.
- Through extensive testing, stakeholder collaboration and work with outside specialists and independent consultants, design and implement a series of improvements to ensure our survey methods meet the highest possible standards of accuracy and quality.
- As we finalize improvements to our fundamental designs, we continue our work to meet customer needs for timeliness, precision, coverage, and special data needs.
In 2004, NOAA Fisheries requested the National Research Council (NRC) conduct a review of all of the surveys operated or funded by NOAA Fisheries throughout the United States and its Territories. The final report was released in 2006.
The NRC made several specific recommendations for improvements to the way NOAA Fisheries was collecting data and compiling recreational fishing statistics, and the MRIP team has worked to address each one. Many of the improvements to our survey methods have been guided by the NRC review.
Specific initiatives that have been implemented based on the NRC recommendations include the creation of the National Saltwater Angler Registry, the re-estimation of recreational catch estimates dating back to 2004, and the overhaul of our angler intercept survey design to remove potential sources of bias. Other issues that we are addressing in direct response to the NRC report include evaluating how to most effectively account for the fishing that occurs in the for-hire sector and how to best survey the numbers of shore and private boat fishing trips being made by recreational fishing participants.
Current Priorities
With significant improvements to our catch surveys now implemented, current priorities for MRIP include:
- Transitioning to the new fishing effort survey for shore and private boats. Results from a series of pilot studies show that a mail survey design is more efficient and accurate than the current telephone survey. The mail survey design utilizes the National Saltwater Angler Registry and is being conducted alongside the current Coastal Household Telephone Survey to continue assess effects the new design may have on both estimates and management processes. View the Transition Plan.
- Facilitating the transition from our previous methods to our new surveys. New survey designs may produce results that are consistently different from old survey methods. These differences can impact stock assessment and management processes. MRIP has created a Transition Team made up of scientists and managers to determine how best to integrate data from new methods into science and decision-making.
Of course, MRIP’s greatest priority is to ensure that the needs of our data customers are met. Fisheries science and management are dynamic processes with evolving needs for data. MRIP has the flexibility to address these demands and be responsive to the needs of the millions of Americans whose lives and livelihoods are connected to our sustainable fisheries.
Research
A primary tenet of MRIP is that all of our survey methods undergo extensive testing before being implemented. This research ensures that the surveys we use deliver high quality results, and that any new method yields tangible improvements. We’re constantly working to update our methods to keep pace with emerging science and the information needs of our users. We work with scientists, managers, fishermen, and others to make sure our research adds value to MRIP and the individuals that are impacted by the program. Some of the changes to our surveys are highly visible and some are behind the scenes. However, all the changes enhance the quality of recreational fishing data.
Example Projects
1. Implementing an improved catch survey design
MRIP developed, tested, approved, and certified a new survey design for the Access Point Angler Intercept Survey (APAIS) - the survey used for estimating how many fish are caught, kept, and discarded during a given sampling period. The purpose of the study was to overhaul the previous survey design to minimize potential sources of bias. Key changes to our catch surveys that resulted from this study include:
- We now sample at all times of the day; we’re no longer assuming that catch rates at night are the same as those during daytime.
- The new design more strictly adheres to probability sampling protocols. Samplers now follow a rigid schedule when conducting interviewing assignments at specified fishing sites and do not have latitude to change where and when they attempt to intercept and interview fishermen.
- We now have an online register of most public fishing sites from Maine to Louisiana. This allows us to make better assignments for our samplers, and is accessible to anyone who’s looking for a place to fish.
2. Electronic logbook reporting with validation for headboats operating in the U.S. South Atlantic
Over the past several years, MRIP has funded studies to assess the feasibility and reliability of electronic reporting of catch for for-hire vessels. We continued this trend by supporting the expansion of electronic logbook reporting to all headboats reporting to the Southeast Region Headboat Survey, an undertaking that resulted in 160 headboats in the south Atlantic and Gulf of Mexico with the option of participating in electronic reporting. A previous pilot study conducted in the region showed a high percentage of both participation and species reporting accuracy compared with the paper logbook reporting.
However, before this effort can be extended and implemented for collecting the necessary data for management purposes, pilot studies will continue. Improvements in the software (including an internet-based reporting system); training of headboat captains in reporting, revising the regulatory structure of electronic reporting; and the addition of specific data elements such as precise locations, fishing depth, and target species are being done with the current survey-wide implementation.
3. Using multiple methods to improve effort survey results.
In 2015, MRIP implemented a new survey to estimate the overall number of recreational fishing trips, or “effort”, during a given reporting period. This new survey is based on a multi-phase series of studies to design a more effective means of reaching anglers than through random-digit-dial (RDD) household telephone surveys. RDD is a well-established survey protocol, but it also has known limitations, especially as fewer and fewer households can be reached by landline phones. To test alternatives to an RDD design, the MRIP team ran a sequences of tests:
- First, we tested whether information from the National Saltwater Angler Registry could be used to reach every potential angler. We found that there are too many gaps in the Registry (e.g., seniors who are not required to register; people who decide to fish without a license; incorrect entries on license applications; etc.) for it to be our sole source of information.
- Second, we tested RDD and license information together. This gave us better coverage, but response rates were still low as fewer and fewer people answer phone surveys.
- Next, we tested to see whether mail surveys would be more effective than telephone. The results showed that by using a mail survey we can get better information from more people in a timeframe that meets our schedule.
- Finally, we matched up the National Saltwater Angler Registry data with the U.S. Postal Service household database, which includes virtually every household in the country. This allowed us to ensure we were able to reach all potential U.S. anglers, while taking advantage of the sampling efficiency offered by use of the angler registry.
Complete descriptions of all MRIP-funded projects are available here. You can also view our MRIP Project Tree, which categorizes and lists all projects based on survey type and relatedness to other projects.
Implementation
Regional Implementation Approach
MRIP is a compilation of regionally-based data collection programs. Regional programs are important because the data needs for effective management can differ substantially among regions, and oftentimes among various fisheries within each region. National standards ensure that, regardless of the specific decisions made by each region with respect to data collection priorities and implementation, all recreational fisheries survey and estimation methods will withstand a rigorous independent peer review, and the resultant fisheries statistics will meet a baseline of quality sufficient to ensure the sustainable use of the resource for recreational fishing.
As MRIP evolves from being substantially focused on developing and testing survey improvements to a program increasingly involved with putting new methods to practice in the field, the Executive Steering Committee (ESC) has established a hybrid approach to MRIP implementation.
MRIP Regional Implementation Workshop Report
Under this system, NOAA Fisheries (through MRIP) will maintain a central role in developing and certifying survey methods and establishing national standards and best practices, and the regions - through the regional Fishery Information Networks (FINs) or equivalent - will have responsibility for selecting survey methods and managing data collection.
Inaugural Regional Implementation Plans
These inaugural MRIP Regional Implementation Plans represent a significant evolution in the course of the program, clearly establishing that each region will take the lead role in determining which survey methods are most suitable for their science, stock assessment, and management needs. MRIP will use these plans to develop a national inventory of partner needs and associated costs, and annually specify priority-setting criteria for supporting those needs.
GulfFIN MRIP Implementation Plan
1 For management of grants from MRIP appropriations
2 For procurement of Regional Survey service
MRIP Standards and Best Practices
Since 2008, MRIP has made significant changes in the way NOAA Fisheries collects, analyzes, and reports recreational fishing data, as well as the way we communicate about that data with our partners, stakeholders, and the public. Because estimating recreational fishing activity is a very complex statistical, logistical, and mathematical challenge, we have procedures in place to confirm the scientific integrity of our methods at several different points in the process.
To promote nationwide consistency in availability of recreational catch and effort data, MRIP has established a rigorous process for assuring that new survey and estimation methods are scientifically sound before they are ”certified” and made available for MRIP partners’ use. Further, MRIP incorporates detailed quality assurance and quality control (QA/QC) requirements to minimize errors in our estimates.
MRIP Standards and Best Practices for Recreational Survey Coverage and Basic Data Elements
Project-Level Review Process
MRIP is meticulous in verifying that the new methods we implement are actually solving the issues they are designed to address. To do so, each project goes through a series of steps before its conclusions are considered valid.
First, the project must be approved by the MRIP Operations Team. This group of NOAA staff and outside partners reviews each proposed project to ensure that it addresses an MRIP priority and its essential design and methodology are sound. This is an iterative process, where the Operations Team and the project team discuss options and potential changes before the project is approved and launched. Once the project is underway, monthly status reports to the Operations Team help make sure the work is on track.
At the conclusion of each project, the findings are reviewed by the Operations Team and – in cases where the study will have a major impact on our surveys – sent to independent peer reviewers. Then, after any issues raised by peer reviewers are fully addressed, the results and any recommendations are reviewed by the MRIP Executive Steering Committee. For projects that will lead to new survey methods, the ESC recommends final approval and Certification of the new design by the agency leadership. Once approved by the leadership, the new methodology is certified and made available for use.
Data-Level Review Process
Even in the best systems, various types of errors occur. To minimize the potential for error, MRIP uses a variety of best practices for quality assurance (QA) and quality control (QC). QA focuses on preventing invalid data from entering the system. To accomplish this goal, we train interviewers, keep data entry intuitive, automate tasks, and have verification interviews. QC focuses on detecting and correcting errors that make it into our data, and can be used to inform enhancements to our QA processes. MRIP uses statistical software (SAS) to systematically identify invalid, outlier and mismatched data before and after we calculate our estimates. At each step of analysis, the MRIP team also manually reviews the data for potential errors. For instance, an unusually high or low catch estimate will be identified and investigated before estimates are published.
In addition, data is made publicly available in two forms, “preliminary” and “final.” Publishing the preliminary data allows anyone with specific questions or concerns to raise them for further review, investigation and, if necessary, correction before the numbers become final. We have also worked to ensure that the tools we use to publish the data and the information we share alongside it – such as providing the option for data to be viewed in graph form – help users understand the limitations of our estimates. For instance, we highlight estimates when the proportional standard error (PSE) is particularly high, meaning that the estimate is imprecise and potentially inappropriate for certain types of analysis. We also stress that sample sizes decrease as estimation periods or areas decrease, resulting in more imprecise estimates.
Transparency
Finally, each step in the MRIP process is conducted in cooperation with, and in the full view of, a wide range of partners, stakeholders, charter boat operators, recreational fishermen, and members of the public. These individuals serve on the MRIP Operations Team and Executive Steering Committee, as well as individual project teams. We also create groups of ad hoc reviewers as appropriate. In the case of the highly complex and technical catch re-estimation, we invited a diverse group of stakeholders to serve on an “observer team” that received detailed, plain-language briefings, exercises, and outreach materials designed to fully explain – and solicit feedback on – the process as it unfolded.
Why This Matters
As the NRC noted in its review of our surveys, gold-standard data is only one piece of the picture. For data to serve its purpose as the foundation of sound ocean policies and sustainable stewardship, it must be trusted by the people who use it – and the people who are impacted by it. So ultimately the quality control process is serving two distinct yet interrelated purposes: ensuring that data itself is of the highest possible quality and instilling confidence in that data among the many stakeholders who use it.
