FLR: a framework for the evaluation of management strategies.

ICES Galway Management Strategy Symposium, Theme 3 Tools for achieving robustness in management.

The FLR paper

The FLR framework is a development effort directed towards the evaluation of management startegies. The overall goal is to develop a common framework to facilitate collaboration within and across disciplines, and in particular to ensure that models and software once developed are more easily validated, evaluated and more widely available for a variety of tasks. In particular it details how to implement a variety of fishery, biological and economic software in a common framework so that alternative management strategies and procedures can be evaluated for their robustness to uncertainty before implementation. The design of the framework, including the adoption of object-orientated programming, its extensibility to new processes and its application to new management approaches (e.g. ecosystem affects of fishing) is discussed. The importance of open source for promoting transparency and allowing technology transfer between disciplines and researchers are stressed.

Management of fisheries increasingly embodies multiple and often conflicting biological, economic and social objectives and, despite constant efforts to regulate fisheries by regional management bodies and national governments, fishing capacity often remains above that necessary to sustainably exploit marine resources, especially in developed countries. This failure has been analysed in depth during the last decade by the scientific community, which has repeatedly recommended substantial changes in incentives and governance, as well as adjustments in the way that fisheries research and monitoring are conducted and expertise is deployed (e.g. Bosford et al. 1997, Gislason et al. 2000, Pauly et al. 2002, Sinclair et al. 2002, Garcia & de leiva, Moreno 2003, Hilborn et al. 2004, Jennings 2004, Sissenwine & Murawski 2004, Grafton et al. 2006). However, while the need to develop alternative novel management strategies is widely recognised, it is almost impossible to develop these by conducting large-scale experiments on fish stocks, with the notable exception of Sainsbury et al. 1997. There has therefore been a trend towards the use of computer simulation to develop management strategies that can meet multiple objectives that are robust to uncertainty (Kell et al 2005a,b & 2006).

Models are simplifications of the real world and the major and first task of modellers is to determine how to simplify without also losing the ability to understand and inform management. Then parameters of interest (e.g. related to fishing mortality or stock size) can be estimated and used to provide management advice. A major failing of conventional fisheries advice in this regard is that it does not explicitly incorporate important sources of uncertainty. For example it is generally assumed that: (i) appropriate input data are available and not biased, (ii) stock assessment models accurately reflect (without strong bias) both population and fisheries dynamics and (iii) management measures are perfectly implemented (Cotter et al. 2004; Peterman 2004; Punt, in press). In others words, the robustness of the advice to uncertainty with respect to both the intrinsic properties of natural systems and our ability to understand, monitor and control them is largely ignored.

Following Rosenberg and Restrepo (1994), Francis and Shotton (1997) and Kell et al. (2005a, b & 2006), uncertainties in fish stock assessment and management can be categorised as:

1. Process error – natural variability, in both time and space, in dynamic processes of the populations (e.g. recruitment, predation, growth and migration) and of the fisheries;
2. Observation error – related to collecting in-situ observations, such as total catch, catch composition, biological data (e.g. length, size, maturity), research survey, effort;
3. Estimation error – arises when estimating parameters of the various models used in the assessment procedure, such as growth models, stock-recruitment models, virtual population analyses, statistical models;
4. Model error – related to the ability of the model structure to capture the core of the system dynamics;
5. Implementation error – because management actions are never implemented perfectly and may result in realised total catch, catch composition and effort that differ from those intended.

In reality many of these error types are interdependent. For example, while estimation error can usually be decomposed into process and observation error separating the two can also be difficult. Estimation error may also occur even in the absence of measurement and process error, for example if there is no contrast in the data it may not be possible to provide precise or unbiased estimates of population parameters such as maximum sustainable yield (MSY). It is therefore not enough to identify the sources of error; their complex interactive relationship should be understood as well.

While the statistical models of Fournier et al. (1998), Michielsens et al. (2006) and Porch et al. (2006) can integrate several sources of uncertainty (e.g. observation and process error), none of the current stock assessment models can rigorously test the robustness of a management strategy (i.e. the combination of specific management objectives and associated implementation measures) to a wide range of uncertainties. Also, traditional stock assessment requires a regular and time-consuming re-evaluation of data, assessment methodology and (usually an annual) update of the actual assessment results before management advice is given. Hilborn (2003) forecasts the end of such a treadmill, where increasingly complex models are run each year to produce an estimate of stock status that then determines management action. Instead, he anticipates the use of Management Strategy Evaluation (MSE) where complex models are used primarily to test the robustness of simpler assessment/management rules before implementation by conducting computer based experiments that embody how the whole fishery system reacts to a variety of management actions. Population and fleet dynamics are deduced from a range of plausible hypotheses and available data sets since the main objective is to develop strategies that are robust to our uncertainty about the “true” dynamics and hence meet the requirements of FAO’s precautionary approach to fisheries management (FAO, 1996). Peterman (2004) compared MSE to flight simulators which “... include detailed dynamic feedback processes to help pilots determine which decision-making protocols are best in the presence of a wide range of possible, but uncertain, simulated contingencies”. Selection of assessment and management procedures for implementation is based upon an evaluation of their performance with respect to explicitly stated and prioritised management objectives.

A specific case of the MSE approach is the development of Operational Management Procedures (OMPs), i.e. a simulation-tested set of procedures or rules used to determine management actions, in which the data, assessment methods and the harvest control rules (HCR) for implementing management actions (i.e. the rules used for decision making) are pre-specified (Butterworth et al., 1997). To date various OMPs have been developed (see Butterworth et al., 1997; Cochrane et al., 1998; Kirkwood and Smith, 1996; Cooke, 1999; McAllister et al., 1999).

The MSE approach requires mathematical representations of two systems: a ‘real’ or ‘true’ system and an ‘observed’ one. The ‘real’ system, represented by the operating model, is a simulation model corresponding to the underlying situation in the fishery. It attempts to capture all existing knowledge and data and in some cases presumptions and opinion about the real system (Hammond and Donovan, in press) including the full dynamics of the exploited populations, the fishers’ behaviour, and environmental conditions as well as interactions between all its components.

The operating model will often contain a higher level of complexity and knowledge than that used within stock assessment models. It also allows the evaluation of the consequences of contrasting hypotheses about the ‘real’ population dynamics on the current management procedures. Fromentin and Kell (submitted) have, for instance, tested the impact of long-term variations in catches attributable to either changes in carrying capacity or the stock’s migration pattern on the current stock assessment and management procedures of the International Commission for the Conservation of Atlantic Tuna.

In contrast, the ‘observed’ system represents the conventional stock assessment and management procedure, from the data collection to the management implementation. It may be based on the current or alternative stock assessment and management procedures and includes: (i) an observation model that describes how simulated observations are sampled from the operating model; (ii) an assessment procedure to derivate estimates of stock status from the simulated observations, (iii) a harvest control rule (or alternatively a simpler management strategy based on technical measures) that defines a set of management actions according to some specified system indicators and (iv) an implementation model that describes how management actions are really implemented. The ‘observed’ system will further act on the ‘real’ system through feedback of the management options. For example, the main management instrument of the European Union’s Common Fisheries Policy is setting the total allowable catch (TAC). This, in turn, affects the fishing mortalities supported by the simulated population of the OM. However reported catches are one of the main sources of data for providing scientific advice, this means that bias in the assessment process can be driven by management advice which in turn is based upon the assessment process.

Butterworth and Punt (1999) noted that the lack of any general software packages was a major impediment to the construction of operating models for use in management strategy evaluation. While the need for a integrated suite of software that allows data exploration, stock assessments and forecasts and the testing of management strategies to be conducted within a common framework has requested by scientists with ICES (2004). For these reasons the EU project FEMS (contract Q5RS - 2002 – 01824) proposed and initially developed a generic framework that is now the FLR inititive (http://www.flr-project.org). From a general perspective FLR aims to provide a solid basis for the development and evalaution of methods in fisheries science. It allows operating models to be developed, conditioned on a variety of data and hypotheses, and alternative stock assessment and management procedures to be implemented in software. FLR can also be used to perform exploratory data analysis and provide estimates of population parameters based upon a range of data and assumptions in order to create and run simulation operating models. It can also perform stock assessments (incorporating existing methods written in Fortran and C/C++ and any new methods that are developed) and test harvest control rules for stock assessment working groups (ICES 2006abcd). All of this can be done in either a frequentist or Bayesian framework. In addition FLR is currently being extended to incorporate mixed fisheries, multispecies and ecosystem and economic models. Routine for automatic differentiation are also being included to help increase the running speed.

FLR is developed using R (http://www.r-project.org), an environment and computer language for statistical computing and graphics which is highly extensible. It includes effective data handling and storage facilities, mathematical operators including those for matrices, a large, coherent, integrated collection of statistical, mathematical and graphical tools for data analysis. The term “environment” is intended to characterise R as a fully planned and coherent system, rather than an incremental accretion of very specific, inflexible and rigid tools, as is frequently the case with other data analysis software and in particular with much fisheries software. R, is also designed around a true computer language, and allows users to add additional functionality by defining new functions or developing their own packages or libraries. FLR takes advantage of these features of R and extends them to fisheries modelling. Using R as a common environment should encourage collaboration within and across disciplines.

Like R, FLR is an OpenSource project which, in simple terms, means that the source code is available to the users. From a scientific perspective, the philosophy of an OpenSource project is that by providing access to the source code allows scientists to check/validate/review the implementation of the methods, computation carried out, assumptions made, etc.; which constitutes an implicit peers’ review process. On the other hand, code sharing is also a way of speeding up the scientific process since by doing so the scientific community has access to a wide set of tools for data analysis. This way scientists can focus on the real issues, instead of wasting time “reinventing the wheel”, i.e. rewriting specific software which was already developed by someone else.

A lot more can be said about the definition of OpenSource (e.g see wikipedia) one of the most important subjects is its relationship with “free” software. In this context “free” refers to “freedom” not to “free beer”. More information about OpenSource and Free Software can be found in GNU Project, OpenSource Initiative and The Free Software Foundation.

The way people can use the source code (inspect, extend, distribute, sell, ...) is defined by the license, in the case of FLR is the GNU General Public License, version 2, which, among other issues, states that one can redistribute modified versions of FLR as long as it is also licensed under GPL. Having several people working on the same code at the same time requires organization and tools to communicate between developers and users, share code, fix coding conflicts, write documentation, etc. FLR makes use of several tools to deal with these aspects of team development and project dissemination. Most services like mailing lists, version control system, package releases, are hosted by sourceforge.net which provides free web services for the development of OpenSource projects.

The main web page for the project (http://flr-project.org) uses a wiki engine and aggregates a long set of documents (how to’s, manuals, examples, etc) which are public and explain the initiative and how to use FLR. Sharing code files is carried out by CVS, also known as the Concurrent Versioning System. It keeps track of all work and all changes in a set of files and allows several (potentially widely separated) developers to collaborate (Wikipedia). Mailing lists allow the communication among developers and users, quickly and simple (by email). The messages are archived and allow searching. Voice conferencing provides a fast and easy procedure to have meetings on specific tasks and all developers/users can participate. The bug tracking system is used to organize bug reports, ideas, features and any other tasks needed to keep the development going.

The FLR framework is implemented using object-oriented programming (OOP) by making use of the S4 classes within R (Chambers, 2000). The essence of OOP is to treat data, and the procedures that act upon the data, as a single “object”. These objects are of particular types or classes, which have been developed to best represent the different elements of the system. Using this approach, different elements of fisheries systems (stocks, fleets, assessment methods etc.) are represented as predefined classes. Users do not need to know the internal structure of a class to be able to effectively use FLR.

As well as core classes which provide a common framework there are secondary packages, containing additional classes and functions, which are intended to be used for specific tasks, (e.g. different stock assessment methods) or to extend the framework (e.g. economic and /or ecosystem models). Further information about the structure and use of these classes can be found in the FLR documentation and in the FLR tutorials: http://www.flr-project.org/doku.php?id=courses:tyflr. Models and methods that are available in FLR are shown in Table 1.

Although the vast majority of programming of the FLR framework is in R, for the sake of speed or when specific algorithms already exist in a language like Fortran or C++, classes may also include code written in other languages. For example, solving nonlinear equations is computationally intensive and so fast routines using automatic differentiation have been written in C++ which are called from R. Existing stock assessment methods, e.g. ICA (ref?) and XSA (Shepherd 1999), have also been integrated into FLR using the original code. Even when classes have additional code that is written in languages other than R, R is still the front end of the FLR framework and the user is unaware of the use of other languages.

Class description	FLR Class	Reference
Short term forecast	FLSTF	http://www.flr-project.org/doku.php?id=pkg:flstf
Calculation of biological reference points	FLBRP	http://www.flr-project.org/doku.php?id=pkg:flbrp
VPA	FLAssess	http://www.flr-project.org/doku.php?id=pkg:flassess
Separable VPA	FLAssess	http://www.flr-project.org/doku.php?id=pkg:flassess
XSA	FLXSA	http://www.flr-project.org/doku.php?id=pkg:flxsa
ICA	FLICA	http://www.flr-project.org/doku.php?id=pkg:flica
SURBA	FLSURBA	http://www.flr-project.org/doku.php?id=pkg:flsurba
Bayesian surplus production	FLBayes	http://www.flr-project.org/doku.php?id=pkg:flbayes
Stock-Recruitment	FLCore	http://www.flr-project.org/doku.php?id=pkg:flcore
Bayesian Stock-recruitment	FLBayes	http://www.flr-project.org/doku.php?id=pkg:flbayes
Pending classes
B-Adapt	FLBAdapt

Table 1. Models and methods implemented in FLR. Further information on the methods, including details of equations and calculations, can be found in the user manual at the appropriate reference.

Figure 2 shows how the conceptual framework (Figure 1) can be implemented using FLR classes. As mentioned above the operating model (OM) represents the “true” system. The biological population is represented by an object of class FLBiol. It is possible, though not essential, to also model the stock-recruitment relationship of the biological population. This can be done be using the class FLSR. The biological population interacts with the fishing fleet, represented by the FLFleet class. The MSE is not restricted to a single biological population and it is possible to model several stocks using the class FLBiols, which is essentially a collection of FLBiol objects.

Observations from the operating model are passed to the management procedure (MP).As we can not observe the real world directly, only through the data that we collect, observation error is an important error process. Observation error is implemented in FLR using the class FLOE and objects of this class act as the the link between the operating model and the management procedure. Observations are generated from the variables simulated in the OM (both biological and human), and are used – directly, or indirectly – in the MP to ascertain stock status. The MP uses the FLStock class to model the stock data and the FLIndex class to model indices of abundance. Stock assessment is carried out using the secondary package FLAssess which provides a standard class for data input, diagnostic inspection and stock status estimation. FLAssess is very flexible and can be used either within a working group setting to perform independent stock assessments, or as part of a formal MSE. The basic FLAssess class can be easily extended to implement a range of stock assessment methods. The use of an extendable, basic assessment class means that problems associated with the development of assessment methods may be avoided. For example, within ICES there are two main stock assessment methods: ICA for pelagic and XSA for demersal stocks. However, the differences between the methods arose mainly as a result of their independent development rather than because of methodological differences. This has resulted in a lack of technological transfer between the two methods. Importantly, FLAssess also incorporates methods for VPA, stock projection and estimation of likelihoods. This allows new stock estimators to be developed through the evaluation of their performance using MSE, rather than relying on a “magnanimous programmer”.

The estimates of stock status obtained from the stock assessment or the data themselves are used by the harvest control rule, which attempts to affect the behaviour of the human elements in the OM (e.g. through the use of TACs) to achieve specific goals within prescribed constraints. Several classes are available in FLR to assist in implementing a harvest control rule including a class for performing a short-term forecast (FLSTF) and a class to calculate biological reference points (FLBRP). The results of the harvest control rule are fed back into the OM. However, in the real world management actions are never implemented perfectly and within FLR implementation error can be modelled in a variety of ways, e.g. by modelling the relationship between fleet capacity, effort and fishing mortality. This should take into account factors that may cause these quantities to differ, such as limitations imposed by bycatch. FLFleet therefore has attributes which record catches, landings and discards made from different biological populations. Additionally, processes such as fleet adaptation, the learning processes of fishers, and economic considerations, are being incorporated within a secondary class, FLEcon, to better reflect the behaviour of the fisheries system being modelled. Mardle et al. (2006) focus specifically on the functional relationships for effort allocation in a mixed fisheries context dependent on TACs, profit and days-at-sea restrictions.

Operating models are simulation models that represent the underlying situation in the fishery (e.g. stock dynamics, fishers’ behaviour). Therefore it is important that they capture the existing knowledge and data for the fishery, including both what is known and what is hypothesised. Operating model components, whether biological, economic or bio-economic, must be ‘conditioned’ on the available data. A model is conditioned on data if it is fitted (in some way) to the data, so that the model predictions of the data are approximately consistent with the actual data.

A set of structurally-different operating models are needed to evaluate the robustness of candidate OMPs against the full range of uncertainty that applies to the fishery under consideration. The Operating Model represents alternative hypotheses about the true system dynamics, because there is usually insufficient knowledge to decide upon the correct biological processes, computing power to model it, or data available to help parameterise it and it is also often impossible to predict future dynamics. The conditioning process can lead to an undesirably narrow range of scenarios, e.g. those have either been observed, or are at least fairly likely given the observed data. However, conditioning need not necessarily lead to a narrow range of scenarios and management strategies should also be tested for problematic cases which have not yet been observed and/or about which there are open hypotheses, but which are nevertheless possible, i.e. representing ‘justified concerns’ to which the strategies should be robust.

In practice a variety of procedures may be used to develop operating models. Kell et al, in press, identified 4 different approaches, expressed mostly in a Bayesian context, but equally relevant within a frequentist philosophy. Many evaluations of management strategies use all four of these ways to some extent. The amount of knowledge, data requirements, and complexity of implementation differs quite markedly among these four types. Examples of all four types have been implemented in FLR.

I The operating model is the currently-used stock assessment model. The use of an assessment model as the operating model implies that assessment model describes the system almost perfectly. However, if a part of a management strategy cannot perform well when reality is as simple as implied by an assessment model, it is unlikely to perform adequately for more realistic representations of uncertainty. Basing an operating model on the current assessment model has arguably the lowest demands for knowledge and data.

II The operating model is a model that represents all of the available (and valid) data, and its estimated parameters depend almost exclusively on the data, for the case under consideration (such examples would include using maximum likelihood estimation or a Bayesian analysis with “non-informative” priors). This approach is based on the idea that only data matter when considering future events and assumes that all data sets are available. The operating model need not be identical to the models underlying the assessments used as part of the management strategy. This approach assumes that, with no information to the contrary, the future will be similar to the past, which is a strong, and most often unrealistic, assumption.

III As for (II) except that, in Bayesian models, informative priors describe in a formal probabilistic way a priori degrees of belief in parameters and processes based upon the knowledge of scientists or experts. Such priors may come from meta-analytic or Monte Carlo methods. This is still a data-orientated approach, but data sources other than those for the fishery under consideration, have an impact when conditioning the operating model.

IV As for (III) except that the emphasis is on expert beliefs and other a priori information about the processes that may affect the management systems in the future (i.e. the focus is on the future, not on fitting historical data). This is a less data-, and more hypothesis-orientated approach. For example, climatic change studies may show that a regime shift is possible (even though one has never been seen in the historical data sets) and should be taken into account when selecting ways to provide management advice. It is important therefore that operating models are flexible so that they can deal with such factors.

A major challenge for fisheries science is to develop a framework for scientific advice that comprehensively accounts for key uncertainties and risks while supporting the sustainable exploitation of marine living resources and maintaining an economically viable fishing industry. An important principle when developing such a framework is robustness to uncertainty since, although it is seldom possible to predict the response of fish populations to management with any great degree of accuracy, it is possible to work out, what strategies will on average work best, i.e. what management option is more robust.

It has also been reported that many ICES scientists involved in stock assessment working groups are experiencing morale problems rooted in a feeling that too often all they are doing is “turning the crank” on assessments (Wilson and Hegland 2005). Instead they would prefer a greater scientific focus and various combinations of reforms such as longer management cycles, development of management strategies that incorporate alternative management measures, fleet, fisheries-based and ecosystem-based approaches and more interaction about advice with managers.

FLR is rising to these challenges on both the operational (e.g. by providing tools for stock assessors, managers and others as part of the advisory process) and strategic (e.g. what choices to make with respect to improving knowledge and/or control in order to better meet management objectives) levels. It is doing this by providing a tool that will enable the crank to be turned faster while also allowing and encouraging greater exploration of the data and testing of assumptions. It will also help to provide tools that can be used to develop alternative assessment/management tools and strategies that are able to meet a broad range of management objectives and are robust to uncertainty about a range of processes.

By using R, an open source initiative, and adopting an open source license and development model, FLR improves transparency and scientific review, in contrast with black box methods where the source code is hidden. Open source also encourages active participation and blurs the distinction between developers and users by allowing participation in the development process. This has been shown to improve the quality of the code and appears to be more in agreement with the coordinated effort that many research projects assume. This is also helped by the extensible nature of the OOP mechanism in FLR, which means that new applications can be developed from existing ones. For example the FLAssess package for stock assessment which contains routines for age-based stock assessment for VPA, forward projection, data input output of parameters, diagnostics and plotting. These routines are all used by particular implementations of VPA such as ICA, XSA & Adapt.

An example of the types of results that such studies can give are shown by the study on North Sea cod (Kell et al, 2005) that evaluated the robustness of short-term recovery and long-term management strategies to different but equally plausible climatic change scenarios. In the short term, climate change had little effect on stock recovery, which depends upon reducing fishing effort to allow existing year classes to survive to maturity. In the long term, climate change has greater effects on stock status, but higher yields and biomass could be expected if fishing mortality was reduced. The implications of climate change for biological reference points of North Sea cod depended upon the mechanism through which temperature acts on recruitment, i.e. on juvenile survival or carrying capacity. The study did indicate that reference points based on fishing mortality appear to be more robust to uncertainty than those based on biomass. However, it was not possible to distinguish between these processes using stock assessment data sets alone.

Historical time-series of Atlantic bluefin tuna (Thunnus thynnus) catches from Mediterranean trap fisheries have shown spectacular long-term fluctuations in catches (Ravier and Fromentin 2001) with cycles of between 100 to 120 years synchronous around the Mediterranean and between sites more than 2000 km apart. Such long-term fluctuations could arise from changes in recruitment or in migration patterns (Ravier and Fromentin 2004). Fromentin and Kell (submitted) showed that identifying the underlying process that generated such long-term variations in bluefin tuna catch is of key importance, because distinct processes lead to different perceptions of stock status. Kell and Fromentin (submitted) then evaluated the performances of VPA based management procedures with respect to: (i) their ability to provide good estimates of MSY, FMSY and BMSY, and (ii) provide estimates of stock status and exploitation level relative to these MSY based targets. It was found that reference points appeared to be more robust to uncertainty about the true dynamics than absolute estimates and trends in F and SSB. F-based reference points were less biased and more precise than biomass- and/or yield-based reference points and F0.1 appeared to be the best proxy for FMSY. However, F0.1 cannot indicate past and current levels of exploitation relative to FMSY when there is uncertainty about the dynamics. An FMSY management strategy based on ADAPT-VPA did not outperform a simple strategy based on a size limit. It was concluded that the performances and robustness of distinct management strategies strongly depend on biological processes (i.e. the underlying dynamics).

However, the two strategies imply also very different management objectives. For example, a size limit will reduce effort or yield for certain fleets disproportionally, whereas an F0.1 strategy implies an equal cut in effort by all fleets. The choice of a strategy cannot therefore be decided on a purely scientific basis, but rather through its performance relative to the main management objectives, fisheries constraints, and biological and ecological processes postulated (Powers 2005, Kell and Fromentin submitted).

This example illustrates that modelling the management of fish populations and fishing fleets requires both biological and economic models, since the former focus on the response to management of the populations and the latter on that of the fleets. For example if two policies have the same biological impact but different ones in economic terms, then an economic impact analysis can help derive a preferred option. For example a reduction on fishing mortality implemented as an effort reduction, may have the same biological effect regardless of whether it is implemented by limiting days at sea or reducing fleet size. However the economic consequences and hence fishers response to the two alternative management measure would be very different. Particularly since if a policy sends a fleet bankrupt, then it is unlikely to get implemented in law or practice due respectively to political pressure or non-compliance respectively. Enforcement costs are also significant and so the benefits of a policy may not outweigh the costs. There is therefore an increasing need to build bio-economic models to perform both cost benefit analyses of enforcement schemes and to conduct impact analyses in order to decide upon the best way to implement management objectives. The cost of computer simulation is much less than the cost of collecting data and the value of forgone yield due to bad management. The approach has success fully been used for small stocks, for example, the Blackwater herring (Roel et al ?), which enabled assessment and management costs to be reduced and for the stock to achieve Marine Stewardship Council certification.

The FLR framework can also be used to develop methods to quantify the value of investing in good management in order to develop a competitive industry. Development is well under way for the bio-economic components of the evaluation framework FLR (Mardle et al.) and its application to a variety of case studies is currently in progress. For example the performance of effort and catch based systems including the cost of appropriate data collection and enforcement schemes will be assessed under this framework.

There are two main areas were FLR is or is intended to be applied within an ecosystem context a) testing the robustness of simple assessment/management rules given that species interactions are occurring and b) to help develop indicator based management systems to assess the impacts of fishing on ecosystems.

Ayim & Gaichas (2006) noted three important sources of uncertainty in multi-species models i) structural uncertainty e.g. aggregation in the food web; ii) functional uncertainty in predator/prey relationships; and iii) data uncertainty. There are often insufficient data to decide upon the main interactions between species or describe the response of individual species to management, and even when data are available, our knowledge of the functional form and precise dynamics of the relationships among species limits our ability to use them in models to provide management advice directly. Therefore the importance of such models will be allow e a range of alternative operating models, with different assumptions, to be developed. Only in that way will it be possible to ensure that the full uncertainty is captured.

Ayim & Gaichas (2006) also pointed out that there are two basic approaches to multi-species modelling i.e.

1. “Minimum Realistic Modeling” (Punt and Butterworth 1995), e.g. adding complexity in a piece meal fashion to improve fits to the data, an example is Multi-species Virtual Population Analysis (MSVPA; Sparre 1991) which extended single species VPA by including predator/prey interactions to estimate natural mortality.
2. “Big Picture”; i.e. models of “the whole ecosystem” or, in a predator/prey context, the whole food web, for example Ecopath with Ecosim Christensen et al. 2005.

Distinction should be made between the uses of “Minimum Realistic” and “Big Picture” models, the main use of models such as MSVPA has been used to improve existing single-species models, while “Big Picture” type models have been used mainly for the exploration or evaluation of hypotheses. It is envisaged that in the future “Big Picture” models will be used to evaluate the minimum level of realism needed when providing management advice, i.e. to evaluate the benefits of adding complexity rather than adding complexity for complexities sake.

For example while the development of a full multi-species model even for just the North Sea is presently an unattainable goal, it may be possible to build on work already carried out to develop limited multi-species model that incorporate key species. In this way our ability to model species interactions which take account of the impact of variations in the abundance of predators and prey for specific fisheries for depleted species such as cod may be improved. Multi-species models may also be used to test the robustness of simpler assessment/management rules before implementation, in particular for species and fisheries where there are important interactions but insufficient data to provide traditional advice.

MSE is increasingly being used to design management strategies for achieving fishery ecosystem objectives (Sainsbury et al. 2000) and in particular to help develop indicator based management systems to assess the impacts of fishing on ecosystems. For example MSE has been applied to evaluate the performance of state indicators in an Australian fishery, where Fulton et al. (2004a, b and 2005) used a relatively complex deterministic model to describe ecosystem dynamics. They then used a sampling model to generate data with realistic measurement uncertainty (bias and variance) for a given sampling design (location and timing) to produce the data required to calculate state indicators. Simulated data were collected for different levels of fishing and fishing combined with other human activities. The performance of indicators derived from the data was assessed in terms of the indicators capacity to track properties of interest. Indicator performance can be measured as the ability of indicators to detect or predict trends in attributes where the true values are known from the models.

A similar system is to be evaluated using FLR in order to develop an EA in the North Sea which will benefit from a relatively good understanding of biological processes and a variety of models already developed in FLR. This is therefore thought to be an ideal system in which to test the implementation of an EA based on indicator systems. It will also allows us to assess how effectively management could be applied in data poor circumstances by comparing the performance of management systems based on suites of linked pressure-state and response indicators with those based solely on routine monitoring of pressure and infrequent monitoring of some aspects of state.

As Kell et al (in press) pointed out, although MSE is a powerful tool, ultimately the aim is to improve the quality of management. Importantly, the MSE approach is intended to do so, not by making it more complex, but by identifying a robust management framework that can handle the often conflicting and poorly defined management objectives, account for many of the uncertainties that are often ignored in the conventional approach, and aid in strategic decision making.

Even developing and implementing an MP will not necessarily improve the quality of assessments, which may still need to be conducted on an annual basis. Rather, it will provide a way by which management can take the unavoidable uncertainties and possible disasters of assessment methodology into account. In this respect, it links together the management and assessment frameworks, and clarifies the roles of the main players, as is demanded in several jurisdictions (e.g., the CFP of the European Union).

As noted by Butterworth & Punt (1999) MSE requires the development of special software to implement the operating model and the model for data generation. Until recently there were no software packages that implemented ‘generalised’ operating models, and even those proved to be not as generic as it was hoped. Hopefully the FLR framework should increase the ease with which the OMP approach can be applied to new fisheries. However, it will never be possible to combine all bio-economic models into a single software program, since experts will never agree about the appropriate way to model population and fleet dynamics or the relative importance of the various processes. However, FLR provides a basis within which different models can be developed, integrated and compared, and their performance evaluated. By using R, a widely used statistical modelling environment, collaboration between and within disciplines will be increased, hopefully making it no longer necessary to have to use software developed by a single individual with a particular view of the way the world works. By following the open source credo it is hoped that transparency will be increased and that the work carried out by others will benefit us all.

Aydin.K.Y. and Gaichas. K.Y. (2006), In defense of complexity:Towards a representation of uncertainty in multispecies models IWC SC/58/E48

Bosford, L.W., Castilla, J.C. and Peterson, C.H. (1997) The management of fisheries and marine ecosystems. Science 277, 509-515.

Butterworth, D.S. and A.E. Punt (1999). Experiences in the evaluation and interpretation of MPs. In Confronting Uncertainty in the Evaluation and Implementation of Fisheries-Management Systems. A.I.L. Payne (Ed.). ICES J. mar. Sci. 56: 985-998.

Butterworth, D.S., Cochrane, K.L. and J.A.A. De Oliveira (1997). MPs: a better way to manage fisheries? The South African experience. In Global Trends: Fisheries Management. Pikitch, E.K., Huppert, D.D. and M.P. Sissenwine (Eds). Amer. Fish. Soc. Symp. 20: 83-90.

Chambers (2000), Programming with data. Wiley, New York.

Christensen, V., C. Walters, and D. Pauly, 2005. Ecopath with Ecosim: A User’s Guide. Fisheries Centre, University of British Columbia, Vancouver. November 2005 edition, 154 p. (available online at www.ecopath.org)

Cochrane, K.L., Butterworth, D.S., De Oliveira, J.A.A. and B.A. Roel (1998). MPs in a fishery based on highly variable stocks and with conflicting objectives: experiences in the South African pelagic fishery. Revs Fish Biol. Fish. 8: 177-214.

Cooke, J.G. (1999). Improvement of fishery-management advice through simulation testing of harvest algorithms. In Confronting Uncertainty in the Evaluation and Implementation of Fisheries-Management Systems. A.I.L. Payne (Ed.). ICES J. mar. Sci. 56: 797-810.

Cotter, A.J.R., Burt, L., Paxton, C.G.M., Fernandez, C., Buckland, S.T. and Pan, J.-X. (2004) Are stock assessment methods too complicated? Fish and Fisheries 5, 235-254.

FAO (1996). Precautionary approach to capture fisheries and species introductions. FAO Technical Guidelines for Responsible Fisheries. No. 2. FAO, Rome [Online]. Available from the World Wide Web: <http://www.fao.org/> [Accessed 6 January 2006].

Francis, R.I.C.C. and R. Shotton (1997). ‘Risk’ in fisheries management: a review. Can. J. Fish. Aquat. Sci. 54: 1699–1715.

Fromentin, J-M and L.T. Kell. (submitted). Consequences of variations in carrying capacity or migration pattern for the perception of Atlantic bluefin tuna (Thunnus thynnus) population dynamics.

Fournier, D.A., Hampton, J. and Sibert, J.R. (1998) MULTIFAN-CL: a length-based, age-structured model fo fisheries stock assessment, with application to south Pacific albacore, Thunnus alalunga. Canadian Journal of Fisheries and Aquatic Science 55, 2105-2116.

Fulton, E.A., Smith, A.D.M., Webb, H. and Slater, J. (2004a) Ecological indicators for the impacts of fishing on non-target species, communities and ecosystems: review of potential indicators. Australian Fisheries Management Authority Final Research Report R99/ 1546, 116pp

Fulton, E.A., Fuller, M., Smith, A.D.M. and Punt, A. (2004b) Ecological indicators of the ecosystem effects of fishing: final report. Australian Fisheries Management Authority Final Research Report R99/ 1546, 239 pp

Fulton, E.A., Smith, A.D.M. and Punt, A.E. (2005) Which ecological indicators can robustly detect effects of fishing? ICES Journal of Marine Science 62, 540-551.

Garcia, S. and de leiva Moreno, I. (2003) Global overview of marine fisheries. In: Responsible fisheries in the marine ecosystem (eds M. Sinclair and G. Valdimarsson), Rome FAO & CABI publishing, Wallingford, UK, pp. 1-24.

Gislason, H., Sinclair, M., Sainsbury, K. and O’boyle, R. (2000) Symposium overview: incorporating ecosystem objectives within fisheries management. ICES J. Mar. Sci. 57, 468-475.

Grafton, R.Q., Arnason, R., Bjørndal, T. et al. (2006) Incentive-based approaches to sustainable fisheries. Canadian Journal of Fisheries and Aquatic Science 63, 699-710.

Hammond, P. S. and G. P. Donovan (in press). Development of the IWC Revised Management Procedure. Journal of Cetacean Research and Management, Special Edition 4.

Hillary et al., in prep. Simulating observation error in fisheries indices using the FLR package.

Hilborn, R. 2003. The state of the art in stock assessment: where we are and where we are going. Scientia Marina 67 (Supplement 1):15-21.

Hilborn, R., Punt, A.E. and Orensanz, J. (2004) Beyond band-aids in fisheries management: fixing world fisheries. Bull. Mar. Sci. 74, 493-507.

ICES 2004. Report of the Working Group on Methods of Fish Stock Assessments. ICES Document CM 2004/D:03 Ref. G, ACFM

Jennings, S. (2004) The ecosystem approach to fishery management: a significant step towards sustainable use of the marine environment? Marine Ecology Progress Series 274, 279-282.

Kell, L. T., O’Brien, C. M., Smith, M. T., Stokes, T. K., and Rackham, B. D. 1999. An evaluation of management procedures for implementing a precautionary approach in the ICES context for North Sea plaice (Pleuronectes platessa L.). ICES Journal of Marine Science, 56: 834-845.

Kell, L. T., Pastoors, M. A., Scott, R. D., Smith, M. T., Van Beek, F. A., O’Brien, C. M., and G. M. Pilling (2005a). Evaluation of multiple management objectives for Northeast Atlantic flatfish stocks: sustainability vs. stability of yield. ICES J. mar. Sci. 62: 1104-1117.

Kell, L.T., Pilling, G.M., Kirkwood, G.P., Pastoors, M., Mesnil, B., Korsbrekke, K., Abaunza, P., Aps, R., Biseau, A., Kunzlik, P., Needle, C.,

Roel, B.A. and C. Ulrich-Rescan (2005b). An evaluation of the implicit management procedure used for some ICES roundfish stocks. ICES J. mar. Sci. 62: 750-759.

Kell, L.T., Pilling, G.M., Kirkwood, G.P., Pastoors, M., Mesnil, B., Korsbrekke, K., Abaunza, P., Aps, R., Biseau, A., Kunzlik, P., Needle, C., Roel, B.A. and C. Ulrich-Rescan (2006). An evaluation of multi-annual management strategies for ICES roundfish stocks. ICES J. mar. Sci. 63: 12-24.

Kell, L.T. and Fromentin, J-M. (submitted). Evaluation of the robustness of MSY-based management strategies to variations in carrying capacity or migration pattern of Atlantic bluefin tuna Thunnus thynnus.

Kell, L.T., De Oliveira, J.A., Punt, A., McAllister, M., and Kuikka, S. In Press, Chapter 15 – Operational Management Procedures: An Introduction to the Use of Evaluation Frameworks. In Motos, Lorenzo and D.C. Wilson (Eds) 2006. The Knowledge Base for Fisheries Management. Amsterdam: Elsevier.

Kirkwood, G.P. and A.D.M. Smith (1996). Assessing the precautionary nature of fishery management strategies. In Precautionary Approach to Fisheries, Part 2: Scientific Papers. FAO Fisheries Technical Paper 350/2: 141–158.

Mardle, S., L. Kell, T. Hutton, R. Scott and J-J. Poos (2006). (Bio-)Economic Modeling in a Generic Simulation Framework: The FLR Concept. Submitted/ presented to the ICES Galway Management Strategy Symposium (27-30 June 2006).

McAllister, M.K., Starr, P.J., Restrepo, V.R. and G.P. Kirkwood (1999). Formulating quantitative methods to evaluate fishery-management systems: what fishery processes should be modelled and what trade-offs should be made? In Confronting Uncertainty in the Evaluation and Implementation of Fisheries-Management Systems. Payne, A.I.L. (Ed.). ICES J. mar. Sci. 56: 900-916.

Michielsens, C.G.J., McAllister, M.K., Kuikka, S., M., Pakarinen, T., Karlsson, L., Romakkaniemi, A., Perä, I. and Mäntyniemi, S. (2006). Bayesian state-space mark-recapture model to estimate fishing mortality rates within a mixed stock fishery. Can. J. Fish. Aquat. Sci. 63:321-334.

Pauly, D., Christensen, V., Guénette, S. et al. (2002) Towards sustainability in world fisheries. Nature 418, 689-695.

Peterman, R. M. (2004). Possible solutions to some challenges facing fisheries scientists and managers. ICES J. mar. Sci. 61: 1331-1343.

Porch, C.E., Eklund, A-M. and Scott, G.P. (2006). A catch-free stock assessment model with application to goliath grouper (Epinephelus itajara) off southern Florida Fish. Bull. 104:89–101 (2006).

Punt, A.E. and D.S. Butterworth 1995. The effects of future consumption by the Cape fur seal on catches and catch rates of the Cape hakes. 4. Modelling the biological interaction between Cape fur seals (Arctocephalus pusillus pusillus) and Cape hakes (Merluccius capensis and M. paradoxus). S. Afr. J. Mar. Sci. 16: 255-285.

Punt, A.E. (in press). The FAO Precautionary Approach after almost 10 years: have we progressed towards implementing simulation-tested feedback-control management systems for fisheries management? Nat. Resour. Model.

Ravier, C., and Fromentin, J-M. 2001. Long-term fluctuations in the eastern Atlantic and Mediterranean bluefin tuna population. ICES J. Mar. Sci. 58 (6): 1299–1317.

Ravier, C., and Fromentin, J-M. 2004. Are the long-term fluctuations in Atlantic bluefin tuna (Thunnus thynnus) population related to environmental changes? Fish. Oceanogr. 13 (3): 145–160.

Rosenberg, A.A. and Restrepo, V.R. (1994) Uncertainty and risk evaluation in stock assessment advice for U.S. marine fisheries. Canadian Journal of Fisheries and Aquatic Science 51, 2715-2720. Rowe (1994) &

Sainsbury, K.J., Campbell, R.A., Lindholm, R. and A.W. Whitelaw (1997). Experimental management of an Australian multispecies fishery: examining the possibility of trawl-induced habitat modification. In Global Trends: Fisheries Management. Pikitch, E.K., Huppert, D.D. and M.P. Sissenwine (Eds). Amer. Fish. Soc. Symp. 20: 107-112.

Sainsbury, K.J., Punt, A.E. and A.D.M. Smith (2000). Design of operational management strategies for achieving fishery ecosystem objectives. ICES J. Mar. Sci. 57: 731–741.

Shepherd, J. G. 1999. Extended survivors analysis: an improved method for the analysis of catch-at-age data and abundance indices. ICES Journal of Marine Science, 56: 584e591.

Sinclair, M., Arnason, R., Csirke, J. et al. (2002) Responsible fisheries in the marine ecosystem. Fisheries Research 58, 255-265.

Sissenwine, M.P. and Murawski, S. (2004) Moving beyond ‘intelligent tinkering’: advancing an ecosystem approach to fisheries. Marine Ecology Progress Series 274, 291-295.

Sparre, P. 1991. Introduction to multi-species virtual population analysis. ICES Mar. Sci. Symp. 193: 12-21

Wilson, Douglas Clyde and Troels Jacob Hegland 2005 An Analysis of Some Institutional Aspects of Science in Support of the Common Fisheries Policy Project Report for Policy and Knowledge in Fisheries Management. CEC 5th Framework Programme No. Q5RS–2001-01782. Institute for Fisheries Management. Publication No. 126