Commentary on Actuarial Standard of Practice No. 56 – “Modeling”
Greg Frankowiak

Commentary on Actuarial Standard of Practice No. 56 – “Modeling”

The Actuarial Standards Board (ASB) of the American Academy of Actuaries (AAA) recently announced the release of the new Actuarial Standard of Practice (ASOP) 56, entitled “Modeling.” This new standard will be applicable to actuaries governed by the AAA for modeling work performed on or after October 1, 2020.

There are roles in the insurance industry, other than actuaries, who build, implement, and monitor predictive models.  And although only actuaries are technically bound by this new ASOP, there is a lot of valuable guidance that this ASOP provides however, regardless of role.

It is also true that getting to the point of releasing the final version of ASOP 56 was no small feat. The history of modeling ASOPs is quite lengthy. According to the AAA website (www.actuary.org), work first began on a standard for modeling in the late 1990s. Different than ASOP 56, this work was more aligned to develop a standard for actuaries to follow if they were using tools such as hurricane or earthquake models that were developed by non-actuarial professionals. The result of this work was ASOP 38, “Using Models Outside the Actuary’s Area of Expertise,” approved in June 2000. Since then, the development and use of predictive models throughout the insurance industry has exploded, and it became commonly understood that a more comprehensive modeling standard was needed.

Efforts truly started in 2010 with a draft standard focused on modeling for life insurance and annuities. That evolved into a 2012 task force focused on developing what eventually became ASOP 56. The ASB put in considerable work to develop a modeling standard that was comprehensive and useful, but general enough to apply to many different modeling situations.

The first exposure draft for the Modeling ASOP was released in June 2013. Three other exposure drafts then followed, with the latest released in December 2018. Each exposure draft allowed for a comment period during which interested parties could submit their thoughts and suggestions on the latest draft. Over these four drafts, dozens of comment letters were received by the ASB, showing a significant level of interest in the topic. 

As someone who has been heavily engaged in predictive analytics for well over a decade, I was one of the individuals who participated in discussions and comment letters on the draft ASOPs. Needless to say, working through all of the comments and developing a relevant standard must have been no small task, and the committees that developed ASOP 56 deserve the most sincere thanks and kudos.

Modeling ASOP 56 follows the general structure of a typical ASOP with Section 1 describing the purpose and scope, Section 2 providing definitions of relevant terms, and Section 4 highlighting important information on communication and disclosures. It is Section 3, however, that really provides the substance of the content. Section 3 does a very good job of illustrating what could be considered technical and business considerations that an actuary should follow when working with predictive models.

One of the first statements in Section 3 is that the actuary “should understand the model’s intended purpose.” This simple statement sets the stage for so much of the work related to predictive modeling, but unfortunately, can often be overlooked. (In fact, it highlights the very beginning of the Modeling Lifecycle—see article linked here: Modeling Lifecycle.) 

Section 3 notes that the actuary should ensure that the model structure, data, and assumptions also align to its intended purpose. This is a test of both the actuary’s technical modeling skills and his or her ability to understand the business context in which the model is going to be used. As insurance experts, an actuary is often uniquely positioned to do just this. After all, even the most sophisticated model is virtually worthless if it does not successfully fulfill the business purpose for which it was built. Section 3 also requires that the actuary understand various strengths, weaknesses, or limitations of the model.

Importantly, Modeling ASOP 56 recognizes that there are professionals, other than actuaries (or other than just actuaries at a single company) who may be involved in the predictive modeling process. Given the complexity of some modeling techniques, and continually evolving technology and data sources, it is a difficult for any single person to be a solo model developer. (And, I would contend, the best models are typically ones where ideas have been exchanged during the process.) Consequently, ASOP 56 has several subsections that note the actuary’s responsibility when placing reliance on others.

Last but not least among its areas of focus, ASOP 56 spends a considerable amount of text discussing evaluation and mitigation of model risk. This includes some of the more standard, but important, aspects of testing model inputs, outputs and formulas from statistical and reasonableness perspectives. ASOP 56 also notes the importance of performing sensitivity testing of assumptions as applicable and evaluating model results across multiple models (such as with prior versions or “champion/challenger”). Model governance is also briefly discussed. And while not emphasized, I believe that model monitoring is another aspect that should be considered in this vein.

The entire lifecycle of predictive models can be lengthy, full of twists and turns, and potentially complex. The recently released ASOP 56 on Modeling does a very good job of highlighting many important considerations that actuaries should follow when performing this kind of work (and non-actuaries should also consider as best practices). At Pinnacle, we have the insurance and technical expertise and experience to cover a wide range of predictive analytics, and would welcome the opportunity to discuss with you how we can help make your modeling project a reality.

Greg Frankowiak is a Senior Consulting Actuary with Pinnacle Actuarial Resources, Inc. in Bloomington, Illinois and has over 20 years of property and casualty actuarial experience. Greg has extensive experience in predictive analytics for both pricing and underwriting, product management and strategy development, underwriting/operations, ratemaking for private passenger automobile and homeowners insurance, as well as regulatory and filing support and business intelligence. He is a Fellow of the Casualty Actuarial Society, a Member of the American Academy of Actuaries and a member of the CAS Ratemaking Committee.

«October 2020»
SunMonTueWedThuFriSat
27282930123
45678910
11121314151617
18192021222324
25262728293031
1234567