October 2012

The Market Conduct Analysis Framework
View as a Print-friendly PDF

• INTRODUCTION

All regulation of any industry requires regulators to know the marketplace they are regulating and make judgments concerning the activities of the entities they are regulating. To be efficient, the judgment of the regulator needs to be discerning. To be effective, the judgment of the regulator needs to be accurate. Analysis of the facts is critical to making correct judgments. Whether there is a rigorous analysis process in place, we are always analyzing information to make our judgments.

Financial regulation naturally lends itself to rigorous analysis. Financial regulation of insurance companies and other financial institutions relies on the formalized, systematic analysis of the financial statements of companies. For properly credentialed accountants and actuaries, the financial condition of a company is transparent and predictable.

Market conduct regulation concerns itself with the behavior of companies and individuals. Human behavior is anything but transparent and predictable. For many years, market conduct regulators had to rely on observations of the marketplace and their professional judgment without the type of robust data found in financial regulation. Like a police officer on the street, they knew their beat, they knew all the players and they knew when something did not appear to meet market conduct standards. Ultimately, however, market conduct regulators were reacting to behavior that had already occurred rather than engaging in proactive analysis based on formalized, systematic analysis.

In recent years, however, the regulation of the market conduct of insurance entities has been maturing toward a more rigorous and systematic analysis process. Pushed along by the 2003 report of the U.S. General Accounting Office titled, “Insurance Regulation: Common Standards and Improved Coordination Needed to Strengthen Market Regulation,” regulators have worked together through the NAIC to establish uniform market analysis practices and procedures. Today, the basic process for consistent and uniform market analysis is documented in the NAIC Market Regulation Handbook.[1]

• WHAT IS MARKET ANALYSIS?

Together, market regulators are working to create a structured and formalized system of collecting and organizing market and financial data. Development and improvement of the market analysis framework is coordinated through the NAIC Market Analysis Procedures (D) Working Group (MAP). Each NAIC jurisdiction appoints a market analysis chief (MAC) who oversees market analysis activities in their jurisdiction. The MAC is also responsible for ensuring the implementation of market analysis procedures approved by MAP and the NAIC Market Regulation and Consumer Affairs (D) Committee.

By collecting the right data and centralizing it for use by regulators from all jurisdictions, market analysis enables regulators to identify general market disruptions and specific market conduct problems as soon as possible within an efficient and effective regulatory framework. Determining what data to collect and how to interpret it is a continual effort, but the basic analysis framework and the types of data to be analyzed have already been established.

Market conduct issues may be contained in a local geographic area, spread over an entire region or be national in scope. They can range from a solitary underwriter or producer refusing to service a class of customers, to a company making a strategic decision to quickly non-renew policies and stop writing new business in a catastrophe-prone region or delay claims payments nationwide. Analysis, therefore, has to occur at these three levels.

At the local or state level, the state insurance departments have traditionally policed the companies on their own, occasionally asking for assistance from a company’s state of domicile if the company is a foreign insurer. On the local level, the analysis of a company’s market behavior has been driven chiefly by analysis of complaint data and information gleaned from earlier examinations.

The insurance industry, however, operates on a national and global scale. The larger a company becomes, the more likely it will conduct its business in a similar fashion across multiple jurisdictions. As market conduct issues cross state boundaries it becomes increasingly inefficient for each impacted insurance department to address each regional or national issue on its own. It is also costly for companies to work separately with multiple jurisdictions on the same issue. While the separate states still rely on their own market conduct departments to address wider-scale market conduct issues, they are increasingly coordinating and collaborating with insurance departments of other states that have the same or similar issues. To assist in collaborative efforts, market regulators work through the NAIC to standardize the market analysis process.

The goals of market analysis are three-fold. First, market analysis aims to find which of the thousands of companies writing insurance in the marketplace warrant further scrutiny by the insurance department. Second, market analysis will identify which market conduct concerns are causing (or may cause) consumer harm and should be addressed by the insurance department. There is a third, arguably more critical, goal: to predict non-compliant behavior before it happens. The third goal has yet to be achieved, and might not be achievable, but predictive market analytics, and the types of data needed to accomplish it, is often discussed whenever methods for improving market analysis are considered.

When good market analysis achieves these goals, the insurance department is more effective and efficient. It targets its resources at the companies with market conduct issues and uses the appropriate amount of resources necessary to bring the issues to fair and equitable conclusions.

To meet the first two goals, market analysts rely on a three-stage process that begins with the entire marketplace of companies in their state and narrows down the number of companies to a manageable number to review more intensively. As the analyst moves from one stage to the next, the data becomes increasingly more detailed.

Market conduct data can be described as either “summary” or “transactional.” Summary data provides a high-level overview of the company. It is a broad look at the operations of the company, such as the number of claims, the amount of premium or how many complaints have been made against the company. On the opposite end of the spectrum, transactional data is specific to each transaction conducted by the company, such as the details of each application for insurance, policy issued, claim handled or lawsuit defended by the company. Almost all of the data used by market analysts today is summary data. Complete transactional data is not currently available to market analysts, other than what might occasionally be requested from a company during an examination or through a specific data call.

• BASELINE MARKET ANALYSIS

The first stage of market analysis is to prioritize which of the entire set of companies in the marketplace warrant the most concern. Of course, it makes little sense to compare property/casualty companies to health or life companies. So, the first step is to segregate the companies by the type of business they are writing in the state. Fortunately, for purposes of reporting their financial annual statements, the data for companies is already split by line of business. All the market analysis tools available through the NAIC are organized by line of business.

The first stage of market analysis is often referred to as the baseline stage. In this stage, the analyst relies heavily on data at the most summary, aggregated level. The purpose is to compare as many companies as possible at the same time. Most of the states have developed scoring systems that put more weight on some data than others. For example, most insurance departments put greater emphasis on complaint information than on premium or loss trends. However, depending on the issues that are most important to the state at a given time, the weighting of the data will change.

  • Baseline: Summary data – marketplace (state, regional, national) – MAPT
  • Level 1: Summary data – company-specific (state and national) – MARS
  • Level 2: Summary and transactional data – company-specific – MARS
  • Continuum: Transactional data – company- and/or issue-specific – MITS

 

To assist market analysts and to make baseline analysis more uniform among the states, market regulators worked with NAIC staff to create the Market Analysis Prioritization Tool (MAPT). Any state market analyst with the proper security role can access this tool. For any of the 11 lines of business in any state or U.S. territory, a market analyst can create a single spreadsheet, organized by company that provides more than 100 separate data elements per company. These data elements can be directly compared and ranked among the companies.

Because MAPT is a prioritization tool, it does more than just aggregate data for market analysts. Built into the tool is a scoring system. For example, for the homeowners line of business, each company will be scored in 13 distinct categories, including complaints and regulatory actions. The higher a company scores in any category, the more concerns an analyst would have about the company in that area.

For example, MAPT’s complaint score for each company is the combination of scores given to the company based on its most recent complaint index and the complaint index over the past three years. [2]A percentile ranking of the companies is performed based on the size of the index for each company. The percentile ranking of a company determines its score. The higher a company’s complaint indices, the higher it sits in the percentile ranking. The higher the percentile ranking, the higher the complaint score will be. Because the score is based on recent activity and three-year activity, analysts will not focus on just the current companies with high complaint counts.

A similar process for scoring is used in all the categories. To support the scores, MAPT provides the analyst with the underlying data. For example, to support the complaint score, MAPT also provides the analyst with the count of the total number of complaints against a company, the count of confirmed complaints against a company (a confirmed complaint is a complaint that was coded with a disposition considered adverse to the company) and the market share of the company.

The scores in each category are added together to derive overall scores. There are three overall scores. An overall national score is calculated based on national data and an overall state score is calculated based on state data. Finally, a comprehensive overall score is calculated. This score is 75% of the state score plus 25% of the national score.

The features in MAPT serve the variety of market analysis needs discussed earlier. First, it is uniform—it uses the same scoring metric in every state. Each analyst is relying on the same types of data and the companies are prioritized consistently. Yet, recognizing the unique nature of each state’s market, an analyst can temporarily adjust the scoring to provide greater weight to data that is more important to the issues of concern.

Second, it is developed primarily as a state tool organized by line of business. The market analyst can only view companies one state at a time, and those companies must be related by line of business. However, recognizing that market concerns can cross state lines, MAPT provides both state-specific and national data for each company conducting business in the state. This allows the market analyst to broaden the scope and consider both regional and national issues.

Third, because it is a high-level overview of the entire marketplace, it relies entirely on summary data. Many analysts refer to MAPT as a “wall of data” because there is so much data provided for so many companies at one time. Nevertheless, MAPT is incapable of identifying, with certainty, companies that are misbehaving in the market. Its primary purpose is to prioritize the companies for more in-depth analysis. As the market analyst moves more deeply into analyzing particular companies, the data becomes less general and more specific. 

Regulators and NAIC staff developed MAPT with a heavy emphasis on the data available at the NAIC. At that time, much of the data at the NAIC was financial data. Data that could be considered strictly market conduct-related data was limited to complaints, market regulatory actions taken by the states and demographic information. Beginning in 2008, however, the NAIC became the central repository of data gathered through the Market Conduct Annual Statement (MCAS.) This has greatly increased the scope of data available to market analysts.

The current form of the MCAS was first collected in 2002 by eight states. It was designed to give the market analysts in those states the market conduct data that they were unable to obtain without a specific data call or examination of a company. It was a quick, uniform method of collecting market conduct data from all companies writing specific lines of business in a state. Like MAPT, the data collected could be compared, scored and used for prioritizing companies of the greatest concern to the market analyst.

For now, MCAS is limited to market conduct data for the auto, homeowners, life and annuity lines of business. The NAIC Market Analysis Procedures (D) Working Group is developing a long-term care insurance blank for MCAS and expects to add lines of business over the next few years.

For the auto and homeowners lines of business, MCAS collects underwriting data regarding non-renewals and cancellation activities. For claims, the data is collected by coverage type (collision, bodily injury, uninsured motorist, etc). Claims data includes how quickly claims are paid, the number of claims closed without payment and the number of lawsuits. Most of the data elements are collected with the intent of measuring prompt and fair policyholder service.[3]

For the life and annuity lines of business, MCAS collects information on new and replacement activity with a focus on the age of the insured/annuitant and surrender activity, particularly of policies that may incur a surrender charge. There are also claims questions for the life products that address the speed of claim settlement and the percent of claims compromised or denied. Suitability and policyholder service are the focus of the life and annuity MCAS.[4]

As a prioritization tool, MCAS ranks companies according to the level of concern to a market analyst. Ratios have been developed for each MCAS line of business utilizing the data elements obtained from the MCAS filing. There are seven private passenger auto and homeowners insurance ratios and eight life insurance and annuity ratios.[5] The assumption behind each of the ratios is that the higher the ratio, the more attention is required from the market analyst. The rankings for each ratio, therefore, reflect how high the company ratio is when compared to the other companies in the state that filed an MCAS. The company’s ranks for each ratio can be added together to arrive at an overall rank. A high overall rank means that a company has higher ratios than a company with a lower rank.

Besides being an effective prioritization tool, MCAS is also useful in analyzing an individual company. Every year, a company provides data regarding such areas as the speed of claim settlements and the numbers of surrendered policies. Over time, the data submitted by a company can be trended to allow an analyst to view multi-year changes in one direction or another. An analyst can also compare the data to the types of complaints filed against a company. The analyst may find correlations between the complaints and the reported data that substantiates consumer concerns. In addition, because most companies are required to submit MCAS data, the market analyst has a good idea what to expect from a company. For example, using four years of data, market analysts now know that the average time to settle a collision claim is around 15 days and that less than 2% of a company’s life claims should take longer than 60 days to settle once the proof-of-loss is complete and properly documented.

Baseline analysis as it currently exists has limitations. Scoring algorithms have been created to allow for a quick comparison and prioritization of companies but in almost all cases, and certainly within the MAPT, the algorithms have not been tested in a controlled way to judge their effectiveness. In addition, because the data being used is at a highly aggregated level, the reasons for an anomalous data point can be multifaceted and have no bearing on market conduct.

Baseline analysis tries to isolate companies that are outside the expected norms in the marketplace in a number of key indicators. Most baseline analysis is static and uses a snapshot of a company’s data as of a certain date. A company may be slowly trending in a negative direction but by amounts never enough to trigger a high score during the prioritization process. For example, a company may begin with a low complaint index that gradually creeps upward from less than .5 (less than half the complaints that would be expected for its market share) to 1 (complaint share equals market share) over the course of four years. It would not be flagged in any one year. Nevertheless, the gradual increase in the index could indicate a nascent market conduct issue that could be nipped in the bud before blossoming into a larger problem for consumers.

A good scoring algorithm should take into account the size of the company and the quantity of data available. Using complaints again as an example, a company with a small market share will generate a large complaint index if it receives even one complaint. Most good scoring algorithms will incorporate a threshold to eliminate companies that do not have a statistically valid data set. In the example above, the analyst may eliminate all companies with a premium less than $5 million or less than five complaints. This, however, eliminates the small companies which, of course, can also have market conduct problems that need to be addressed. The reverse of this is also a concern. A large company that dominates a line of business can impact what is considered the norm for a state or a region. It is often useful to segregate out small and large companies for separate analysis.

Baseline analysis cannot, and should not, be used to definitively identify companies with market conduct concerns. The data used in baseline is at too summary of a level to draw any conclusions with certainty. To attempt to bypass the second and third stages of the market analysis process would result in going down too many rabbit holes with a wasteful use of resources. A good baseline methodology, however, will winnow a marketplace of hundreds of companies to a manageable number for the department of insurance. This manageable number of companies can be analyzed to determine why they stand out in relation to the other companies in the marketplace.

• LEVEL 1 [6]

The second stage of the market analysis process that was designed by regulators through the NAIC is commonly referred to as a Level 1 review. This stage moves the focus of market analysis from the marketplace as a whole to a company specifically. As with the baseline process, this stage of review has long been a staple of the market analysis process. Prior to any examination or audit of a company, an insurance department has to know what they need to focus on during the examination or audit. This requires reviewing a company’s financial statements, the company’s communications with the insurance department and consumers’ communications with the insurance department.

Regulators converted this pre-examination analysis into two parts: a preliminary first part (Level 1) and an in-depth second part (Level 2). By developing the Level 1 and Level 2 analysis process, regulators succeeded in standardizing this analysis and incorporating it into a framework for collaborative actions by the insurance departments. They also made its use more prevalent by requiring jurisdictions to conduct a minimum number of Level 1 reviews.

The Level 1 analysis of a company was designed with the intent to be quick, yet thorough enough to isolate and focus an insurance department’s attention to specific operations of a company. The expected amount of time to be consumed by a Level 1 is no more than three to four hours. Like the baseline analysis, the data provided to analysts for use in the Level 1 review is summary data, but it is presented in greater detail.

For example, a company may stand out in the baseline analysis because of high complaint indices for the past three years. For purposes of baseline analysis, this is enough to flag the company for additional review. In the subsequent Level 1 analysis of the company, the market analyst will still only look at the total number of complaints, but the complaints will be organized by the coverage type, reason and final disposition. The analyst now has a much better idea whether the index is the result of claim handling, marketing, producer issues or something else.

The Level 1 review is a template with a series of questions that guides an analyst though a comprehensive review of data available in the NAIC’s market and financial information systems. There are 12 sections to the review, each section dealing with a specific area of concern for a company: operations and management; financial ratios; special activities; regulatory actions; examinations; market initiatives; premiums; market share; loss and expense ratios; resisted and unpaid claims; complaints; and MCAS results.

The analyst accesses this template through the NAIC Market Analysis Review System (MARS). MARS is located alongside the other financial and market system tools. Each section of the Level 1 review is populated with the data necessary to answer the questions about the company and links to the relevant NAIC financial and market databases. This ensures a uniform, standardized process of in-depth analysis that is repeatable, as necessary, on a company. Additionally, because the reviews are located in MARS, they can be reviewed by analysts in other jurisdictions. To ensure the quality of the reviews, every review is approved by the MAC of the jurisdiction prior to being made accessible to other state analysts. Only analysts with the proper security roles can view and enter reviews in MARS.

Prior to conducting the Level 1 review of a company, an analyst should review the reasons the company was chosen as a Level 1 candidate. If the company was identified in a baseline review because of increasing losses and defense cost expenses, the analyst will want to concentrate on these areas during the Level 1 review to determine the cause of the increases or the types of claims or issues that may be driving the costs upward. If a company’s MCAS filing created concerns that prompted the Level 1 review, the analyst will want to concentrate efforts on the MCAS issues that drew the company to the state’s attention. However, although analysts should have a focus during the review, they need to be thorough in the review of all areas of the company in order to not overlook other, possibly more critical, indicators of market conduct issues.

A well-performed Level 1 review should give a clear picture of the company’s overall financial condition and possible market conduct issues. In fact, the Level 1 review forces the analyst to draw a conclusion based on the results of their analysis. The analyst is provided with a menu of conclusion possibilities from which to choose. They range from “no further analysis is necessary” to recommendations for contacting the company, collaborating with other states regarding the company and even scheduling an examination. All conclusions need to be justified in final concluding comments.

A successful Level 1 review is not necessarily a review that uncovers a market conduct issue and recommends immediate action. In fact, more than half the time, Level 1 reviews conclude that no further analysis is necessary. This, too, is a benefit to the insurance department and to the company. Without tying up days, weeks and months of insurance department and company resources, the analyst develops an overview of the entire company and concludes that the jurisdiction’s resources are better served focusing on other, more critical, concerns.

However, more than one-third of the conclusions of Level 1 reviews in 2011 were either to take some regulatory action with the company (17% of the time) or to conduct a Level 2 review of the company (18% of the time). Because the Level 1 review uses summary data with a limited level of detail, it can sometimes be difficult to say with the necessary certainty that regulatory action is needed. Even if it is certain that there is an issue and what the issue is, additional supporting information is often required to be sure nothing is overlooked when the company is contacted.

• LEVEL 1[7]

The third stage of the market analysis process is the Level 2 review. As with the Level 1 review, an analyst will complete this review within MARS. This promotes uniformity and collaboration on market conduct actions. All Level 2 reviews approved by the MAC of the domiciliary jurisdiction are available for viewing by market analysts that have the appropriate security roles in the other states.

A Level 2 review relies on data that is more specific than in the previous two stages. It should be at the transactional level wherever available. The data utilized in the Level 2 review is often unobtainable through the NAIC market information systems and is often accessible only to regulators in the state that is conducting the Level 2 review. After the Level 2 review, when viewed in conjunction with the data from the previous stages of the market analysis, the analyst has as complete a picture of the company and its operations as possible, short of an actual examination.

The Level 2 review requires thoughtful responses from the analyst. Unlike the Level 1 review, there are no “yes” and “no” questions. The analysts are required to report their analysis findings in six core areas: consumer complaints; continuum activity; examinations; interdepartmental communications; market analysis; and regulatory actions. The MARS Level 2 template also provides space for the analyst to comment on the results of analysis in 15 additional, non-required, areas.[8]

Using data available to them from within their insurance department, an analyst can analyze specific complaints, the work product and transactional data available from current and recent examinations, as well as the results of current and previous market analysis. State analysts are encouraged to contact other regulators within their own department or other state insurance departments to build a complete profile of the company’s market conduct activities and issues.

The Level 2 review also includes an analysis of regulatory actions and continuum activity. To promote collaboration among the states, the design of each stage (Baseline, Level 1 and Level 2) includes both state and national details. By reviewing regulatory actions, the analyst becomes aware of issues already addressed by their state and other states. If other states are handling similar issues, it may be beneficial for the states (and for the company) that they be addressed uniformly at one time, rather than in a piecemeal fashion.

• CONTINUUM

Continuum activity is any regulatory response to a market conduct issue. The range of possible responses is referred to as a continuum because of the multitude of possible ways to work with companies in understanding and resolving market conduct concerns. The possible responses can range from a telephone call or letter to audits and examinations and everything in between, such as an interview or survey. As noted above, successful market analysis results in an efficient use of insurance department resources. Good market analysis will provide enough information to allow the insurance department to determine what type of regulatory response is needed to resolve a market conduct concern.

• CONCLUSION

The States regulate insurance entities to help ensure financial solvency and to promote a fair and competitive marketplace. Regulators rely on data gathered from a variety of sources to monitor the marketplace. The goal is to spot disruptions as early as possible and resolve them effectively and efficiently. To do so, the data must be organized, analyzed and evaluated. This is the responsibility of financial and market analysts.

Because the insurance market operates across state lines, regulators in each state must have confidence in the analytical abilities of the other states. Working through the NAIC, market conduct regulators have developed a three-stage process that all members of the NAIC are encouraged to use. The three-stage market analysis cycle promotes uniformity, completeness and collaboration between the states.

The three stages organize the market analysis cycle into a funnel shaped process moving from a large overview of the entire marketplace to single concerns about individual entities. At each stage of the process, the level of detail that the analyst uses moves from summary information to transaction-level detail. By the time the analyst completes the analysis, the insurance department should have enough detail to choose the appropriate next step(s) from among the continuum of regulatory responses.

Market analysis does not, and maybe cannot, have the same exactness of financial analysis because of the nature of what it is attempting to analyze: human behavior and the arena in which it takes place—56 NAIC jurisdictions with distinct and occasionally different market conduct regulations. Market analysts, however, continue to refine their methodologies in an attempt to be predictive. Regardless of the methodologies, predictive analytics are not possible without adequate data. The only data adequate to the task is transaction-level data. Regulators, particularly market analysts, must continue to push for greater access to transaction-level data from companies.


[1]The NAIC Market Regulation Handbook is available to regulators, industry and the public. It can be purchased through the NAIC Store at http://store.naic.org.
[2] A company’s complaint index is simply the company’s share of the total number of complaints filed against all companies divided by the company’s market share of premium. A complaint index of “1” means that a company’s share of all complaints is the same as its share of the market. An index higher than “1” means a company has a greater share of complaints than would be expected based on its market share. The complaint index can also be calculated based on the company’s market share of policies or market share insureds.

A market analyst should always be aware of the actual number of complaints against a company and whether they are confirmed. A company with a large number of complaints will, of course, warrant the attention of the market analyst. However, a large number of complaints, considered out of context, will not be used to draw conclusions. A company with a greater market share will usually have more complaints. The complaint index puts the quantity of complaints in the context of the company’s position in the marketplace.
[3]For 2011, insurance companies that write at least $50,000 in premium in private passenger auto or homeowners insurance in a jurisdiction were required to file a MCAS with that jurisdiction. A sample blanks for the private passenger auto and homeowners MCAS is available at: ADD LINK (I need to link these documents to the CIPR website so that readers access them. I plan on including all the attachments at the end of the “PDF” link of the article we include on the web page, we can use that web address here).
[4] For 2011, insurance companies that write at least $50,000 in life premium or annuity considerations in a jurisdiction were required to file a MCAS with that jurisdiction. A  sample blanks for the life and annuity MCAS is available at: ADD LINK
[5]The seven private passenger auto and homeowners insurance ratios are: (1) percentage of claims closed without payment; (2) percentage of claims unprocessed at the end of the period; (3) percentage of claims paid greater than 60 days from receipt; (4) percentage of company-initiated non-renewals to in-force policies; (5) percentage of company-initiated cancellations greater than 60 days from original issuance date; (6) percentage of company-initiated cancellations less than 60 days from original issuance date; and (7) percentage of lawsuits to claims closed without payment.

The eight life insurance and annuity ratios are: (1) percentage of replacements to policies issued; (2)percentage of replacements where the insured is older than 65 or the annuitant is older than 80; (3) percentage of surrender compared to policies issued; (4) percentage of surrenders where the policy was in-force less than 10 years; (5) percentage of deferred annuities issued to annuitants older than 80; (6) percentage of claims paid greater than 60 days from the proof of loss; (7) percentage of claims denied, compromised or resisted; and (8) number of complaints per 1,000 policies.
[6]Detailed information of each area of a Level 1 review is available in Appendix B of the NAIC Market Regulation Handbook. A copy of Appendix B is available on the CIPR website at: ADD LINK.
[7]Detailed information of each area of a Level 2 review is available in Appendix C of the NAIC Market Regulation Handbook. A copy of Appendix C is available on the CIPR website at ADD LINK.
[8]The additional areas of review for a Level 2 review are: departmental filings; dispute resolution activity; financial analysis; financial rating agencies; geographic analysis; human resource department; Internet/World Wide Web; legal information; NAIC bulletin boards; other governmental and quasi-governmental agencies; producer licensing; Special Activities Database; state-mandated items; trade publications and other media sources; and voluntary accreditation/certification programs.
Copyright © 2012 NAIC, All rights reserved.