Blog

Top 10 Modeling Mistakes for Community Banks

By AMG

In our discussions with clients, prospects, and regulators, we get a look at a lot of interest rate risk models being used by community banks.  In the past the modeling process was a "horse shoes and hand grenades" kind of approach, and as long as banks had a basic understanding of the process and were in the right ballpark, they got a free pass in exams.  However, those days are long gone, as the expectations are now much higher.  The guidance released in January of 2010 raised the bar considerably, and the level of sophistication is now expected to be much higher even in community banks with fairly simple balance sheets.  After seeing the process used by dozens of community banks, I have compiled a Top 10 List of the most common modeling mistakes.  These mistakes are also among the most common cited in exams, so when review your process, it makes sense to include these items in your check list.  Without further ado, here is the Top 10 List:
 

1. The data is not properly reconciled, with the most common errors found in mismatches between the balances listed in the loan and deposit data files and the balances in the general ledger files.  Most of our clients have files that do not balance perfectly, which is generally not a problem if the balances involved are not material.  However, you should have a process for comparing these balances and a threshold where you correct significant errors.
2. The major assumptions being used are not statistically significant, or are not even based on a quantitative analysis.  The big assumptions (pricing betas, non-maturing deposit lives, and prepay speeds) can change both the direction and the magnitude of the exposure.  For this reason, they must be right, and need to have a statistical basis.  Banks that do not have the resources or skill set to do this in house often use educated guesses or industry averages.  Examiners have been less forgiving of this approach, and banks should look into spending the necessary money to get good assumptions.
3. The board is not familiar enough with the modeling process, including the model's objectives, limitations, and major assumptions.  Make sure you include your directors enough that they are generally aware of the model and its implications on their decisions.
4. The forecast is not included in board reporting.  Most board reports are backwards looking, and only analyze past results.  The bank is generating a regular forecast, and a knowledge of what is expected to happen in the coming months should be a part of the board's strategic discussions.  This also ensures that there are no major surprises as to performance when rates move.
5. The assumptions for pricing betas are not updated on a frequent basis.  Many banks do an analysis of how their loan and deposit offering rates move in relation to market interest rates when they set up their model, and then do not update those models unless it becomes an issue with examiners.  In reality, these pricing relationships are dynamic and will change over time in response to markets and competitive pressures.  The model should reflect this, and this assumption needs to be updated on a regular basis.
6. The model uses industry averages for assumptions about average lives on non-maturing deposits.  Every bank will have deposits that behave differently depending on demographics and the bank's competitive environment.  Based on our experience, customer behaviors can vary wildly between banks, and using industry averages can create skewed results for market value of equity calculations.
7. The model results are not backtested on a regular basis.  The guidance is fairly clear about the expectations on backtesting and validating the model.  At a bare minimum the projected rates on each section of the balance sheet should be compared to the actual results on a quarterly basis.  If the model is off by a meaningful amount, then the major underlying assumptions need to be backtested.  On a related note, those that are doing the backtesting are often not having it done independently.  This can be difficult in smaller banks, but an effort should be made to have someone not directly involved in the modeling process checking the results.
8. The model uses stale assumptions for prepayment speeds on loans and securities.  Most banks use reasonable prepayment speed assumptions on securities, but a majority are using bad data for prepayment speeds on loans.  One of the more common sources has been the prepayment data compiled by the OTS.  However, this data is stale, and often does not apply to a bank's loan portfolio since it is compiled in the thrift industry.  Prepay speeds change quickly in today's markets, and models need to use the freshest data possible, as these assumptions can have a substantial impact on the outcomes of the simulation.  This is an issue that will be reviewed by many banks since the OTS data will not be available in the future.
9. Liquidity reports are not incorporated in the report package.  Liquidity and interest rate risk are very closely related, and banks should avoid looking at either risk in a vacuum.  Incorporating liquidity measures in the report package ensures that both factors are considered in the decision making process.
10. Knowledge of the model is concentrated in one employee.  Interest rate risk models have become increasingly complex, and for this reason, knowledge of the inner workings of the model often reside with one employee.  This is especially true in smaller banks.  However, in order to have proper controls and to have the integrity of the model properly questioned and reviewed, multiple employees need to have some working level knowledge of the model and the major inputs and outputs.