Scorecards’ and Rating Models’ regular validation is vital for the optimal function of the decision tools. The operating environment of the models is not stable. Changes may come from various factors such as systems, policies and processes, customers’ profiles, behaviour patterns, economic environment, competition, marketing initiatives, etc. Thus, regular Scorecards’ validation is imperative in order to identify any shifts in the models’ performance and decide on potential actions.
One of the side-effects of the 2008 Financial Crisis was the increasing interest by Supervisory bodies on the models’ Development and Validations practices followed by Banks, for which directives, guidelines and consulting papers are routinely being issued since. The use of models for Provisioning purposes, as part of the adoption of the IFRS 9 Standard, further increased the importance of a regular and holistic Validation process for those models.
Furthermore, independency on the Validation Function has now become a prerequisite, along with the the establishment of multiple and independent Validation levels: Validation by the (internal) Validation Unit, by the internal Audit Unit, by third parties.
Why StatDec?
StatDec Scorecards’ and Rating Models’ Validation Services are customized to the model type and use of the Model. They include statistical tests and performance metrics commonly used in the industry, while more advanced statistical approaches are employed where needed. Furthermore, the Validation Framework may be customized to the Client’s specific requirements, such as existing validation framework, specific tests and support in coding for internal monitoring.
StatDec Validation Services are not just an array of statistical tests. Though usually emphasis is placed on the statistical side, we introduce in the validation process a business perspective by aiming to answer on questions such as:
- Is the model being used properly?
- Is the model discriminating risk?
- Are the decisions optimal?
- Are the risk estimates by score accurate?
- Are the estimates on client’s profile accurate?
- What are the next steps?
The aim is to provide knowledge and insights on the functions of the Validated Models but also on what is happening on the Portfolios themselves.
The validation tests are conducted as separate blocks.
| Qualitative | Quantitative |
|---|---|
| Score Inputs & Data Processing | Discrimination Power |
| Data Quality & possible limitations | Accuracy |
| Model implementation accuracy | Distribution Stability |
| Exclusion Analysis | Concentration Analysis |
| Definitions review | Model Inputs Analysis |
| Model Use | Model Use |
When examined as a whole, one can understand what is happening in the Portfolio.
Findings on systems, policies, marketing activities and collections can be identified and presented as part of the Scorecard Validation. In many occasions, focused analysis will take place based on findings throughout the analysis so as to track the mechanisms and reasons behind the observed patterns.
Results are intuitively benchmarked to expectations, as set on the specific portfolio but also making use of market experience on similar portfolios.
Validations are accompanied with a proposed customized action plan based on the findings of the analysis. Best practices are recommended, that are not necessarily restricted to Models’ changes.
The use of StatDec Validation Services allows for an expert and in-depth validation of an organization’s Model inventory. In situations that an independent validation or an additional validation layer is required, StatDec can support Businesses with professionalism and deliverables of the highest quality.