Application assessment
Set up indicators to measure the usability, cost, quality, performance, and risk of applications. Evaluate and score your business applications based on qualitative inputs. You can translate abstract information of applications based on surveys and assessments into more tangible concrete metrics. These assessments help you make strategic decisions on whether to replace or upgrade applications.
You can use existing assessment metric types or configure them per your requirements.
Framework setup for application assessment
You can create indicators and score profiles based on which you can assess your applications. Application indicators are business metrics that help derive application scores.
Enterprise Architecture is integrated with key applications in the ServiceNow platform to provide a deep insight into the applications. These integrations help you:
- Identify cost saving opportunities
- The Hierarchy of Segments in the Financial Management application tracks the cost allocations at the application level, which provides a complete cost breakdown for the application.
- Organize applications to determine their rationalization
- You can identify multiple applications assigned to the same application category, region, or business. This information helps you to know who is using the applications, the usage frequency, the application status, and make informed decisions.
- Identify opportunities for modernizing and investing in application
- You can identify applications that have contracts to renew, low usage, or low customer satisfaction based on surveys results.
Preconfigured indicators and their source applications
The preconfigured Enterprise Architecture indicators and the applications they have been sourced from help you to assess the applications across dimensions such as cost, quality, and risk. You can create additional indicators, apart from the preconfigured indicators, by copying and modifying them.
| Indicator name | Frequency | Type | Source | How is it calculated? | Description | Jobs |
|---|---|---|---|---|---|---|
| Facilities cost | Quarter | Custom Script | ITFM product. ITFM_Allocation_Aggregate table | Data will be available in the ITFM tables only after the financial modeling process is completed | Facilities cost for business application | |
| Hardware cost | Quarter | Custom Script | ITFM product. ITFM_Allocation_Aggregate table | Data will be available in ITFM tables only after the financial modeling process is completed | Hardware cost for business application | |
| Labor cost | Quarter | Custom Script | ITFM product. ITFM_Allocation_Aggregate table | Data will be available in the ITFM tables only after the financial modeling process is completed | Labor cost for business application | |
| Other cost | Quarter | Custom Script | ITFM product. ITFM_Allocation_Aggregate table | Data will be available in the ITFM tables only after the financial modeling process is completed | Other cost for business application | |
| Services cost | Quarter | Custom Script | ITFM product. ITFM_Allocation_Aggregate table | Data will be available in the ITFM tables only after the financial modeling process is completed | Services cost for business application | |
| Software cost | Quarter | Custom Script | ITFM product. ITFM_Allocation_Aggregate table | Data will be available in the ITFM tables only after the financial modeling process is completed | Software cost for business application | |
| Application TCO | Quarter | Custom Script | ITFM product. ITFM_Allocation_Aggregate table | Data will be available in the ITFM tables only after the financial modeling process is completed | Total application cost from all the buckets | |
| Application's Incident Count | Quarter | Custom Script | incident | Data will be available in the incident table only after the business application is associated to the incident. | Indicator that gets the count of all incidents associated to the business application tied to the scoring profile of which the indicator is part. | |
| Application's Instance – Incident Count | Quarter | Custom Script | incident | Gets incident count attached to all Application Instances, which are mapped to a business application and rolls it up to application. | Indicator that gets the count of all incidents associated with application
instances. The application instances, in turn, are associated to a business
application tied to a scoring profile of which the indicator is a part. The incident count is calculated first at the application instance or application service level, and then it is rolled up to the business application level. |
|
| Usage | Month | Query Condition | APM product. cmdb_ci_business_app table | Calculated from the Active User Count field | Number of user sessions and users for the application for a given fiscal period. | |
| Number of Incidents via Service | Daily | Performance Analytics | Mapped to
Source = Incidents.New (Incident table) |
Number of incidents opened today. Calculated from the Impacted Business Applications of the incident record. | Number of new incidents. Daily and historic data collection | [PA Incident] Daily Data Collection [PA Incident] Historic Data Collection |
| Number of Problems via Service | Daily | Performance Analytics | Mapped to
Source = Problems.New (Problem table) |
Problems created today. Calculated from the Service field of the problem record. | Number of problems opened today. Daily and historic data collection | [PA Problem] Daily Data Collection [PA Problem] Historic Data Collection |
| Number of Changes via Service | Daily | Performance Analytics | Mapped to
Source = Changes.New (change_request table) |
Number of changes with a registration date (change_request.opened_at) on collection date. Calculated from the Impacted Business Applications of the change request record. | Number of changes opened today. Daily and historic data collection | [PA Change] Daily Data Collection [PA Change] Historic Data Collection |
| Customer satisfaction (CSAT) | Quarter | Assessments | Assessment Metric Type: Customer Satisfaction Assessment Metric Category: CSAT |
Template NPS | ||
| Functional fit | Month | Assessments | Assessment Metric Type: Functional Fit Assessment Metric Category: Functional Fit |
Template Net Promoter Score (NPS) | ||
| Technical risk | Month | Assessments | Assessment Metric Type: Technical Risk Assessment Metric Category: Performance |
Technical risk captured through survey for the fiscal period. Template NPS |
||
| Technology Lifecycle Risk | Month | Custom Script | Assessment Metric Type: Functional Fit Assessment Metric Category: Functional Fit |
Get the technology lifecycle risk of a business application for a selected fiscal period. | ||
| Business value | Quarter | Assessments | Consolidation: Average | Template NPS | ||
| Total change hours | Month | Performance Analytics | Mapped to
Source = Changes.Closed (Change_Request table) Fields: Opened, Closed State = Closed, Business Application = any of the Enterprise Architecture Business Applications, Closed today |
Script: Change.CloseTime.Hours. All Change Requests closed today considered | Summed duration of closed changes for an application for the given fiscal
period. Time taken to close the changes in hours. Daily and historical data collection. |
|
- To set the properties, navigate to All > System Properties > All Properties and search for the following properties:
- Populate Impacted Services based on Affected CIs (com.snc.incident.refresh_impacted.include_affected_cis)
- Populate Business Application related list for incidents (com.snc.incident.populate_business_application)
- Populate the Business Application related list for change requests (com.snc.change_request.populate_business_application)
- To show up the .
- To see the impacted business applications for the Number of Incidents via Service and Number of Changes via Service indicators, you must refresh the Impacted Services and CIs related list for that record. For instructions, see Refresh impacted services and CIs for Change and Refresh impacted services and CIs for incident.
Performance Analytic indicators to measure application performance
Use performance analytic (PA) indicators to know the count of incidents, problems, and changes logged against a business application and use this insight to improve the performance of your applications.
Enterprise Architecture uses indicators that are sourced from Performance Analytics (PA). These indicators give a count of incidents, problems, changes, and the number of change requests that were closed on a given day. Follow the given order to run the PA jobs at the scheduled time, and get the scores of the indicators to evaluate the performance of your business applications.
Order in which to run PA jobs and generate scores
You should run the scheduled jobs in the following order:
- [PA Incident] Daily Data Collection.
- [PA Change] Daily Data Collection.
- [PA Problem] Daily Data Collection.
- [Enterprise Architecture Scheduled job] Load Application Indicators and compute Application Scores.
If there are historic data, then run them in the following order:
You require Performance Analytics Premium for Enterprise Architecture (com.snc.pa.premium.apm) plugin to retrieve historic data that are older than six months.
- [PA Incident] Historic Data Collection.
- [PA Change] Historic Data Collection.
- [PA Problem] Historic Data Collection.
- Regenerate Enterprise Architecture scores for required time period. This action deletes the existing scores including daily scores and generates new scores instead of just updating the existing scores.
Frequency at which indicator scores are generated
Scores are generated as per the scheduled run of the job that executes the script. If the indicator frequency is:
- Monthly
- scores are generated only on the last day of a month.
- Quarter
- scores are generated only on the last day of a quarter.
- Yearly
- scores are generated only on the last day of a year.
Collection of PA indicator score data
The period unit (days, weeks, or month) at which the PA indicator scores are collected and preserved depends on the frequency of the data source indicator. However, the frequency at which the application indicator collects the PA indicator data source scores varies.
In Enterprise Architecture, the frequency of the application indicator must be greater than or equal to the frequency of the data source indicator.
The following table describes the frequency at which Enterprise Architecture collects data from the data source indicators after the job runs:
| Enterprise Architecture frequency | Data source indicator frequency |
|---|---|
| Monthly | Monthly |
| Quarterly | Monthly and Quarterly |
| Yearly | Monthly, Quarterly, and Yearly |
If you are an Enterprise Architecture customer, who has upgraded to the Australia release, then the Daily frequency of Performance Analytics data source indicator is not available. RemoveDailyFreqAndUpdatePAIndicator fix script automatically removes the Daily frequency of PA indicators and updates the frequency to Monthly.
Limitations to display application breakdowns in PA scoresheet
If there is a large number of business applications installed, then all the breakdowns are not displayed in the , as there is a limitation set in the system properties: com.snc.pa.scoresheet.max_elements and com.snc.pa.scorecards.max_breakdown_elements. To reconfigure the property limitation:
- Navigate to .
- Enter the maximum number in the Maximum number of elements of a breakdown in
Scoresheet field. The number must be greater than or equal to the number of
business applications installed in your system.
Abbildung : 1. Performance Analytics Scoresheet - Enter the maximum number in the Maximum number of breakdown elements in scorecard lists field.
- Click Save.
Job schedule to compute application scores
After you set up indicators, create score profile, and attach profile indicators, schedule a job to periodically compute the application scores.
Understand how the system calculates application scores and create your application score profile per your requirements.
The assessment framework calculates the application score for each application on a scale of 1–10, where 10 is a good score and 1 is a low score. Assessments are based on various configured indicators, which you can configure. Each of these indicators periodically captures the related application data, which is used to derive the application score. These indicators with their respective value (weightage) are added to an application profile. The application is then associated with the application profile, which calculates the application score.
Normalization of application scores
The indicators and their respective weights are used to calculate application score profiles for each configuration item. Use the score profile to calculate application scores and assess the applications. Apply these scores to compare applications and make strategic decisions about which ones to keep, replace, maintain, or invest more in.
The preconfigured indicators or the indicators that you created retrieve their related data based on the frequency set at the indicator definition stage. This data is captured in the Application weight column of the Application Indicator Score [apm_app_indicator_score] table. The Target maximum and Target minimum that are set while creating an application indicator are for calculating the applications normalized value.
The normalized value of the application score, which is measured on a scale of 1–10, is derived from the following formula:
(Application Weight - Target minimum)/(Target maximum - Target minimum) * 9+1- If the Target maximum and Target minimum are not set, then the maximum value within the range of applications is taken as the target maximum value. Similarly, the minimum value within the range of applications is taken as the target minimum value.
- If the Target maximum and Target minimum are set and the Consider Absolute Values check box is selected, the entered values are considered.
- If the Target maximum and Target minimum
are set and the Consider Absolute Values check box is cleared, the
values are considered based on the following intelligent logic.
Target maximum = Minimum value of (Target maximum value defined in the Indicator [apm_metric] table, Maximum value of Application Weights for the fiscal period)For example, consider a scenario where:- The application weights are 10, 20, 30,.…., and 1000.
- Value entered in the Target maximum field is 100.
With these assumptions, the Target maximum value considered is 100, as the defined Target maximum value (100) is lesser than the maximum application weight (1000).
Target minimum = Maximum value of (Target minimum value defined in the Indicator [apm_metric] table, Minimum value of Application Weights for the fiscal period)For example, consider a scenario where:- The application weights are 10, 20, 30,.…., and 1000.
- Value entered in the Target minimum field is 100.
With these assumptions, the Target minimum value considered is 10, as the defined Target minimum value (100) is greater than the minimum application weight (10).
The Application Weight that is lesser than or equal to the target minimum is given the lower score, which is 1.
The Application Weight that is greater than or equal to the target maximum is given the maximum score, which is 10.
When you set the application indicators, you can also configure the Direction as Maximize or Minimize. The application with the maximum value gets the minimum score when the direction is Minimize. The application with the minimum value gets the maximum score when the direction is Maximize.
If the Direction in the indicator is Minimize:
(10 - above calculated Normalized value) + 1Application profile weightage is then applied on the Normalized value to derive the Indicator Score:
Normalized Value * Weightage as in application score profile %After the indicator score is calculated for each of the indicators, the application score is calculated by summing up all the indicator scores used in the profile.
If the source of the indicator is Indicators in the Data source field, then the application weight is calculated as the sum of the normalized scores of all its dependent indicators.
- The normalized score of the parent indicators is then calculated in a similar manner as it is calculated for all the other indicators.
- The normalized value, indicator score, application weight, target maximum, target minimum, and total weight are all rounded to two decimal places only.
In the figure, since the Cost and Incident indicators are set to minimize, the applications with lower costs and lower number of incidents have higher scores.
Normalized value and application score for an assessment
If the source of the indicator is Assessments in the Data source field, then the Target maximum, Target minimum, Application weight, and Total weight values are considered as zero.
For a business application to be considered for scoring, it must be mapped to a respective application profile to which the indicator is associated. You can check the scoring profiles in the apm_application_profile list.
All the business application related Assessment Category Results having assessment groups created within a fiscal period are considered for calculating the score. You can check the assessment category results for a business application in the asmt_category_result list.
For an assessment group to be considered for computing scores, all the related assessment instances must be either in completed or cancelled state.
scaleFactor = (9/ scale factor on metric type)appAsmtScoreSUM = SUM of ratings of category results groupedBY source for each assessment groupappWeight=((scaleFactor * appAsmtScoreSUM )+1;Normalized value = total app weight of BA/appOccurences.totalIndicatorsWeightage = the sum of all indicator weightage mapped to a scoring profile.appIndWeightage = weightage of the current indicatorIndicator Score = normalizedValue * appIndWeightage / totalIndicatorsWeightageThe app occurrence is the occurrences of a business application.
Visualization of application performance
Visualization of the performance of applications in different dimensions on a bubble chart, in a dashboard, and in an application 360 view helps you to take decisions on the applications.
Having set up indicators and attaching application score profiles and running the scheduled job to calculate its scores periodically, your application scores are now ready for viewing.
- Use bubble charts to visualize your business application data plotted on a chart in three dimensions, which helps you to compare and evaluate applications based on their indicator scores.
- Use Application 360 to focus on the business applications that require your attention.
- Use Application Assessments dashboard to view the trends of indicators for different applications.