Introduction

Imagine a world where every significant initiative, whether a training program, product launch, or marketing campaign, directly correlates with positive results. Enter "APM as a Service", a transformative tool designed to accurately measure the impact of events on targeted outcomes. Gone are the days of relying on assumptions and biases. With APM, we can now quantify the efficacy of events, understand their real impact on business goals, and make informed decisions backed by data.

This case study chronicles the collaboration between our team at Priorise and a leading technology giant to harness the potential of APM as a Service. Our objective was not just to evolve our client's existing flagship model, Accountability Performance Matrix (APM), but to redefine how we perceive the influence of the various interventions aimed towards improving sales efficiency. By focusing on measurable outcomes, we sought to offer a solution that was not limited to the usual few metrics but could cater to a wide array of measurements, emphasizing the general applicability and versatility of the APM tool. Whether you're looking to gauge the success of training programs, customer engagement initiatives, product launches or any other event, APM as a Service stands as a testament to the evolution of data-driven decision-making, making impact measurement accessible and straightforward for all.

Challenge

Our client faced the predicament of relying on a traditional APM solution that was restrictive in its capabilities

Role Limitation

The traditional system primarily catered to measuring outcomes for sellers, leaving out a wide array of other roles that contribute significantly to business processes.

Metric Limitation

The earlier system's scope was confined to measuring outcomes only for two metrics - Annual Contract Value and Pipeline generation, neglecting other essential Key Performance Indicators.

Event Limitation

It lacked the capability to measure outcomes across diverse events.

These limitations posed a significant challenge as our client sought a more holistic approach to understanding the impact of their various interventions across different roles, KPIs, and events. They needed a versatile tool that could break free from these constraints, providing a comprehensive view of their business's performance. Enter the need for "APM as a Service", which promised not just to measure but to revolutionize the way our client perceived the impact of its diverse range of initiatives.

Objective

The objective of this case study is to revolutionize impact measurement by introducing "APM as a Service," a comprehensive tool that measures outcomes across various roles, Key Performance Indicators (KPIs), and events. The goal is to provide a holistic view of business performance and enable data-driven decision-making.

Matching

Drawing on data from our client, we employed a matching approach. By comparing treated subjects with their non-treated peers sharing similar characteristics — like role, region, industry, and market segment — we could measure the precise impact of treatments, eliminating potential selection biases.

We ensured accurate and robust treatment effects estimation using a blend of matching best practices and cutting-edge techniques like generalized random forest with double robust estimation.

APM as a Service Process

Data Input

Users provide the required data for analysis to the API, complete with a plan detailing the timeline, resources, budget, and data limitations.

Parameter Selection

Users choose statistical model parameters, including confidence levels and significance thresholds.

API Execution

Upon data and parameter specification, the API runs the statistical model, churning out actionable insights.

Output Delivery

Customizable reports or dashboards present insights tailored to user preferences.

To streamline this solution, the process is divided into four modules:

Module 1
image

Data Management

This involves data ingestion, preparation, and quality checks, ensuring data integrity and readiness for subsequent modules.

Module 2
image

Machine Learning Application

Two primary models, a classifier and a regressor, serve as inputs for causality analysis.

Module 3
image

Causality Analysis

Conducted using R, this module dives deep into understanding causal relationships in the data.

Module 4
image

Reporting

Post-processing culminates in the delivery of an executive summary generated by leveraging a large language model (LLM), APM outputs, and detailed reports on data quality and ML model performance.

With this robust solution, we've positioned our client to move beyond mere measurements, enabling them to harness the full potential of data-driven insights.

Conclusion

This case study underscores the transformative potential of "APM as a Service", emphasizing its capability to drive data-informed decisions across varied business domains and set new benchmarks for success. The introduction of "APM as a Service" marked a paradigm shift from traditional performance measurement tools. By seamlessly integrating advanced data analytics, machine learning, and causal inference tools, the service has redefined the way our client gauges the impact of its diverse initiatives.

Sales Programs

APM's prowess was showcased when it measured the impact of over 900 Sales Programs. The insights revealed a staggering positive impact exceeding 120 million dollars on ACV, reaffirming the value of the tool.

Professional Services

Beyond sales, APM was leveraged to gauge the effect of PM certifications on Project Managers. The results informed decisions regarding KPIs related to project management, optimizing processes and enhancing service delivery.

Performance

For Account Executives (AEs), APM has been showcasing the impact of a flagship intervention project for our client across eight different KPIs beyond ACV and PG, including several lead quality and call transcript metrics.

Project Information

Client:

The Sixmothers Group

Location:

Jaipur, Rajasthan, India

Impact and Results

The implementation of "APM as a Service" brought about transformative results across different sectors:

  • Sales Programs: APM's prowess was showcased when it measured the impact of over 900 Sales Programs. The insights revealed a staggering positive impact exceeding 120 million dollars on ACV, reaffirming the value of the tool.
  • Professional Services: Beyond sales, APM was leveraged to gauge the effect of PM certifications on Project Managers. The results informed decisions regarding KPIs related to project management, optimizing processes and enhancing service delivery.
  • Performance: For Account Executives (AEs), APM has been showcasing the impact of a flagship intervention project for our client across eight different KPIs beyond ACV and PG, including several lead quality and call transcript metrics.

This broad spectrum of successful applications emphasized APM's versatility and potential to drive data-informed decisions across varied domains, confirming its transformative promise.