Client Situation
- Legacy commercial analytics system had multiple issues with Performance, data quality, recency problems, eroding trust in data and insights.
- On-premise solution struggled with the volume and variety of data sets (25+ from sources like IQVIA, Veeva, SHA, SAP, Marketing, and Specialty Pharma), resulting in prolonged processing and sluggish dashboards (reporting solution for 3500+ users across different Business units & Sales Teams -Retail, Non-Retail, SP, Managed Market, Marketing data).
- Hindered strategic planning and resulted in discrepancies in incentive compensation metrics and payouts.
Objective
- The vision was to revamp the analytics system to develop a scalable, robust solution
- To bring agility and efficiencies at scale to enable faster time to market, increased productivity, and maintain organizational alignment
Approach/Solution
- Interviewed business and technology stakeholders to grasp requirements, identify unmet needs, and address issues in the current system.
- Categorized requirements into user scenarios, focusing on data flow and metric views. These scenarios were then transformed into interactive Figma mockups, which underwent collaborative review with field leadership for valuable input.
- AWS Cloud was chosen as a scalable system to meet business needs.
Crafted a high-level design using open-source tech (Spark, Python, Azkaban, etc.) with AWS EMR.
Reviewed with the client's architecture team to align with long-term tech plans
Modular system was designed for easy configuration, enabling parallel execution with minimal code changes for daily business updates
- Incorporated a modular Python-based data quality management engine, addressing standard, business, and outlier checks for each module in real-time.
- In addition to the UAT, multiple parallel runs were conducted to ensure sanity between the base layers of existing and new system.
- Following thorough testing, a pilot phase was implemented with a diverse user group. Concurrently, end-user training sessions were conducted to acquaint the broader user base with the new system. Pilot users played a key role in promoting awareness, sharing tips, and contributing to the high adoption of the system.
Impact
Reduced lag between the data arrival to report refresh from 2 weeks to 3 days
Availability of reports on mobile devices including offline capability
Faster turnaround for usual business changes through the IC period to ensure field users get the IC changes within a week
Ensured robust data quality through thorough QC, ensuring consistent report performance. Achieved quick refresh times, even for data-heavy tabs displaying customer-level information, reinforcing strong trust in the system
