Cisco Systems - Technical Innovation of the Year
Company: Cisco Systems Inc., San Jose, CA
Company Description: Cisco (NASDAQ: CSCO) is the worldwide technology leader that has been making the Internet work since 1984. Our people, products and partners help society securely connect and seize tomorrow's digital opportunity today. Discover more at thenetwork.cisco.com and follow us on Twitter at @Cisco.The company was founded in 1984 and is headquartered in San Jose, California.
Nomination Category: Company / Organization Categories
Nomination Sub Category: Technical Innovation of the Year - At Organizations With 1,000 or More Employees
Nomination Title: Innovative Analytics & Insights
Tell the story about this nominated innovation since January 1, 2016 (up to 650 words). Describe how the innovation was discovered, developed, refined and/or deployed. Provide an assessment of how the innovation has impacted or will impact your organization, your industry, markets or society.
At the core of every successful client-facing organization is providing best-in-class customer experience. Cisco recently embarked on a digitization journey which fuses the IT systems and business processes in a cohesive manner to transform the experience for internal and external customers.
With increasing demands on IT infrastructure coupled with a constantly evolving technology landscape, there was a need within Cisco for a unified monitoring, performance and capacity management solution across all stacks of infrastructure and applications. The Enterprise Data and Platform Services (EDPS) team pioneered and developed an IT platform health management framework – Hadoop Insights – with Big Data technology to address issues such as disparate monitoring capabilities scattered across multiple tools and an absence of a common framework which led to delays in root cause analyses and a lack of insights into compute- and storage-related metrics which led to system availability issues.
Before Hadoop Insights, IT support teams were faced with the daunting task of manually monitoring platform health across the IT spectrum. With this implementation, the aim is to leverage the framework for enterprise-wide service functions such as Supply Chain, Order Management and Finance consisting of any infrastructure systems, data and application platforms.
EDPS’ Big Data Hadoop ecosystem supports 2500 application users with 6.2 petabytes of storage and 235 servers. The Hadoop Insights tool consists of two modules – Service Assurance and Compute Analytics. The Service Assurance module has intelligence built in for real-time monitoring of Hadoop ecosystem services, server load, memory and application events. The Compute Analytics module provides a real-time analytical dashboard which reflects the status of the storage, platform growth trends, usage drill down and growth forecast.
Technology Used: The Hadoop Insights framework loads usage data, events and compute and applications logs using Kafka. Data is stored in MAPR-DB with analytical intelligence built using open-source Python and Apache Spark. The visualization layer utilizes AngularJS and Highcharts. A Solr search platform is used for data acceleration and faster search performance.
Modular code for reusability.
Easily portable to other environments.
Anomaly detection and threshold-based notification.
Events correlation for faster resolution.
-Analysis of historical usage patterns.
-Prediction algorithm for visibility of compute and platform growth
-Service status summaries and alerting.
-Reports for proactive planning of quarterly storage expansion.
-Security scan for access violations.
-Job SLA prediction using Machine Learning.
-Recommendations for platform storage optimization.
Business Outcomes: Hadoop Insights, with its advanced analytical models, has enabled a multitude of benefits. Among the measurable benefits:
-Persona-based insights to all levels of the management chain, catering to different personas and job needs, enabling improved proactive decision making.
-Cost avoidance with storage optimization and repurpose (400 TB, ~6% of 6.2PB).
-Case management mean time to resolution reduction of 80% with event notification and correlation.
-Productivity improvements with automated alerts and centralized monitoring, reduced workload by 25%.
-Troubleshooting time for issues reduced by 80% to an average of 5 minutes per issue.
-Support cost reduction by 25% through manual process elimination.
-System availability improved with self-healing capabilities and over 40% of issues proactively mitigated.
-Self-service capabilities on the tool have resulted in 50% reduction in service request volume.
This framework is now being used by over 2000+ Hadoop administrators, policy makers and IT users. There has been consistent good feedback from users. One such feedback is from Alex Garbarini,Senior Engineer at Cisco , “The tool exemplifies what capabilities a central monitoring hub should have, great job with delivering Hadoop Insights tool “ .
The Hadoop Insights framework embodies the digitization journey at Cisco. With this framework, the EDPS team is able to serve its customers better and at the same time, effectively utilize constrained IT resources. The tool provides a centralized service assurance and capacity management module for monitoring long-standing industry problems and providing actionable intelligence for prevention. EDPS’ continuous innovation will expand to other service verticals, harnessing the power of the Hadoop Insights framework.
In bullet-list form, briefly summarize up to ten (10) chief characteristics or benefits of the nominated innovation.
-Hadoop Insights provides real time, end-to-end visibility of Hadoop platform storage and compute status.
-Hadoop Insights dashboard is designed with re-usable framework.
-Varied levels of Persona based insights helps Management, Admin and end-user community to know the status of the platform.
-Identifies security vulnerability to ensure platform is more secured.
-Service Assurance Dashboard provides visibility of critical components and services.
-Prediction algorithm provides the visibility of the platform growth, and predicts timeline of future space consumption.
-Information provided by this tool, helps diagnose operational issues under a minute (MTTR improved by 50%).
-Provides the complete visibility of the storage (6.2PB) in terms of Hadoop replication and usage snapshot.
-Provides storage optimization opportunity – identify stale data, unused space and uncompressed data (optimized by 15%).
-Trends provides visibility of the recurring issues and enable permanent fixes.