Printing tool Download PDF

Karthik Rajagopalan

Lead Data Engineer, Target Corporation


A Data Engineer, with more than 11 years of experience, have a proven track record of delivering Big Data and Machine Learning  solutions in Production. 

Work experience

Jun 2015Present

Lead Data Engineer - Data Collection

Target corporation
  • Developing Anamoly Detection Platform for Target Digital
  • Write Kafka Consumers (High Level) to ElasticSearch. Balance Consumer Push ~ Elastic Pull bandwidth capabilities
  • Set up ElasticSearch Index, Visualizations to monitor System and Application Health 
  • Query, Analyze and Mine Insights on Search, Top Selling Items and Bots
  • Write Sensu Monitors to fulfill product monitoring requirements
  • Stack: Front End: Angular JS; App Layer: Kafka - Scala/Python; Data Layer: Hadoop - Hive/ElasticSearch/InfluxDB; Monitoring: Sensu + PagerDuty; ML Algos: Moving Average, Clustering, KNN
Jun 2014Oct 2015

Founding Partner & Chief Technology Officer

6 months) Founding Partner & Chief Technology Officer at Confidential(In stealth mode)
Dec 2014Jun 2015

Data Analytics Intern

Soleo COmmunications
  • Mined existing data schemas(fields and relationships), internal and external to the company, to identify fields(Metrics: Accuracy, Availability etc.) that can be used to fulfill(newly developed) API requests.
  • Analyzed existing Data Warehouse(Hadoop Clusters) and wrote Hive Queries to deliver business development and sales support data. Deliverables were used as insights to brainstorm potential business opportunities and support/validate“ Go-To-Market” decisions.
  • Delivered PRD(including user archetypes) for Search and Reporting. Developed product test cases and related acceptance criteria.
  • Developed python and R scripts to assist reporting requirements and load testing.
  • Stack: Application Layer: Java/J2EE, Data Layer: Live API, Reporting & Analytics: Hive, R (exploratory & causal analysis), Monitoring: Nagios
Apr 2012Dec 2014

Principal Solutions Architect

Eko Infomatics Solutions Pvt Ltd
  • Architect products and solutions to satisfy RFP requirements; Define and develop POCs and participate in technical demos.
  • Assist CEO with pricing, resource and effort estimations and proposal completions.
  • Strategize solution offerings and define technological product variants to allow platform independence
  • Exhibited solutions for log analytics, security intelligence and campaign management
  • Preferred Stack for Solutions: Streams: Kafka/Rabbit MQ, Data Storage: HDFS, Visualization: Splunk / ElasticSearch Data Analysis / Classifiers: Python / R Packages
Jun 2014Aug 2014

Product Manager

eBay Inc
  • Benchmarked internal to eBay and industry standard mCommerce dashboards and presented findings to product stakeholders
  • Interviewed and Surveyed leaders and stakeholders to identify key requirements for the mobile leadership dashboard
  • Reviewed metric data availability with core analytics teams and defined the ETL requirements for newly identified metrics
  • Presented Tableau prototype of the leadership Page1 dashboard; worked with UX Engineer to create dashboard wireframes Delivered PRD, FSD, Data Deep Dive findings, Wireframes(with UX Engineer) and Development Roadmap
Mar 2011Sep 2013

Subject Matter Expert

  • Led the Target.Com Metrics & Measurement team in the innovation, design and development of “ Infoboard”, a connected commerce business intelligence product that integrated data sources.
  • Architected, developed and delivered custom ETL tool based on Data Warehouse best practices to accommodate multiple data formats such as flat files, recommendations/reviews, logs and RDBMS.
  • Defined models and algorithms to measure and optimize KPIs to predict business outcomes and scenarios. Created What-if analysis and reported functional analysis diagrams to picture user experience behavior patterns.
  • Executed analytics/statistics induced continuous service/process improvement initiatives: improved website response time(~ 14%), increased work force efficiency(~ 7%), and retired under-utilized servers(~ 8%).
  • Stack: Front End: SAS Visual Analytics / D3 Charts, Application Layer: SAS Cubes (build on SAS EG), Data Layer: Multiple Data Sources ETL: Custom build Dot Net Tool
Mar 2008Mar 2011

Business Intelligence Developer

  • Led the offshore team to deliver IT Service Delivery Analytics & Reporting products worth $2.4 M across the energy, finance & telecom sectors.
  • Established centralized ITSM metric template and repository that automated performance analytics and enabled faster build and cheaper maintenance.
  • Developed sales strategies to help position HP in the ITIL-Service Level Management solutions arena.
  • Supported the Mphasis pre-sales team in completing client RFP by offering specific solutions across multiple applications involving different technology and skills in the IT Service Management space.
  • Stack: Application Layer: C++, C#,  XML Front End: Oblicore Guarantee Visualizations, Data Layer: Technology Agnostic
Jul 2005Mar 2008

Software Developer

  • Enhanced HP software to improve productivity & reliability and satisfy customer's specific requirements.
  • Re-designed HP Software to reduce system incidents and customer complaints Developed auxiliary software tools to reduce overall supporting time thus improving HP Software servicing efficiency.
  • Evaluated implementation feasibility of HP software in client environments and participated in over 20 clientengagements. Projects Market Research & Content Generation for a Precision Timing Company
  • Stack: Application Layer: C#, XML Middleware: Biztalk 2003 Server, 



M.B.A - Emphasis on Data Analytics and Machine Learning

SIMOn School of Business - Univ. of rochester
  • Intro. to Business Analytics CIS 442A
  • Business Analytics Projects CIS 442B
  • Marketing Analytics MKT 437
  • Advance Marketing Analytics MKT 451
  • Social Media Analytics CIS 442C
  • Product Management - Screening & Generating Ideas ENT 422
  • Product Management - Technical Entrepreneurship ENT 425

B.Eng - Computer Science & Engineering

Manipal Institute of Technology - Mahe


Data Mining & Maching Learning
  • Problems: Classification, Clustering, Regression, Association
  • Supervised Learning (classification & regression): Decision Trees, Bagging & Boosting, k-NN, Linear & Logistic regression
  • Clustering: k-means, hclust
  • Analysis: Sentimental Analysis, Predictive Modeling, Topic Modeling, Exploratory Analysis, Causal Analysis
Technical tools & Programming languages
  • Programming Languages: Python, R, Scala, Java, C++
  • Database Languages: SQL, Hive HQL, Influx's SQL
  • Real Time Streaming & Analytics Stacks: Elasticsearch Elastic, InfluxDB TICK
  • Message Broker: Kafka 
  • Build: Maven, Gradle (Beginner Level Only)
  • Version Control: GitHub
  • Monitoring: Sensu, Compuware, HP OVO
  • Agile: JIRA
  • HDFS: Map Reduce

Top 3 Awards

Next Big Idea Target Co - Innovation Council November 2012

This was in recognition to the innovative and entrepreneurship skill exhibited by me. My friend and I came out with a concept that would enable Target Co. to understand Target's mobile guests.

Key Employee Retention Program Mphasis an HP Company March 2010

This was provided as a recognition of my performance and to motivate me further.

SeaTUG Tableau Viz Competition Slalom Consulting September 2014

My team won the 2nd Prize in the SeaTUG Viz Competition organized by Slalom Consulting. My approach in defining Sentimental Analysis using Tableau was seen as very innovative and was highly appreciated.