Natural Food
Enabling an FMCG company to
improve their data science
operations
UI/UX Design
Data Engg & Analytics
Mobile App
Food, an integral part of Indian culture shares the same diversity as language and tradition. Traditional Indian food has an exceptional blend of flavours, colours, seasoning, nutritional balance, aroma, taste and visual appeal. An Indian woman gets trained in culinary skills right from a tender age. However to get the perfect taste she ends up spending a lot of time in the kitchen.
Industry
Retail
What we did
- Data Engg & Analytics
- Website Design
- Mobile App Development
Objective
A mid scale FMCG company runs a retail chain throughout the country. They have packaged goods that are manufactured at their factories and then sent to their distributors which then sell to their retail outlets in many cities.
Challenges
The sales team had a problem that most of their retail outlets were going out of stock with a lot of the items and hence wanted to reduce those cases and increase business. The data science team were tasked to come up with a model that would forecast the demand and hence suggest what products need to be stocked up in which retail stores in order to meet demand with a good capture rate (Goods sold/ Goods stocked). Some of the challenges the data team faced with were.
The historic data was made available in multiple inventory management systems, SAP databases and flat files. Bring all this together needed a lot of data engineering effort.
The data science team also worked with business, domain experts and statisticians to arrive at features for the forecasting model. These features were calculated using spark and the output was stored on AWS S3. It was challenging to constantly work with DevOps to spawn up spark clusters to execute these workloads.
The forecasting models used the S3 files is an input. The sample data was run on the model first using R and Python and then the full data set was broken into chunks and run on multiple parallel instances with GPU. The results had to be combined and stored in redshift for further analysis, this is time taking and prone to data loss in case any one of the machines fail.
The custom code built was really fragile and any new changes in the source had a trickle-down effect throughout the pipeline and caused a lot of issues which needed to be fixed at every stage of the entire process.
Solution
Data Integration
The data engineers on the team could easily configure their sources and bring them into synctactic to then further prep the data. Once this was done, a search based interface was presented to them where they could easily search and understand where their data is stored and in what structure.
Machine Learning Models
Another project was created and the processed data with all the features were then added to the Python and R models using the code engine. This was then run on the same cluster with improved efficiency. The output was then pushed into Redshift.
Data Pre-Processing
The preparation of the data was simple. A project was created and a pipe was defined to join data between the source files and then the code engine was used to upload the transformation scripts and the output was stored back in S3. The data processing took a few minutes using serverless functions to execute the transformations
Collaboration and Version Control
Any new changes to the existing project or pipes could be branched out as new versions and users with access to these projects could collaborate in real-time hence improving productivity and reducing bottlenecks.
Outcome
With a great workflow and platform in place, the team could bring in more data sources, collaborate with other members of the team on projects and could scale up as and when required with the help of Synctactic. Not only did the team improve time to market for all their data science initiatives, they also did it at a fraction of the previous cost. Synctactic truly solved their data science problem by providing them with a platform that is Simple, Smart and Cost-effective.
Faster data processing
2
x
Improved team collaboration
Recent Work
Launch your product with Arokee
Partner with us for a digital journey that transforms your business ideas into successful, cutting-edge solutions.
