- July 28, 2023
Data Analysis & Visualization of Heavy Machinery Data Using Snowflake
Problem Statement
The company wishes to analyze Industrial level Machinery behavior with various analytical approaches by using its raw data, this will help companies to identify the behavior patterns under different conditions well in advance and be able to take upfront action in order to prevent these Machinery damages
Use Cases
Early failure detection
The organization want to develop a process to analyze raw data from this heavy instruments to find the status of them. This process will help the companies to identify the failure or any defects well in advance, hence there is no sudden machine failures, and they have well advance time to do maintenance
Optimizing Performance & maintenance cost
This Analysis will help to identify the best optimizing way to leverage the Machinery performance, this process also plays a major role to reduce the maintenance overhead cost and reduce the manual maintenance burden.
Tools & Technologies
- AWS
- Snowflake
- Python
- Mysql
- MongoDB
- Power BI
Benefits
- Advance Alerting System
- Centralized Monitoring System
- Less Human Intervention
Project Architecture
Our Approach
Extract
- Extracting data from the source (sensors push data to MySQL & MongoDB)
- Using Python script to connect to sources to extract data
Load
- Load the extracted raw data into Snowflake tables
- Creating Snowpipe for continued data integration
Transform
- Apply required techniques such as Cleansing, Standardizing, Duplicates remove, and multiple Stored Procedures.
Analysis
- Connect with Snowflake using Python
- Generate Analysis based output for models
- Creating Threshold-based alerts
- Push final model output to the Production table
Report
- Generate PowerBI report for Visualisation
- Connect to PowerBI Using Snowflake