Data Engineer
Job Description
Must Have
- Python programming (Advance)
- Various Data Matching Algorithms (String Similarity based, distance-based, phonetics algorithm)
- Basic Graph DB / Vector DB knowledge
- ML-based solution approach to define/ refine data matching criteria
- Azure platform ( Databricks / Fabric / OpenAI etc.)
Good To Have
- API programming (FastAPI or Flask)
- We have a need for two Azure data engineers for a master data project. One person should be a senior lead engineer almost at architect level and second could be mid to senior engineer.
- The project requires data acquisition into Azure, creating data pipelines, normalize JSON data into relational format.
- Model and design table structures
- Analyze data and identify relationships,
- Match the data with another dataset to find patterns and relationship
- Create combined datasets by using various matching and merging techniques and logics
- Build APIs to search the data
- Build lightweight UI to enable search functionality for users
- Main idea is we need self starter who can work independently and figure out the solution on their own, work with client team, discuss and modify accordingly.