Business Services & Consulting • all cities, PA 39
Database Engineer with asset management (39)
all cities, PA 39On-sitePosted 16 hours ago
Business Services & Consulting
About the Role
Database Engineer
Position – Database Engineer with asset management with Python and Kafka exp.
Location: Dallas / Denver / Remote.
Must Have:
The following skills are mandatory:
Streaming data processing through KAFKA
ETL/ELT development through Python
Data analysis through SPARK/SPARK SQL
GCP public cloud
Job Description
Candidate is expected to profile data, understand the characteristics of data, identify outliers and anomalies within data, discuss with stakeholders to define cleansing, transformation rules. Based on the rules defined, build data pipelines (ETLs/ELTs) in Python. Must have strong experience in handling with Streaming data using Kafka. (Must have)
Must have strong experience and thorough knowledge in SPARK, Python, Airflow, Snowflake or BigQuery and Google cloud in general.
Must possess good design skills pertaining to processing Data in Motion, Data at Rest, ETL/ELT frameworks (Error & warning handling, Audit & reconciliation, process controls)
Must have strong experience in analyzing data (SPARK, SPARK-SQL & ANSI SQL) and building data pipelines (Python, Spark, Kafka)
Must have good knowledge in all aspects of data management and prior experience in asset management is mandatory.
Must have good understanding of Asset management business domain.
Database Engineer
Position – Database Engineer with asset management with Python and Kafka exp.
Location: Dallas / Denver / Remote.
Must Have:
The following skills are mandatory:
Streaming data processing through KAFKA
ETL/ELT development through Python
Data analysis through SPARK/SPARK SQL
GCP public cloud
Job Description
Candidate is expected to profile data, understand the characteristics of data, identify outliers and anomalies within data, discuss with stakeholders to define cleansing, transformation rules. Based on the rules defined, build data pipelines (ETLs/ELTs) in Python. Must have strong experience in handling with Streaming data using Kafka. (Must have)
Must have strong experience and thorough knowledge in SPARK, Python, Airflow, Snowflake or BigQuery and Google cloud in general.
Must possess good design skills pertaining to processing Data in Motion, Data at Rest, ETL/ELT frameworks (Error & warning handling, Audit & reconciliation, process controls)
Must have strong experience in analyzing data (SPARK, SPARK-SQL & ANSI SQL) and building data pipelines (Python, Spark, Kafka)
Must have good knowledge in all aspects of data management and prior experience in asset management is mandatory.
Must have good understanding of Asset management business domain.
What You'll Do
Streaming data processing through KAFKA
ETL/ELT development through Python
Data analysis through SPARK/SPARK SQL
GCP public cloud
Must have strong experience and thorough knowledge in SPARK, Python, Airflow, Snowflake or BigQuery and Google cloud in general.
Must possess good design skills pertaining to processing Data in Motion, Data at Rest, ETL/ELT frameworks (Error & warning handling, Audit & reconciliation, process controls)
