Chao Qian

Chao Qian has extensive work experience as a Data Engineer in various companies. Chao started their career in 2013 at 上海锐战网络科技有限公司, where they worked as a Data Warehouse Engineer. In this role, Chao was responsible for ETL, data cleaning, and management. Chao developed a tool to extract data entries and provided consolidated statistics using Hive and Hadoop MapReduce.

In 2015, Chao joined Garena Group as a Data Engineer and later transitioned to a Software Engineer role. During this time, they worked on a Hadoop/Spark cluster, focusing on maintenance and performance optimization. Chao was proficient in Java, Scala, Python, and Shell scripting. Chao implemented a near-real-time log collector, used Spark for data processing, and built a scheduler system for different tasks.

In 2016, Chao joined ADVANCE.AI as a Data Engineer Lead. Chao set up and maintained a Data Platform on AWS using EMR, optimizing the cluster with Autoscaling. Chao also implemented web services for credit scoring API.

Currently, Chao is working at Atome as a Data Engineer Lead since 2019. Chao has built an off-line data platform using various technologies like Spark, Kafka, Kudu, Hive Matasotre, Airflow, Kyuubi, and Jupyterhub. Chao's responsibilities include building data pipelines to sync databases on an hourly basis and supporting internal users in data analysis and creating dashboards.

Chao Qian obtained a Bachelor of Science (BS) degree in Computer Science and Technology from Fudan University. This educational experience took place from 2009 to 2013.