“The way to get started is to quit talking and start doing.” - Walt Disney

Backend Development Project

crowdETL

Development Stack

(subject to change)

  • Hosting Platform

    • Google Cloud

  • Database:

    • Cloud SQL/Postgres

  • Development Languages

    • Google Cloud Composer

    • Python

    • Django Framework

Basic Project Requirements

(more detail requirements will follow)

  • Be able to process different file types

    • CSV, jSON, TXT, XML

  • Different data sources

    • Databases

      • SQL Server

      • Postgres

      • MySql

    • FTP

    • Web API (RESO)

  • Clean incoming data based on user requirements

  • Allow user to defined columns of incoming data

  • Map columns to new data set

  • Push data to outside destination