EuroPython 2015

Building a multi-purpose platform for bulk data using sqlalchemy

At Blue Yonder, we’ve built a platform that can accept and process bulk amounts of data for multiple business domains (e.g. handling retail store location and sales data) using SQLAlchemy as a database abstraction layer. We wanted to use as much of SQLAlchemy as possible, but we quickly found that the ORM (Object Relational Mapper) is not suitable for handling large amounts of data at once. At the same time, we did not want each team of developers working on individual business domains to have to handcraft their own SQL statements. To solve this problem, we built an application configuration that closely resembles an SQLAlchemy model, but also contains application-specific logic settings.

In this talk I will demonstrate:

  • an application architecture for multiple business domains

  • the structure of the domain configuration utilized in the generation of the SQLAlchemy model, SQLAlchemy core statements, and other application functionality

  • how the domain configuration is used throughout the application (consuming and parsing incoming data, storing it in a database and ensuring data quality)

in on Thursday 23 July at 16:45 See schedule



  1. Gravatar
    Yacine OURDANI - Founder & CEO
    m-BioSecure Project Manager (Mobile Bamking Security)
    Start-up : Algerian IT Security Group 
    Address : Cyber Park de Sidi Abdellah
    E1 S6 ST 15
    Incubateur Techno-bridge 
    Route Nationale n° 63 Rahmania Zeralda 
    BP n° 61 Mahelma 16121 Alger - ALGERIE 
    Mobile : 00 213 551 258 291 
    Email : 
    Skype : ourdani.yassine 
    LinkedIn : Yassine OURDANI
    — OURDANI Yacine,

New comment