Building a RESTful real-time analytics system with Pyramid

CeleraOne tries to bring its vision to Big Data by developing a unique platform for real-time Big Data processing. The platform is capable of personalizing multi-channel user flows, right-in time targeting and analytics while seamlessly scaling to billions of page impression. It is currently tailored to the needs of content providers, but of course not limited to.

• The platform’s architecture is based on four main layers:
• Proxy/Distribution – OpenResty/LUA for dynamic request forwarding
• RESTful API – several Python applications written using Pyramid web framework running under uWSGI server, which serve as an integration point for third party systems;
• Analytics – Python API for Big Data querying and distributed workers performing heavy data collection.
• In-memory Engine – CeleraOne’s NoSql database which provides both data storage and fast business logic.

In the talk I would like to give insights on how we use Python in the architecture, which tools and technologies were chosen, and share experiences deploying and running the system in production.