Exploring ways of handling irregular and sudden bursts of multiple files for data processing using event driven architecture on AWS. This blog posts showcases how to use S3 notification with EventBridge to trigger a Glue Workflow that has number of events and batch window trigger conditions.
I’m thrilled to present pytransflow, a Python library I developed in my free time. pytransflow simplifies record-level processing through transformation flows defined in YAML files. I hope you find this library engaging and that it sparks your interest to both use and contribute to it.
Exploring the implementation of Lua scripting for dynamically altering API requests in an Nginx Reverse Proxy. This investigation opens up possibilities to write and run dynamic content using Lua scripts directly within the Nginx server, making it a powerful tool for web applications.
Sharing my AWS Data Engineer Associate exam experience, resources, tricks and tips, study notes etc. I hope this post will give you some insights and help you find resources to prepare for the exam.