![]() Let me know if you have any questions or comments in the comment section below.This tutorial shows how to quickly run PostgreSQL and the pgAdmin management tool in a Docker container using Docker Compose. Next time you are required to build a EL pipeline try out Singer and see if it can fit your use case, before writing your own logic. You can easily write an Airflow operator using Singer specs which keeps its state in a cloud storage this would be a good option as well. There are some open source projects like Pipelinewise Although Singer is a bit rough around the edges it provides a good framework to build on top of. Hope this article gives you a good idea of how you can use Singer to automate CDC.
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |