How to load a file in the bucket into Bigquery in an event-driven way?

Cloud function is an event-driven tools that we could use when we don’t need to fix computer resource or network resources but just based on the resource used when we use. But what’s is event, when we put a file into a bucket, file added, that’s an event; when we delete a file, file changed, that’s an event; when we modify a file, file changed, that’s an event. We just need to pay for the resource we used.

When we are in GCP, we could use…


As we do need to connect with Database using python when we do the data science or do data engineering, there are many ways that we could connect with them, for Mysql, there will be PyMysql, I have used it many times, but most time will combine with Pandas so that we could get a DataFrame for later step use case, but I find another excellent library called: Records named: SQL for humans, we are the humans 😄.

I have to say that for a Python developer, there are so many great modules that we could use for our daily…

Guangqiang lu

Working on machine learning and deep learning.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store