Chapter #9 - How to design data pipeline on gcp (Google Cloud Platform) ?

I m wondering why people are not making step by step tutorial for data engineering for etl and streaming like we have so many for azure

Thanks

Dataform is another amazing tool for ELT jobs

great content..thank you so much...

Thanks a lot

Too good learning material!

Really Great video, been looking for such explaination

3rd party -> gcs stage can also be done via arbitrary code inside a container on "cloud run jobs". I did that, and downstream what is shown in the video almost exactly on one of my freelance projects.

Awesome Explanation.....

Sooooper Simple, you made sir, Thanks a lot

This is one of the great video to make me understand how Project architecture is there in GCP, I would request to please create a series to build end to end demo project in GCP utilizing all these services.

good one. what is datamart, for datamart you have been using data studio,automl,looker? can u pls explain why??

brilliant brilliant explaination!keep it up!

Thank you, mate! I like your optimism as well as the explanation.

Dear Anshul, how are you ? I have been working as Data Engineer in on-prem solution of our company for more than 10 years and now wish to move to Cloud Based solution, like GCP Data Engineer. Can you please do help me with the below queries I have :-

1. I am having total 15+ years of IT Experience. Will it be a good decision to switch to Cloud Data Engineer Roles now ? It seems to me bit late to start journey in this domain now. Please do correct me , if I am wrong.

2. In order to start / work with the GCP Data Engineering Role, can you kindly suggest me what are the topics / tools / GCP Services I need to learn ? I have basic ETL Concept and also working with PLSQL , Python stuff.

Thanks in advance !!

Great

It's really excellent with kind of explanation .
Sir do you have gcp pde paid course as well,plz comment

when it comes to data engineering, i think you're the best! thank you for sharing your knowledge for free.

what can be the most cost efficient way to ingest a few tables from snowflake to GCP bigQuery

can you do an end to end project in google cloud using this services.Then it will good for us.A lot of thanks to you for giving this type of content.

Excellent.

It’s perfect explanation….you are genius man

Thank you. Very useful video.

Awesome video - thank you! 👍

Would love to see an equivalent video for AWS and Azure also.

Hey Thanks for this video, I have 1 question. If we want to do performance test for this type on msging queue application, what should be our approach.?? and how we can do this.. plz

Please make a video real time projects in gcp

excellent

Thank you very much. Great video.

Very nice video, great overview that is very helpful in navigating the jungle of google tools. Thanks a lot!

Amazing !

Good explanation Sir, could you please make video on Streamsets pipeline?

Great video! I really appreciate the effort you put into this video and it really helped me out. Thanks a lot1

Très intéressant merci

Absolutely, absolutely wonderful and insightful presentation. Thank you very much.

Just an observation. With BigQuery as your data warehouse, companies need not go for datamarts any more. BigQuery being a scalable solution, personnel from the various departments such as Sales, HR, Marketing, Finance can run their queries against a single database and still get results back without compromising on performance. Security can be implemented at the user/department level.

Phenomenal Content.

Very nicely explained. All the best. thank you!

You might make my next job a bigger paycheck,nice video

Profound knowledge you are sharing with us. Thank you so much! One request: whenever you refer your other videos, can you please display them in those bubbles in a corner so that we can click on them and go to those videos? You know those small bubbles with link that pop up..

Very good explanation, please make some videos related to complete data engineer project flow with pyspark or SQL

Really good video and good explanation learned a lot. I just wanted to ask one thing ,the cloud model you showed is it conventional ETL flow or its more of ELT with subset of ETL , because here we are using both the flow or its some kind of new approach which we can use, if would be really good if you could share your thoughts on this

really enjoy watching your video, very clear and well-explained.

I'm trying to build a data lake for an undergrad cloud class and honestly I was feeling pretty hopeless but after watching your video I feel way more prepared to tackle this!!!! Thank you!!!

Nice explaination. 👍 Want to have some more videos like this for real time data processing with Google cloud (for example IOT data,sensor data)

Very nice ......thanks for enlightning with so many information

As a Data Engineer who worked on Cloud solutions for 7 years, I have never seen such a correct and clean solution. I am interested to see the same solution on AWS as well.

Thanks for the video..!

Good one :) :D

Very informative and detailed! Thank you!

Great explanation..one request .. please use mic 🎙️ as voice is Little low as compared to other educational videos.... great job..