A/B Testing Fundamentals: What Every Data Scientist Needs to Know!

Emma, Thank you for this helpful video! Could you please help me understand the difference between "have enough randomization units" and "size of an exp".

Hi Emma, thank you for the helpful video! I found on the side when you were introducing Experiment Design, the bullet points number for "size of an experiment" and "how long to run an experiment" are both 2.

Thank you!

Hi, thank you so much for the great content! But I can not receive the "product case interview cheat sheet" even I registered my name and email through the link. How can I receive it? Thank you so much!!!

Hi Emma, Any book you recommend to learn Root Cause Analysis or Failure Analysis ? Thank you

COULD I GIVE YOU A SUGGESTION? IT IS BETTER TO ARRANGE YOUR VIDEOS IN ORDER.

An experiment. I still dont get it. How can I estimate how long to run A/B testing?

Thanks for the great video! How would you design an A/B test for a social feature on a social media site (e.g. facebook stories)? The users are interconnected with each other via posts, shares, likes, etc., rather than having their own isolated experience such as on e-commerce websites. If we randomly spilt users into 2 groups for the A/B test, users in the A group are going to be interacting with users in the B group. Could that bias the results?

Thanks for uploading this video, Emma. It' very helpful. I had a doubt regarding A/B/N testing and multivariate testing. I designed a full factorial experiment where I had 6 groups (factor A had 3 levels and factor B had 2 levels) without a control group. I checked that my residuals had constant variance and my data was normally distributed and so I was able to use Tukey's multiple comparisons. The result was that the best two groups (highest mean response indicates best in my use case) were not statistically different and even though it was practically significant as per an arbitrary self established metric, I would not recommend the winning group as it was not statistically significant (I believe this is the sanity check as the investment in winning group was worth it). I wanted to know if this problem falls under A/B/N testing or multivariate testing? Thanks in advance!

Hi Emma! Great video as always. You mention that in experiment design you use seasonality, day of week, primacy and novelty effect to determine how long to run the test for. But in your A/B testing interview video you only mention sample size. I must’ve missed something but are you able to explain why there’s that difference? Thank you!

This is really helpful!! Not sure how easy this is to do but I was wondering if you would be able to make transcripts available for the video. (I personally have a learning disability so it really helpful for me to have a transcript to accompany a video. Thought this might be also helpful for people where english isn't their first language too)

Love these videos on AB Testing! Any chance we can get a video on methods of experimenting when AB-Testing is not possible? (i.e. Synthetic Control methods, switchback experiments, etc.)

HI emma. I have a question, so why don't people do sanity check before running the experiment? So they can save energy on running an unqualified ab testing.

Hey Emma, thanks for uploading this. It's very helpful. And I've a question here: what's the difference between sample size in experiments and the number of randomization units? Thanks in advance.

Hi Emma, I found a lot of case of A/B testing if about the website users. What is there is a national retail chain company that has 1000 stores around the country. And they want to test if a coupon can help increase sales. In this case, what is the randomization units here? the stores or the customer? Does A/B testing still apply in this case?

@Data Interview Pro Please can you share all slides of A/B testing using google drive or any other application etc.

Hi Emma. Thank you so much for your video. Can you make a video on what to include in a resume to increase the chances of resume selection for DS interview?

Do you plan to make the slides available? It would be extremely useful. If so, you could also put a watermark on it because I bet it would be shared very often.

Emma thanks for another amazing video! 希望能看到你出更多的product sense相关的视频:D

This video helps me in learning statistics and probability. Thanks Emma for making this video.

谢谢Emma! 你的视频真的很有帮助～继续加油！

nice video, your blue sweater perfectly echoed with google color schema test!

谢谢分享！视频条理清晰！期待更多视频！

Emma guided me through my interview prep, and I got 3 DS offers, she had suggested this book to me, this book is awesome.

thanks for your videos! Plz keep updating these great contents and it is super helpful!

Feel like I have a lot to learn about ab testing. Can’t wait to see the next few videos!

Thanks Emma , for making these videos. Looking forward to the next part 👍

thank you. See you soon!

I would love to have more of this videos.. 😃