100% CODE COVERAGE - Think You're Done? Think AGAIN.

The software development elders are 100% true...

Nice content

is it a good idea to have your tests rely on internal details like the tax rate? with the code thats written if the tax rates change then the tests will fail.

Unittesting is only step 1 then there is component test and system test.

Superbe vidéo thanks!

You should introduce HYPOTHESIS for unit tests

Why do you prefer pytest? I’m also only familiar with it but this seems so much simpler. I wonder why more projects don’t use base unittests..

It's seems odd that you chose to use unittest for these example, it seems much heavier weight than pytest to set up a test: you need to write it as a class, you have to inherit from the unit test classes etc etc. All this seems like a big disadvantage to using it in videos. Plus isn't it far less popular than pytest? What led you to choose unittest?

edit: oh, nvm, I just realized you probably chose unittest because it's a Python builtin. It's funny that almost everywhere I go that references testing, pytest is listed first and unittest and nose later. It seems like, despite unittest being a builtin, pytest is far more common in the community.

Love the videos! Keep up the good work!

how about like 2% coverage...

In hardware design we have access to tools suites that can take test plans with requirements written in spreadsheets, and when writing unit tests you attach those requirements to tests. As the tests pass it checks it off on the test plan. Do you happen to know of something similar in Python?

Hi Arjan, I'm having trouble running "coverage" in a setup where you have your package code and test code in different folders. The "vehicle_info" package is not found (nonetheless it works correctly in vscode). Any tips for using the "coverage" library in a different setup than this video (I believe the most common setup is to have test files and package files in different directories ?)

Excellent summary of unit testing and TDD!
Although, personally, I prefer to write one unit test at a time and pass it before moving on to the next one.

Mutation testing can bring you a lot closer to 100%. But for non-trivial applications I doubt, that we'd ever reach true 100%. Of course 100% line/statement coverage, but not true coverage.

Hi Arjan, thank you for your excellent video. Some comments: 1) As others already have pointed out, you really should not dismiss random testing for unit tests. With a fixed random seed, you can have determinism in regression and release tests. I would rather say, random testing is an essential testing technique for non-trivial software and multiple state-of-the-art variants such as "property based testing" exist. 2) There are many coverage variants such as path coverage, branch coverage, condition/decision coverage, etc. It would have been good to point out that you talk about line coverage which is a rather weak coverage metric. 3) Personally, I think line coverage is often overrated and if developers focus on having a high coverage too much, they tend to write poor tests that do not really verify the functionality, however simple increase the coverage number. It is good to find verification holes though.

Certain kinds of randomized testing have their place (even in non-medical device code). For instance, property-based testing, as exemplified by the Hypothesis library in python, lets you generate data according to a spec and then assert that the result of a piece of code obeys some properties. But the data spec and the properties are usually only discovered once you write a lot of manual unit tests, so what you've said here still applies. :)

Testing can show the presence of errors, but not their absence.
–E. Dijkstra

Mutation testing can help to find issues with tests. The test framework will subtly modify your application during execution and check that your tests fail.

I really like my randomized data. You've made a good point though, I gotta overthink that.

Thanks a lot, fellow SDE!

Just subscribed to your channel. You're helping me take my Python game to the next level! Learnt alot, keep on throwing out content like this please!

I would like to recommend PyPI (pitest for python). It's a mutation framework that runs the tests, then changes the code and runs the tests again. If no tests fails the second time around, it is recorded as a "mutation that survived" and it will fail the build.

Your videos makes me want to sneak away and secretly code during the vacation with the family. If you have any tips on how to pull that off while still staying married that would be of great help :)

Hey Arjan. These are great videos you are producing, however, I missed the part where you described "test-driven development". Perhaps you could do a dedicated video on this topic.

He is the real MVP

"Make your unit test 100% deterministic"
Alrighty!
But then generating thousands of random test cases starting from a fixed RNG seed should be preferable over just using one test value, right?

Or would you argue that is more prone to introduce the same bug in your calculation in both the main code and the unit test, since the test value is (likely) calculated dynamically as well?

Also, would you agree that testing cases that are "close to" edge cases is advisable? E.g. here, testing tax exemptions that are just above and just below (and equal to) the pre-exemption tax amount to be paid?

Thanks for this video; I'm in data-analysis where there isn't a lot of versioning, but I'm still gonna add unit-testing to my Trello templates.

Nice video! A quick note on your advice never to use stochastic tests. It's awesome if you can avoid it (obviously)! But since I have an electronics background myself I can tell you that most of the digital circuit verification is based on randomized tests. Once the number of possible code paths starts exploding you have to resort to stochastic code analysis. Same goes for very complex software. Certain critical applications like code that flies airplanes and runs on critical medical devices obviously needs to obey to a much higher standard.

Great video Thank you <3 <3

Great video, thank you for explaining how bugs can exist despite 100% coverage.

A great video as always, Arjan! Thank you.

You Videos are awesome man !!!!

randomized unit test should have some use case I believe

Great video! Sugggestion: Can you turn down the like/subscribe sound effect? It was a bit loud

100% video coverage completed - 1 bug detected: the software development elders did not want to be mentioned.

Great video! When you pull up the html.index, do you do that inside VSCode (and what extension is that?) or is that just in your browser?

Another quality video, thanks Arjan 😊 A small tip: instead of having to define your vehicle object for every unit test, you can define it once using the setUp() method under your test case class as follows:

def setUp(self): # Camel case required
self.v = VehicleInfo("BMW", False, 10000)

You can then refer to self.v in all of your tests, and there's also a tearDown() method if needed

Great Video!

Please be kind enough to do a OOP video on python for developers whore are using other stacks.

sorry if it's a dumb question but what does it mean on the method that "-> bool" because i never saw using arrows unless programming in R

This is great, but I’d love to see how to completely cover functionality of an entire API! This includes API error handling, input validation, lower level database integration functionality and the business logic surrounding all this. Obviously testing the entire API call results in some seriously hefty tests and mocks so it’d be great to know how to split this out?

Would you unit tests the lower level functionality, and test the api layer independently of this lower layer by mocking the lower layer call?

Thanks

I really appreciate your videos Arjan. Great content. I predict a very fast growth for your channel. Keep it up. :)

Why do you use comments above the function or method definition header and not docstrings?

Awesome job with this tutorial series Arjan!

another great video, but that semicolon almost gave me a heart attack

Your videos are sooooo good, man! It pains me to such few views and subscribers