Fixed-point math is better than floating point (sometimes)

🎉

Old trick. Do not use decimals but multiple the value so you get rid of decimals. Works with weight and money calculation at least.

The most annoying thing to me is that IEEE754 support in languages usually only ever support the binary case but decimal floating points also covered under IEEE754 are so much more useful even if they are slow. Things suck

Really entertaining video. Thanks!

Actually 10011100 is -100 is two's complement representation.

IEEE 754 entered the chat

I think what would have been nice to mention is that floating point is essentially scientific notation, i.e. 12.34 is 1.234e1, just that floats use 2^n instead of 10^n for the exponent, which is where the scaling you mention comes from

for fp_abs why not just return abs of a?
return abs(a);

seems to work fine in C#:
public FixedPoint Abs()
{
return new FixedPoint(Math.Abs(Value));
}

This is a really nice demonstration by example, and it does have great utility. However, there is one vital part to any type of mathematical or arithmetic library especially when it is being evaluated within a computational framework, context or domain especially within the integer domain, and that is integer division in regard to its remainder as opposed to just the division itself. No such library is complete without having the ability to perform the modulus operator. Not all but many languages use % to represent this type of operation. It would be nice to see a follow up video extending this library to include such a common operation. Even though the modulus operator itself is fairly considered an elementary or basic operation or operator, its implementation is complex enough that it would almost warrant its own separate video.

Why do I mention this? It's quite simple. If one wants to use this as an underlying math library and wants to extend this into using it within other domains such as with performing or evaluating trigonometric functions such as sine, cosine, tangent, exponential functions such as e^n or even logarithmic functions as well as extending into other types of number systems such as in various vector spaces, particularly but not limited to the complex numbers; having the modulus operator as being an already well defined and operational operator between two operands is vital into performing most other complex types. In simple terms, the modulus operator (%) is just as important or significant as other operators such as +, -, *, /, ^, root (exp, rad). And this is just the arithmetic half, there is still the logical half of operators. Other than that, great video!

😀

All your content is top rate. Love the low level stuff that we dont need to know, but cant sleep w/o knowing.

From the first time I read about fixed-point for the 286 in an old DOS book I've always liked it more than floats, I think it's going to make a comeback after decades of FPU!

struct timespec is a great example of the fixed-point integer number. You have tv_sec, which is just time_t signed integer type, and tv_nsec, which is a signed long type that only purpose of is to represent values from 0 to billion minus 1 (999,999,999) inclusive. With some helper functions you can do very robbust and easy math if you treat tv_nsec just as accumulator that when overflows adds 1 to tv_sec and when it underflows subtracts 1 from tv_sec. Easy, quick, no floats needed. Not all systems even have that kind of precision for timestamps, so having nsec precision is good enough.

Nice that you did this in 32-bit... I've been looking for a "nice" 32-but fixed-point implementation for a long time... I have this idea of building a synthesizer on a network of PIC32s... and, floating point, aint nobody got time for that!
... I had in mind to do this in Zig... because then I could use `comptime` to turn my human readable constants into my chosen fixed-point format. But this is entirely an armchair theoretical project at the moment.

One nice thing is that fixed point arithmetic gives you exactly the same result on every computer architecture, but floating point often does not... because floating point implementations make different choices with the least significant bits of number representation... not so much during simple arithmetic operations but definitely for reciprocal, trig, exponent and log. Sometimes close is not enough and exact identical results are more useful. Also, sometimes the generality of floating point requires more CPU cycles than equivalent fixed point operations....

The quickest way I use to explain fixed point is instead of $4.20, you have 420 cents.. it's obvious those are the same even though 4.2 != 420

In fp_ceil, why use the fp_frac function. Wouldn’t it be quicker to just AND with the frac mask and check if the value is greater than 0. Given that we don’t actually use the value, just the presence of any bit set would be enough to know it’s got a fractional part.

Also, fp_floor for positive numbers is just fp_floor(a+ Half) and negative is Fp_floor(a-Half)

Would it be worth it to make those functions and macros branch free? Or does the compiler do it already? Is it even possible? Or not worth it?

Any thoughts on posits?

The issues with floating point vs fixed point is quite simple.
Floating point - Why the hell are you looking at those digits, you ought to damn well know that format doesn't support that many significant digits.

Fixed point - Why the hell are you looking at those digits, you ought to damn well know that your data doesn't justify that many significant digits.

To illustrate, the vast majority of numbers you manipulate on a computer are actually approximations of some other non-representable exact value. Fixed point suffers from what's called "false precision". To illustrate, I'll calculate the circumference of a circle with a diameter of 123. I'll do it twice. Once with a fixed point decimal format with 5 integer digits and 5 fractional digits. Again, with a floating point format with 8 mantissa digits and an exponent from -49 to 49. So we have PI * 123. Let's see what happens:

Fixed point
123 * 3.14159 = 386.41557

Floating point
123 * 3.1415927 = 386.41590

Actual value to 10 digits = 386.4158964

The thing to notice is that the fixed point value's last 2 digits are WRONG. They are wrong even though the multiplication occurred with no rounding or overflow. The reason for the error is as I said earlier, most numbers manipulated by computers are approximations of some other non-representable exact value. In this case, the approximation for pi only had 6 significant figures and as such, you can't expect more than 6 figures in the result to be correct. For the floating point case, the approximation for pi had 8 significant figures and as such, its result is correct to 8 places.

False precision is a definite problem with fixed point math. And it's a rather insidious problem since the actual mathematical operations are frequently done with no overflow or rounding. But you can't trust your results for any more digits than the smallest number of digits used for your inputs or that of any intermediate results. But with floating point, the number of significant digits remains relatively constant.

What a timing: yesterday I decided to look into fixed point numbers because I was having some problems with my floating point rasterizer. This video is immensely helpful with getting a better understanding of fixed point numbers. I'm looking forward to learning about trig functions for this stuff.

True heroes use fractions and or binary coded decimal 😅

You know what's also a surprisingly useful algorithm when dealing with fractions if all you have is integers? Bresenham's line algorithm! The whole "drawing a line" thing is a bit of a diversion of the true genius kernel of that algorithm: how to do error-free repeated addition of fractions, and only trigger an action every time you "cross" a whole-number boundary (in the canonical case: drawing a pixel). And all you need is three integers (an accumulator, a numerator and a denominator), integer addition, and an if-statement. Even the lowest-power hardware can do that!

Surprisingly, integer multiplication and division are generally slower than floating point multiplication and division on modern x86/x64 CPUs! I have no idea why as I'm not a hardware guy, I just spend too much time reading instruction tables.

So I literally just spent the past two weeks implementing the base-10 fixed-point math in an emulator I wrote. The emulator was something I created during the pandemic from and old interpreter I had written in the late 80s on an early 80s 32-bit machine. I guess emulator is a bit misleading as this is a ROM emulator (i.e. it doesn't emulate the hardware of the machine, but rather the software). You can find the integer only version by searching for ZXSimulator on the web (the ROM emulator is for the ZX81 that is itself implemented on a 32-bit 80s system called the Sinclair QL -- yes, nesting of old computers but hey, it's a hobby).

The original interpreter was integer only to keep it fast (it was meant as a scripting language). Also, the 80s 32-bit machine it was implemented on had a limited C compiler with no floating point support (i.e. Small C). It does provide an add-on floating point library but it's a bit wonky as it uses function calls and a 3D 16-bit int array data structure (so 48 bits) for holding the floating point values...so it's going to be very slow. The scale factor is 100 and you multiply and divide by it, same as you did above with left and right shifts, it's just a bit slower since they are math and not bit operations..

Btw, since I don't have the XL type (32 bits is all), there is a way to fix the overflow issue for multiplication (I haven't figured out what to do about division though).. You can break up the whole number and fraction parts (for me, using / and %) and then this is your multiplication formula: a*b1 + ((a2*b2)/100) + (a2*b1); where a represents the entire number (upscaled by 100), b1 represents only the whole number part of b (gotten by / 100), and a2 & b2 represents the fraction part (gotten by % 100). This will increase the range of numbers (i.e. size) you can multiply (although at a cost of speed).

You can do sin/cos with your library, but you already know this, just being a bit pedantic. It's the Taylor expansion but it's quite compute-heavy. You can do it without division by using some precomputed polynomials. And there's the preferred way, which you will probably present next. Hopefully it's not lookup tables :)

This video is so good... taking high level concepts that we often think of as a simple, almost atomic, operation and breaking them down to the next lower level. I like to play with assembly language for very simillar reasons.

I've implemented plenty of fixed point arithmetic in DSP data paths in wireless communication chips.

This was really interesting I might actually try implementing it myself for a bit of fun.

Slide rules

One day we'll get hardware support for posits and then all of this will be solved

Instantly subscribed!

Personally I loved how easy it is to do fixed point maths using integers. Floats is a complicated format and either needs a lot of code to emulate in software or a lot of silicon to do it in hardware. But for fixed point, all you need is an ALU :)

For FFT, Floating point is way better, be prepared to wait a longer time with fixed point.

Yes, decimal arithmetic is essential for exact arithmetic. But... Instead of the extra code for scaled integers or decimal data types in custom or provided libraries, you can just do this:

01 WS-ORDER-TOTAL PIC 9(4)V99 VALUE 40.50.
ADD 1.50 TO WS-ORDER-TOTAL

Still used in critical systems today and introduced in 1960.
So understandable that even an auditor can check it. :-)

When you use fixed point, usually your main objective is keeping your resolution as small as possible. Therefore dedicating a large number of bit to the integer part seems wrong to me. What I usuall do is dedicating one bit for sign (if any), one bit for integer part and all remaining bits for fractional parts. To do so, you need to normalise all values first.

Furthermore, I found that 16 bits for fractional part is more then enough. This is why fixed point in FPGAs uses typically 18 bits.

What about a video on tips/tricks on how to avoid the floating point issues when doing calculations?

Awesome. Subscription well earned!

Thanks for the video, a question, what if my hardware doesn’t support float points operations, how would do the conversion between float and fixed point . As you multiply the float by scale to convert in your case

Fantastic video! Thanks so much for doing this!

I think that your implementation of floor is wrong for negative numbers, because for floor you round towards 0 always, whereas floor should round towards -inf.

So, for example, floor(-18.2) should be -19, and not -18 as you corrected for. This is also what happens in Python, and what is shown on the wikipedia page for IEEE-754.