To begin, just a couple of comments on your existing formulas:
The final "if" in your Z4 formula is redundant and in fact if B4 is 5.45 it will return what is probably an unexpected value of FALSE. Also the quotes around 0.3, 0.2 and 0.1 are, at best, superfluous - the formula as you have it is returning text not numbers. Your Z4 formula can be rewritten more simply as:
Formula:
=IF(B4>=12.5, 0.3, IF(B4>=5.5, 0.2, 0.1))
Similarly for your B5 formula the final "if" is redundant. It can be written as:
Formula:
=IF(C4>36, B4-Z4*(C4-36), B4)
Lastly, if you wished to avoid the need for your hidden Z4 formula you could consolidate the Z4 logic into the B5 formula as follows:
Formula:
=IF(C4>36, B4-IF(B4>=12.5, 0.3, IF(B4>=5.5, 0.2, 0.1))*(C4-36), B4)
That said, keeping them separated may keep the separate formulas more readily understandable.
Now to your actual question - I am having difficulty seeing the logic that you want. With reference to your example, if B4 is 12.4999 you are happy with your current algorithm which subtracts a fixed multiple (2) of 0.2 from B4 and yields B5=12.0999. Also, if B4=12.5000, just 0.0001 larger than the last example, your current algorithm will subtract a fixed multiple (2) of 0.3 from B4 and yield B5 = 11.9 which is already a discontinuity. Is this OK and what you expect? Now you want to change the algorithm to subtract first a multiple of 0.3 and then an additional multiple of 0.2. This is going to yield an even bigger discontinuity.
In your example, exactly what value do you expect for B5 when B4 = 12.5?
Can you explain in more general terms what you are trying to achieve? That might help provide enough insight to be able to offer a solution - or maybe someone else can see what I can't and jump in.
Bookmarks