If you are designing some circuits, you probably know or by intuition add bypass capacitors to every IC power rail. Almost everywhere you will find scarce info that you need to place cap as close to IC as possible. In order to get deeper understanding why these caps are needed, sana syed on electro-labs wrote a tutorial about bypass capacitors explaining the meaning of it and what size of capacitor is needed.
The main reason why bypass capacitors are used is to filter out high frequency noise and glitches from DC voltage. It can be power supply or DC biasing. If you know fundamental electronics, then its clear that capacitor allows AC to pass while stops DC. So if carefully selected capacitor it passes high frequency noise directly to ground and leaves DC voltage cleaner. It turns out that not any capacitor will do its work as expected. Some calculations may help refine the choice and filter out undesired AC noise or simply pass AC signal without affecting DC part.