Unfortunately, the IET wiring regs always defer to the manufacturer's instructions. If the manufacturer specifies a 3A fuse, then there must be a 3A fuse to meet the regs.
The requirement is that an electrician must be able to isolate the fan to work on it in safety. It's debatable whether natural daylight is sufficient if it's a dull rainy day or evening call-out. Isolating the lighting circuit at the MCB may be sufficient, but in the case of a serious fan fault (smoke) means that you can't get the lights back on... but this is all secondary to the wiring regs requirement that I mentioned above, you need to fuse it at 3A to be compliant.
P.S. If you look at the Toolstation website for Click Grid modules, it's possible to fit a 3 pole fan isolator switch (Permanent live, switched live, and Neutral) and a fuseholder into a single width front plate. Product codes 75262, 93193, and 49473. Not expensive.