The product designed for this purpose is the Huawei BBU15-D Battery Backup Unit, which BT used to supply with its FTTP installations. This plugged into the mains and took 4x AA rechargeable batteries(*), additionally it came with a 12v cable so it could plug straight into the router.
However, an Internet search for these and other UPS/Battery backup for router devices indicates they are no longer as readily available as they were 1~2 years back.
There are several units being advertised on Amazon claiming to be suitable for routers, however, I note they provide a maximum output of 12v 1a - the EE routers all require a 12v 1.5a power supply. Having accidentally attempted to use 12v 1.5a power supplies on routers requiring 12v 2a, I would not recommend attempting to use these devices. [Aside: using 12v 2a power supplies for 12v 1.5a equipment doesn't cause problems as the equipment will only draw 1.5a.]
I also would not recommend using a UPS intended for a computer, unless it is also being used by the computer. This is because the energy draw of the router is typically too low for the UPS to register and so can result in battery drain and damage.
(*) This is useful as the batteries will cease to hold charge after a few years and so can be easily replaced by any rechargeable battery available on the high st.
In general, my experience of power cuts is that they are mainly short, hence simple battery solutions will work, for longer outages (4+ hours) where laptop batteries are starting to run low the only real solution is to get in the car and visit somewhere outside of your area...
