You will not need to disconnect from the internet in order to access your router's management GUI. You simply need to make a wifi connection to the router and need to know the IP address of the router on your local network/subnet (it's likely you can also find this as the gateway on your connected device, such as a laptop or mobile).
The reason you are able to access your router even with an active internet connection is because the router is configured with its own private subnet for serving devices within your home and this private subnet will not clash with anything routable on the internet. Typically your router acts as your gateway to the internet and will apply NAT to hide all your other devices behind a single IP address (historically this IP address has been an internet routable public address but more recently CGNAT has been more common and the IP address your router receives is not routable on the internet).
I'm in the process of leaving a service provider whose name is also a number and they have different APNs, some using CGNAT and one (possibly more) providing an internet routable IPv4 address. I've moved to my current service provider, who shall not be named, and I believe its SIMs use CGNAT for internet/data access.
