Smart Hub 7 Pro 5 GHz failure reproduced on two hubs - channel/firmware issue?

[Mellman]

Hi all,

I am starting a separate thread for this because although the problem originally came to light while troubleshooting repeated YVM102/connectivity problems with our EE TV boxes, the testing we have done since then appears to have uncovered a wider Smart Hub 7 Pro 5 GHz issue.

We are on EE Full Fibre using an Openreach ONT and a Smart Hub 7 Pro. We also have an EE TV main box downstairs, an EE TV Mini upstairs and an EE WiFi extender.

The original symptoms were intermittent connectivity problems with the TV boxes and other devices. Eventually I noticed that our gaming laptop could still see the normal EE WiFi network but would sometimes be unable to connect to it over 5 GHz. If I changed the wireless adapter back to dual band it would immediately connect using 2.4 GHz.

After extensive troubleshooting with EE, including a factory reset of the original Smart Hub, Executive Complaints arranged for a replacement Smart Hub 7 Pro to be sent.

I deliberately installed the replacement in a very controlled way. The EE extender was left completely powered off and initially nothing else was connected to the replacement apart from the Openreach ONT. I connected my desktop directly to the new hub over WiFi.

The desktop uses an Intel WiFi 6E AX210 adapter. On initial setup it connected normally to 5 GHz with a strong signal and no packet loss. At one stage the replacement hub was using channel 36 and everything was working as expected. The gaming laptop, which uses a separate Intel AX200 adapter, also worked normally.

The replacement hub then remained apparently fine for around four days.

Unfortunately the same problem returned.

The gaming laptop lost its 5 GHz connection again. When its adapter was restricted to 5 GHz only, it could see the EE SSID but Windows reported "Unable to connect to this network". Returning the adapter to dual band allowed it to connect immediately over 2.4 GHz.

I then repeated the same test independently on the desktop. When I restricted the AX210 to 5 GHz only, it also could not connect.

At this point the EE WiFi extender was still completely powered off, so the extender or mesh roaming can be excluded from these tests.

Hub Manager showed the main 5 GHz radio as enabled and Smart Channel had selected channel 100.

I ran:

netsh wlan show networks mode=bssid

on the desktop. Initially the only BSSID being shown for our EE network was the 2.4 GHz BSSID. The Smart Hub's 5 GHz BSSID had disappeared completely from the scan even though Hub Manager continued to report 5 GHz as enabled on channel 100.

I then used the 5 GHz Smart Channel Rescan option in Hub Manager. It remained on channel 100 and did not resolve the problem.

Next I switched only the Smart Hub's 5 GHz radio off, saved the setting, waited around 30 seconds, switched it back on and saved again.

After doing that, the 5 GHz BSSID became visible again on channel 100 with a strong signal of around 80 percent.

However, neither the desktop nor the gaming laptop could actually connect to it.

This was particularly useful because it showed that simply making the 5 GHz radio broadcast again did not restore the ability of clients to associate/authenticate successfully.

I generated a Windows WLAN report on the desktop while the fault was live.

One failed connection attempt is particularly interesting. Windows recorded that wireless association with the Smart Hub's 5 GHz BSSID succeeded. WPA3-Personal security then started, but approximately five seconds later the security stage failed.

The reported reason was:

"Dynamic key exchange did not succeed within configured time"

Windows also logged:

"PSK mismatch suspected"

The WiFi password was definitely correct. The same SSID/profile/password continued to work on 2.4 GHz and had previously worked normally on 5 GHz on the same replacement hub.

The WLAN report also confirms that the Intel AX210 supports WPA3, SAE and H2E, so this is not simply an adapter which lacks WPA3 support.

As a further test, I enabled the Smart Hub's Compatible WiFi network on 5 GHz. This uses WPA2-Personal rather than WPA3.

Hub Manager showed Compatible WiFi 5 GHz as enabled on channel 100.

However, when I restricted the AX210 to 5 GHz only, the Compatible WiFi SSID disappeared from the available network list completely. The normal EE 5 GHz SSID remained visible but could not be joined.

That makes me less convinced that this is simply a WPA3 problem. It seems possible that the failed WPA3 key exchange is one symptom of a wider 5 GHz radio, virtual AP, Smart Channel or firmware state problem.

At this stage I performed a normal restart of the Smart Hub through Hub Manager. This was not a factory reset.

After the restart, Smart Channel selected channel 149 rather than channel 100.

The result was immediate.

The desktop automatically connected successfully to 5 GHz again on channel 149 using WPA3-Personal.

The gaming laptop also immediately regained proper 5 GHz connectivity. Windows shows it connected on channel 149 using WiFi 6/WPA3, with a current link speed of around 721 Mbps receive and 360 Mbps transmit.

The upstairs EE TV Mini box, which had also started having connectivity/interface problems again at approximately the same time as the 5 GHz problem returned, also began working normally again after the Smart Hub restart.

I am not claiming that this proves channel 100 itself is the cause because the restart changed two things at once. It reset the hub's internal state and also caused Smart Channel to move from channel 100 to channel 149.

However, the observed sequence on the replacement hub so far is:

5 GHz working normally while the hub was using channel 36.

5 GHz subsequently failing while Smart Channel was using channel 100.

Two completely independent computers unable to use 5 GHz.

EE extender completely excluded from the setup.

Smart Channel Rescan did not resolve the problem.

Cycling only the 5 GHz radio caused the BSSID to become visible again but did not restore client connectivity.

Windows WLAN diagnostics recorded a WPA3 dynamic key exchange timeout against the Smart Hub's 5 GHz BSSID.

The WPA2 Compatible WiFi test also failed to provide a usable 5 GHz alternative.

A full Smart Hub restart then moved 5 GHz to channel 149 and restored normal 5 GHz operation on both computers.

The EE TV Mini also returned to normal operation following the same restart.

What makes this especially concerning is that this is now the second Smart Hub 7 Pro on which we have experienced very similar 5 GHz association failures.

The replacement hub is running firmware:

r4.29.1-R-1930437-PROD-1

The previous Smart Hub was also running this firmware when we reproduced the earlier 5 GHz failure.

The desktop and gaming laptop use different adapters, Intel AX210 and AX200 respectively, and both are now behaving normally again following the hub restart. That makes a fault with one particular client considerably less likely.

The lack of manual 5 GHz channel selection on the Smart Hub also makes this very difficult to isolate further. Ideally I would like to be able to lock the hub to a known working non-DFS channel such as 36, 40, 44 or 48 for several days and see whether the fault ever returns, but as far as I can tell the Smart Hub 7 Pro does not allow this.

For now I am leaving the hub exactly as it is on channel 149 and monitoring it.

If the fault returns, I intend to capture the selected channel, BSSIDs, client behaviour and WLAN diagnostics again before restarting or changing anything.

I would be very interested to hear from anyone who has seen similar behaviour with the Smart Hub 7 Pro, particularly a situation where 5 GHz remains visible but clients suddenly cannot associate/authenticate until the hub is restarted.

I would also be grateful if someone from EE could advise whether these findings can be passed to the team responsible for Smart Hub 7 Pro wireless/firmware development. Having now reproduced similar behaviour on two separate hubs, with multiple independent clients and the extender excluded, I do not think another factory reset or replacement hub alone adequately explains what is happening.

[JimM11]

@Mellman Good lady has a AX211 in one the other is an AX201 so not fully same as yours but zero issues on either.

[JimM11]

@Mellman On the Main wifi if that is what you are using, then GO look at what the securities are set at, got NO idea what the Pro has available for use, do NOT think anything is to do with CH being selected so that just may be a red herring, my units apart from the later ones that are newer, but the Asus is set WPA2-PSK or Personal that some prefer to call it, think you may not be able to push the Pro main down but just look see. Have PM'd so you are aware, EE MOD will go edit out all your MAC/SSID's do not publish them it is a public Forum for all to see... xx out what you don't need to display! 

The WLAN-AutoConfig entries may show you if the unit is struggling securities connection in it's logging file. 

Nothing in your history so when did you do the 6 Plus to the 7 Pro jump, did ignore your Third party kick off so you know that opened up old wounds but has spurred me on to sort out Sister's mess end off Sept.... 

I did a couple off Netspot, PC version captures couple weeks ago if you need to go look see linked them below.

Re: Home Hub Pro & WPA - Page 2 - The EE Community

[tnj]

It is only a thought but you had more luck on channels 36 and 149 and these are the channels which are not DFS channels. 100 sits in the DFS range and if the router detects a radar signal it should stop transmitting for around 30 mins or it could change to a non-DFS channel. I have no idea how this is implemented on the EE routers and you have little choice in bandwidth and getting a channel to lock. It may not be your issue but it could be something to consider.

[JimM11]

@tnj Valid point normally will fold back still on ch100 but to a 80Mhz wide so 100(106) and yes small blip does happen at times, OP is saying though 100(114) still there blasting away but devices have just hung up there spur's so to speak....

Really just need to get a single device that is happy and unaffected, for me when issue with the Samsung mobile, sat it couple feet away from the Main Hub, it never once miss behaved for the week or so, took it walkabout the home again and sure did it's down 30Mb/s  no recovery apart from wireless off/on, additional AP sorted it out and i would swear blind and until this day NEVER wifi weak is not the problem, sugar happens and just this one mobile the wifes was fine no matter what.

[Mellman]

Thanks both. I agree DFS is definitely something worth keeping in mind as a possible trigger.

The bit that makes me unsure whether this is normal DFS behaviour is what happens once the fault is present.

During the latest failure WiFi Analyzer was still showing the main Hub 5 GHz BSSID strongly on 100(114), 160 MHz. Windows could also see that same BSSID at around 82 percent signal, but multiple clients had fallen back to 2.4 GHz and an AX210 forced to 5 GHz repeatedly failed to associate.

I also tested cycling just the Hub's 5 GHz radio during the previous recurrence. That caused the 5 GHz beacon to become visible again, but clients still could not associate. A full Hub restart restored everything immediately.

After that restart the Hub initially chose channel 149, but later automatically returned to 100(114), 160 MHz and continued working normally there. So channel 100 itself definitely works with these clients when the Hub is healthy.

That makes me wonder whether a DFS or Smart Channel event could potentially be triggering a bad internal state rather than the failure simply being the expected DFS evacuation itself.

One other difference from Jim's Samsung example is that this is affecting several unrelated clients at the same time. I have an AX210 desktop, an AX200 laptop and a Samsung phone all dropping to 2.4 GHz, with the EE TV Mini developing its connectivity symptoms at the same time.

I've now sent EE Executive Complaints a full technical report and the original WLAN diagnostics, and they are forwarding it to the relevant team, so hopefully they may be able to see something useful at the Hub/firmware end.

[JimM11]

@Mellman Stick to it you are doing everything needed, i really get very little DFS issues 25 miles middle off two Airports but i am direct in the Landing path OH flights wise, Channel 100(114) never folds back sticks well up on 160Mhz, then again and the current versions used still on connection but the Asus is just so rock solid and no PW issues what so ever, if a device ie my Samsung actually few devices that are not chain linked Asus wise, the phone will latch onto a node does stay up on the 5Ghz mesh so does not drop back to 2.4Ghz band ssid's are all split apart from the PL node in the garage if it does get slow 5 second off on the wifi connection for the mobile then it not going to say 100% but up in the high 90's will connect correctly and back to the 1200mb/s sync rate....

All the models are still on @Mellman and not seeing any issues at all with wifi connections. Laptop with the 6E on it have chained it to various nodes over the last few day's and holds as expected, will take a look see later when i get a minute.

Edit:- nothing to report all still working as expected device wise, Laptop still connected to the furthest away node on the 160Mhz channel speed down due to the distance but no disconnect's at all well not logged anyway.

It may be worth looking at MLO connection just incase your devices/hub are getting confused and not knowing which way to path...

[jak26]

I don't think I'd have patience with daily issues on 2.4Ghz or 5GHz. I like to keep the ISP router (7+ for EE) connected to the ONT to make support calls easier - and the 7+ works fine in Compatible 2.4GHz WPA2 Only - but use a Asus tri-band with multiple 2.5Gb/s LAN ports as an AP nearer to devices.

ISP routers are only on loan for the duration of the contract - ones own router as a ISP router replacement or as an AP is a small investment if you have a good number of devices and an interest in network things.

The Asus seems to pick a non-DFS 5GHz channel away from neighbour routers but I don't think channel changing is an issue for most clients - it's a basic Wi FI function. One time DFS might be a factor is just after a restart when there's a delay before the 5GHz SSID becomes visible.

Similarly with WPA - Ive got one device that needs WPA2 Only but all 5GHz devices work OK with WPA2/WPA3 and 6GHz with WPA3 - everything connects and stays connected

The 6GHz band has issues as well as range as some devices seem reluctant to see the SSID - so although 6GHz can be 2x speed, 5GHz and wired LAN do the job.

[JimM11]

@jak26 Used to have the day's upon day's DFS from restart, but will say once the stupid Asus does make it's mind up it hang's in there no issue, may be a completely different but since the last Asus FW update, the Main Node fires up CH/Width right out off the power on and does not mess about anymore, straight up and at it far as i am concerned that's what it should do, it knows what the area is like had before been up and done so why mess it up, can always fold back to 80 wide if it needs to do so. No 6Ghz for me on the Tri Band so not a factor, would never work in my home anyway no way to get through the brick walls....

[jak26]

After checking again the Asus is picking a DFS channel - there's not that many non-DFS to pick from - just 80 wide but that's quick enough for most things - but at least it stays steady. If it was to change then clients should just reconnect after a bit but I haven't noticed any streaming radio or TV etc dropping. On a previous AP it had a rule that it would only change to a non DFS to avoid constant changes. The few times I have main enabled on the 7+ (like the switches back on after a restart bug) it defaults to DFS as well.  I guess the give away is the delay before the 5GHz SSID appears.