Proof-of-Coverage and Data Transfer degraded for some Hotspots in AU/NZ
Incident Report for Helium
Resolved
This incident has been resolved.
Posted Apr 04, 2023 - 05:37 UTC
Update
We are continuing to monitor for any further issues.
Posted Apr 04, 2023 - 02:32 UTC
Monitoring
The Configuration Service has been updated. All AU/NZ Hotspots should be receiving the AS923_1 configuration now. Hotspots make take up to 30 minutes to receive the changes. Thereafter, Proof-of-Coverage Beacons and Data Transfer should resume as normal.
Posted Apr 04, 2023 - 02:32 UTC
Identified
Unfortunately, many Hotspots in Australia and New Zealand do not appear to have the required firmware elements to support the new "AS923_1C" band plan being delivered by the Configuration Service. We will be rolling back the Configuration Service to deliver the older AS923_1 plan over the next several hours.

Once this rollback is complete, we will ask the various Hotspot Makers to ensure that they have support for the AS923_1C plan and re-enable it.
Posted Apr 04, 2023 - 01:17 UTC
Monitoring
The Core Developers have applied the fix to the Configuration Service. All Hotspots in the AU/NZ region should now be receiving the correct configuration, and if the Hotspot Maker has set up the firmware correctly, the Hotspot should see the correct configuration and adopt it within 30 minutes.

We are seeing evidence, however, that some Makers are not prepared to use the information correctly when it is provided. We will update the community as we learn more.
Posted Apr 03, 2023 - 22:50 UTC
Identified
The Core Developers believe that the issue stems from a configuration in the Configuration Service, which tells Hotspots which region plan to use. A fix to the Configuration Service is being tested.
Posted Apr 03, 2023 - 20:23 UTC
Investigating
We are investigating an issue where Hotspots in Australia appear to be listening and operating on the wrong frequencies after several Makers have updated their firmware to use the newer gateway-rs firmware. Affected Hotspots will not Beacon, Witness or transfer data correctly.
Posted Apr 03, 2023 - 19:17 UTC
This incident affected: Proof-of-Coverage, Helium Packet Router, and Configuration Service.