App/Explorer ETL behind for 90 minutes
Incident Report for Helium
Resolved
The ETL that serves the App and Explorer was a having a hard time processing block 1,139,915 and took about 90 minutes to fully catch up to the tip of the chain. The root cause is the activation of PoCv11 and the new H3dex that can take a very long time to cache and store.

While ETL was catching up, the pending transaction queue also took time to clear. Some users have observed up to 90 minutes to clear a transaction. The types of transactions include:
* Add Hotspot
* Update Location
* Payment Transactions
* Transfer Hotspot
* Burn transactions

The core team is working on performance improvements but until then, we can expect a few more slowdowns in the future but we are actively monitoring.

The chain continued moving forward at this time, and there was no chain outage.
Posted Dec 15, 2021 - 04:36 UTC