We will be refunding the affected duplicate charges in bulk as soon as possible. Customers may not see the changes reflected on their account for up to 5-10 business days, which is determined by their bank. Subscription status will not be affected, and future rebills will process normally. Some rebills which would have been included in today's process have been skipped and will run tomorrow.
UPDATE (10/1 13:00 UTC):
The bulk of subscriptions affected were charged twice on the gateways but have a single active subscription related to that rebill. We have reached out to our payment processors to refund these subscriptions in bulk as soon as possible.
A smaller subset of subscriptions was charged once, but those charges weren't recorded in our platform. These were placed in dunning and would run the risk of deactivating before we had a chance to refund them. For these transactions, we have updated their next date to October 3 in the platform to make sure we have adequate time to avoid deactivation.
UPDATE (10/2 21:00 UTC):
97% of all duplicate or otherwise improper charges have been refunded. We expect to have any remaining cases cleared up Monday. The complete resolution of the effects of the incident continues to be a top priority. A thorough analysis of the root cause and preventative measures for the future will also be part of our overall action plan.
Posted Sep 30, 2020 - 18:12 UTC
Monitoring
We are evaluating the next best steps to refund buyers any duplicate charges.
Posted Sep 30, 2020 - 16:52 UTC
Identified
The cause of the issue has been identified as a duplicated server. The problem has been remediated for subscription rebill charges after 1515 UTC, and we expect no duplicates after that time. New orders were not affected.
Posted Sep 30, 2020 - 15:15 UTC
Investigating
We are investigating cases where the rebilling process for subscription caused duplicate charges on some orders.