Get In Touch
Salesforce Development

Salesforce Permissiongeddon: Outage Recap

On Friday, May 17th, Salesforce experienced system-wide degradation issues, impacting many Salesforce and Pardot users around the world.

What Happened

The Salesforce Technology team deployed an application database script that was supposed to be applied to a subset of Pardot users. It was inadvertently applied to all users in orgs that had Pardot provisioned at any time, either currently or historically. This deployment allowed for all users (including community users) to view and modify all Salesforce records.

Fearing that this would lead to unauthorized access, Salesforce made the decision to block access to the Salesforce service for all orgs (within both production and sandbox environments). Once the Technology Team was able to isolate the affected organizations, they restored access for non-affected customer organizations.

Salesforce outage map
SOURCE: DOWN DETECTOR

The Fix

On Saturday, May 18th, customers who were unaffected by the scripting error had their access restored in full. Affected organizations had access restored to their System Administrator users, but other profile types were still altered and did not allow for full access or functionality. Providing access to administrators allowed organizations to restore profiles and permissions internally, but in parallel, the company also began to execute automated provisioning to restore previous settings on a rolling basis.

As of Monday, May 20th, a subset of customers was still impacted, despite the automated provisioning. Many of these users ultimately had to choose between opening a support ticket (and waiting yet again) or restoring settings internally.

The Takeaway

Salesforce is usually a highly reliable service, with few outages in its history. This incident, in particular, caught Salesforce customers off-guard. Many users hoped to use their recently refreshed Sandboxes as a backup source, deploying permission and profiles back to production. However, the application database script impacted both production and sandbox orgs, leaving that option as a non-starter for most.

This entire incident has highlighted the importance of keeping multiple back-up options in place. Waiting on the Salesforce Technology team for can be crippling when an incident is widespread, as they have to deal with thousands of accounts.

While most users are back up and running in full capacity, it is still possible that you will run into issues with permissions and profiles. If you experience an issue, either open a ticket with Salesforce or contact us to let us know.


Jessica Hope

I consider myself a Marketing Cloud Account Engagement (Pardot) evangelist. I’ve focused almost exclusively on Pardot for the last seven years, and I've had the great honor of working in hundreds of accounts on every imaginable type of project. I spend most of my time working directly with clients to meet their every last Pardot goal, but I also enjoy writing about and/or training on all things Pardot.

Let’s connect to see where I can help. Email me at: jessica.hope@parquetdevelopment.com

Leave a Reply

Your email address will not be published.

Categories
Privacy Settings
We use cookies to enhance your experience while using our website. If you are using our Services via a browser you can restrict, block or remove cookies through your web browser settings. We also use content and scripts from third parties that may use tracking technologies. You can selectively provide your consent below to allow such third party embeds. For complete information about the cookies we use, data we collect and how we process them, please check our Privacy Policy
Youtube
Consent to display content from Youtube
Vimeo
Consent to display content from Vimeo
Google Maps
Consent to display content from Google
Spotify
Consent to display content from Spotify
Sound Cloud
Consent to display content from Sound