How to prepare for Bitbucket outage?
When it comes to intellectual property, not only C-level but every conscious security and DevOps leader, as well as senior developer or architect, understands the importance of its protection. They search for a source control system that will allow them to experiment with the features without impacting the source code they create.
One of such git-based source code repository hosting services is Bitbucket. The service was launched by Atlassian in 2008. It boasts over 6 million developers and 1 million professional teams. It offers commercial plans and free accounts with an unlimited number of private repositories. It is a trustworthy git-based service platform and we can add more and more highlights and advantages to enlarge this list, but… There always appears some “but.” What if there is an outage? Who will be responsible for data security? And the most important is how to get the data back immediately?
Outages still happen
Atlassian is one of the most reliable services, though, let’s face the truth, troubles happen. If we look at the Bitbucket Status at the official Atlassian website, it is possible to track all the unpleasant incidents Bitbucket users experienced. Only this year incidents occured in April, June, August, September. Though, these Bitbucket outages weren’t as serious as the one that happened to the Atlassian Jira environment in April 2022 when about 775 users couldn’t access their data for an unimaginable long two weeks.
August 2022
Atlassian Bitbucket platform experienced a partial outage for git operations via SSH in Germany on the 30th of August 2022. Some customers couldn’t access their data for 15 hours. It may seem not much, but what loss could those companies suffer?
September 2022
Next, September the 25th. At 1:46 PM UTC Attlassian reported the Bitbucket partial outage across all services. Some Bitbucket Cloud customers were unable to access their repositories for 7 hours. “The outage was caused by a firmware upgrade resulting in a subset of their storage clusters failing to update correctly.” – that was said in the summary report on Atlassian support webpage.
However, the main conclusion that should be drawn is that the Bitbucket environment needs proper protection if you want your DevOps team to keep coding continuously. Well, who is actually responsible for data accessibility? At this point we have approached a very tricky topic – the Shared Responsibility Model.
Atlassian is a service provider. And, as any other vendors, it operates in accordance with the “so-called” Shared Responsibility Model. It means that the provider is responsible only for the security of the service and infrastructure while the customer should take care of the account and data security.
Let’s sum up: Atlassian figures out all the issues concerning the system, hosting and application focusing only on their own business and integrity. A user has to deal with his data protection by himself as account-level protection and restore is not included in the service’s responsibilities and competence. Then comes the question: “What should a user do to enhance his enterprise’s business continuity and improve data accessibility for his DevOps team?” The answer is obvious – Bitbucket backup (even recommended by Atlassian itself).
Bitbucket Backup: Why It Is Vital
Third-party backup software, like GitProtect.io can drastically reduce the enterprise’s responsibilities. It will guarantee data recoverability and accessibility in any event of failure. Outages can happen due to different reasons, including human error, like it happened in April with Jira, software or hardware failures, or ransomware. So, what a comprehensive backup should include to keep the data constantly available and help you meet the shared responsibility model duties?
Backed up data: what to include?
For the most reliable protection, backup should cover not only repositories, but also all the metadata your Bitbucket account has. And under metadata we understand PR, wikis, issues, LFS, commits, tags, branches and so on and so forth. .. everything that makes your developers’ work meaningful.
Retention: Why it matters
Every piece of data needs to be valued and kept for further reference or analysis. Thus, long-term retention can not only help to recover the data from 5 or 10 years ago, but also it is a step forward to such security standards as SOC 2 or ISO 27K.
3-2-1 backup rule
Such a backup rule as 3-2-1 has already become a worldwide standard which explain how to secure data in the best way. According to this rule data is copied at least three times, and kept in two different places, one of which must be offsite. Thus, you get guarantee of data accessibility and recoverability.
Bitbucket restore and Disaster Recovery: How It Should Work
When it comes to backup, first what the enterprise should pay attention to is how fast it can recover the data to eliminate downtime and save the DevOps team’s time and the company’s budget. The second is what number of actions the organization should take to perform the recovery process fast – a step-by-step guidance in the event of any disaster scenario.
Thus, one should have different opportunities to restore his data depending on his needs, situation, and time frames. A chosen backup solution should give the customers an opportunity to recover their data immediately using any point-in-time restore to many destinations – the same or new organization account, to local device as a file, or cross-over to another git hosting platform (to GitHub or GitLab).
Also make sure that the Bitbucket backup software can ensure granular restore of repositories and only chosen metadata if you need to instantly restore only part of your data.
Thus, summing up all the mentioned backup and recovery features, you can create a complete answer to any disaster scenario.
Atlassian is down: recovery roadmap
For example, let’s imagine that Atlassian is down. You need to react fast to resolve this situation, right? In the ideal world, it is beneficial to have a few options to recover your Bitbucket environment immediately. For example, you can instantly recover repositories and metadata from the latest copy or some point in time to the local machine as a file. Another option is a recovery to the company’s Bitbucket local instance or a crossover recovery to a different git hosting service. Any of these options will help an organization to work continuously.
The enterprise’s infrastructure is down: actions to follow
Here it’s important to remember the 3-2-1 backup rule, which is based on the proportions: 3 copies, 2 storage instances, one of which is offsite. Though, the formula is not so strictly stick to this number. It’s a great advantage when an organization has an opportunity to keep their data on a number of storages. In this case if one backup copy or fails to recover, the enterprise has a few other in their pocket. Same, when it comes to infrastructure failure.
Third-party backup solution is down: the way to resolve it
It is a must for every enterprise to find out from their backup provider what emergency plan it has before they start any cooperation. Ideally, the backup service should provide its customer with the on-premise app installer. In this case, the company will manage to access its backup data and restore it.
Takeaway
Nowadays is the time when data threats appear almost every second. There can be outages provoked by human mistakes, software or hardware failures and many other unpleasant situations. Hence, it is important for every enterprise to be prepared for any Bitbucket outage to robust them fast and enhance the company’s business continuity. What is the best way to do it? Plan and build a comprehensive Bitbucket backup that will foresee any event of failure.
