Object Storage – What it is, and why consider the opportunity for your future infrastructure
Imagine you are building the next great mobile app that is so fun and addicting that it’s going to cause people to run into each other because they’re constantly looking down at their phone. Suddenly an A-List celebrity tweets about your app and your growth in users skyrockets. This is a good thing, but it also could result in a problem you may not have fully anticipated… how am I going to manage all of this data?
Having a lot of data early on in your exciting new application is both good and scary. It’s good because this data that your users are producing is very valuable, but scary because it can be tough to tell what the best solution to approach storing and being able to easily access this data for the users as well as the data-miners and researchers. For a large company like Facebook or Google to acquire your app for a hundred million dollars or so, they’re going to want a lot of user data, user data that they can quickly access, make sense of, and use to their advantage. Before your big payday you need to get the best storage infrastructure in place. First let’s take a look at traditional storage models.
First, we have file storage, which stores data as a single piece of information inside of a folder to help keep it organized along with other data. This is also referred to as hierarchical storage, and it imitates the way we store paper files. Whenever you need to find the data, your computer system just needs to know the path to find it.
Then we have block storage. This method breaks a file down into individual blocks of data and then stores the blocks as separate pieces of data. Storage systems can pull this off without having a file-folder structure since each block of data has a unique address. This structure allows a storage system to spread the smaller blocks of data wherever in the storage system it finds most efficient. Then, the storage system software pulls the necessary blocks back together to assemble the file whenever it is accessed.
While these two storage methods can certainly help you store and retrieve your data, there is a third and newly emerging method that can make your life much easier and run your data storage more efficiently.
Object Storage Defined
You want to save money, develop long-term retention, and use storage to gain insights on your users/customers? Well, then look no further than object storage. We’ll start our explanation with a simple analogy: parking your car.
Imagine for a moment that your car is a piece of unstructured data. As you run into the busy mall to do some last minute Christmas shopping you first need to ‘store’ or park your car in one of 3 different parking lots. Your options include the backlot block storage parking lot, the five story file storage parking garage, or your friendly object storage valet attendant. As you drive to the block storage backlot, you will need to keep your eyes peeled for the rigidly defined block or parking space that your car is designated to, this can be time consuming. Over in the file storage parking garage, your car will need to find which floor it belongs to which is determined by hierarchical levels, this is also time consuming and you have 6 more people on your Christmas shopping list. Your last option is to pull up to the front door of the mall with your car, hand the keys to the object storage valet, and get to shopping. With your object storage valet parking, there’s no need to waste time worrying about storage details because your valet attendant is actually very smart software that controls the whole lot.
How does Object Storage work?
It all starts with a name. With a name that is unique to the user or the application (e.g. ‘holiday-shopping-list-2017’) data is identified quickly through a highly optimized internal process designed for fast retrieval and reliability. If the physical location of an object is changed in the back-end, then it is simply updated in the internal lookup mechanism. The object’s name or ID (‘holiday-shopping-list-2017’) will remain the same, allowing users and applications to easily access it at any time.
In a storage system, the location of each item of data is identified and managed within a ‘namespace’. In the parking lot analogy, the namespaces for the block storage backlot, and file storage parking garage can become a limiting factor as the amount of data grows. Your friendly object storage valet attendant, on the other hand, is ideal for scalability major growth of unstructured data, because the namespace is virtually limitless.
The other major benefit is the fact that object storage lets smart software deal with any complexities, making it very easy and simple to use, manage, and maintain. With object storage you are also getting the use of multiple, simplified, lower-cost commodity storage devices of many different types. This is because the smart software links everything together into a single high-performing ‘cloud’ of storage. When it comes to your valet attendant looking after your car, you’re in good hands because each data object (car) is ‘mirrored’ with at least three copies in different geographic locations. Each object also has a ‘checksum’ value allowing any sort of data corruption to be easily detected, resulting in a new ‘good’ copy of the object to be generated to replace it. Someone saw some presents in your car and tried to break in? That’s okay, the entire car with the presents inside of it has been replaced for you.
Object Storage’s Secret Weapon: Metadata
What truly sets object storage apart as a solution is the use of ‘metadata’ or ‘data about data’. In our example the piece of data is already labeled with a unique ID (‘holiday-shopping-list-2017’), but much more than that, it is also tagged with information about the data in the object. For example, this shopping list would be stored with more information about items in the list, costs, sale prices, stores, names on the list, and so on. And this metadata about this object is fixed and stored with that object, it does not need to be held in a separate database, because if it was you would have quite a large mess on your hands.
Accessing this information is also very easy. According to Greg Rhoades, CTO with SIS “Newer generations of applications can address the storage themselves without having to traverse all the way through operating systems and/or file servers or complicated mechanisms for applications to gain access to that storage. They interact directly with the storage system through lightweight APIs that are typically presented as HTTP….the same protocol that runs the web…. Geographical distance doesn’t matter when it comes to object storage.”
Lastly, with object storage you are letting your data work for you and actually help you make informed business decisions. A powerful aspect of this solution is that the metadata can be customized and tailored to the needs of the storage users and applications. Meaning you can actually make sense out of the metadata where it can be analyzed to gain new insights into the preferences of the users/customers. With your hot new mobile app, you will be able to see trends happening for better or for worse through this metadata analysis and be able to pivot your business in the right direction to keep your users happy.