The urge to record things and pass them along is inherent in our nature, and humans have followed this instinct for thousands of years. However, since the start of the 20th century, our ability to capture, store, and transfer data has expanded to the point where we now need sophisticated technology to manage it.
As big data continues to increase, we must find new ways to manage it at scale. It’s vital for business success to understand how much data exists, how we can expect it to increase in the future, and to identify ways to manage and optimize its value.
The Origins of Data Collection
The earliest account of gathering data to track and control business interests dates back to 7,000 years ago. Methods to store data ran the gamut of tablets, scrolls and later books, until 1932. This period saw the creation of the first magnetic drum, which remained a common data storage method until the 1960s. In 1947, the first random access memory (RAM) device was designed that recorded data on cathode ray tubes. This technology was followed by the magnetic tape drive in 1951, the first Hard Disk Drive (HDD) in 1956, the floppy disk in 1967, and the compact disc or CD in 1982. Data storage then ran the gamut from the Zip Drive in 1994 to the DVD in 1995, the SD Card, USB flash drive, and Blu-Ray optical discs by 2003. Finally, cloud storage was born in 2006.
How Much Is Data Growing?
The rapid evolution of data storage devices and media directly results from the increasing quantities of data humans collected over the years. Data quantities have grown exponentially since the beginning and continue to do so, but how much data is in the world currently, and how much is it likely to increase? Statistics tell us the following:
In 1997, Michael Lesk concluded that there “might be a few thousand petabytes of information, all told.” Since then, data generation has soared. By 2008, the world’s servers processed 9.57 zettabytes of information, according to a 2011 report by the McKinsey Global Institute.
Current projections from Statista show that, by the end of 2021, the world will generate 79 zettabytes of data, an increase of 15 zettabytes over 2020’s amount, and 34 zettabytes more than in 2019. Given that a zettabyte equals a trillion gigabytes, that’s a lot of data. Live internet statistics show:
5.15 billion people are mobile users
97.2% of organizations are currently investing in big data and artificial intelligence
30 billion-plus Internet of Things devices interact online daily
Internet users generate 2.5 quintillion bytes of data each day
More than 3.5 billion Google searches take place every day
Data creation is projected to increase to 181 zettabytes by 2025.
Data science is transforming industries like aviation, where the new generation of aircraft collect more than 30 times the amount of data the previous craft produced. Estimates show that by 2026, an aircraft could create between five and eight terabytes per flight—80 times more than older planes currently produce.
Across the board, data is changing the way businesses operate and the way in which the world functions. With better intelligence, companies can access performance and competitor benchmarks, create laser focused marketing campaigns, improve customer service, and leverage cost efficiencies.
Outside the business arena, data connects people and societies, provides new ways to measure and control activities such as healthcare, identification, security, and even home management. Data automates activities, enables new discoveries, and caters to perceived human needs in ways we could never have imagined a century ago.
Prime Data Generators
Just as some companies generate more revenue than others, when it comes to data, certain industries are more prolific than others. In addition to the amount of data produced by aviation, a few specific environments create significantly more data than others. These “prime generators” have higher needs for data storage, processing, and transfer capabilities, too. In many instances, the data produced in these environments is under-utilized, often because it’s already obsolete by the time it’s transported and becomes accessible for wider usage or distribution.
Before the Covid-19 pandemic, airline travel was one of the most dynamic industries in existence. According to the International Civil Aviation Organization’s (ICAO) report on The World of Air Transport in 2019, passengers carried on scheduled services rose to 4.5 billion, traveling on 38.9 million flights. Aircraft generate enormous quantities of data that must be captured and analyzed for efficient flight experiences. If each flight creates an average of 6.5 terabytes of data, that’s 252.8 million TB in a year.
To realize the full potential of aviation data, it needs to be processed, interpreted, and the findings implemented—a tall order when it takes hours or days just to upload it to the cloud or transfer it to a data center. BRYCK®’s data management solution makes it possible to download and process airline data at the edge in seconds, or to transport it quickly and cost-effectively to a data center.
Media and Entertainment
Big data plays multiple roles in the media and entertainment industry, such as delivering content via streaming services, predicting audience interest, and understanding engagement. From filmmaking to customizable news broadcasts and on-demand music services, the industry generates massive quantities of data each year and existing storage and transfer solutions are inadequate.
With BRYCK®, users can upload data rapidly from locations on set and transport it to a post-production facility for processing within a day. This method eliminates overheads, as well as costs associated with storage in the dailies management system and data transfer over a high-speed, dedicated network.
As the world explores ways to combat human error, vehicle manufacturers work hard to deliver the first reliable autonomous vehicle. The automotive companies collect massive amounts of data from prototype vehicles. Analysis of driving data aims to identify potential flaws in the design and ways to improve them ahead of mass acceptance and production. Even once these cars are in common use, however, each vehicle will continue to generate data on every single trip.
As we inch closer to this reality, it’s critical for scientists to access and process data rapidly and make timely adjustments. BRYCK® integrates directly with vehicle data acquisition units to capture data at the edge. The solution can then process the data in place immediately or transport it quickly to the data center for long-term management.
Data Centers and Cloud Storage Facilities
These facilities often provide scalable storage for large enterprises. The data is synchronized across several data centers for backup and disaster recovery purposes. Although most facilities use dedicated WAN networks for interconnection, the amount of data generated is higher than the bandwidth. This causes a lag in synchronization, which could result in data loss. Additionally, a dedicated WAN is expensive, with 100Gbps over shared backbone internet costing $1.2 million each year. BRYCK® can eliminate the cost of inter-connectivity and reduce synchronization time from days to hours.
Government and Defense Organizations
Federal agencies such as the Social Security Administration, the Federal Housing Authority, and the Food and Drug Administration use data to analyze issues such as disability claims, housing forecasts, and health risks of the U.S. population, among others. The Department of Defense makes use of data for risk analysis, understanding its supply chain network and devising possible defense scenarios. All this activity generates large amounts of data, which is not only stored and processed but is subject to the highest security standards.
Implementing the BRYCK® solution enables these organizations to capture and process the data in place and transport it rapidly using government couriers with secure clearance.
Enterprises in environments such as healthcare, aviation, and financial services regularly need to exchange data between data centers and locations such as branch offices. With the amount of data exceeding 100TB per week and growing, transfers are slow, and connectivity is expensive. Since 100TB can take up to 6 days to transfer over a 1Gbps connection, the delay in updating their data leaves organizations exposed to multiple risks. The BRYCK® solution enables fast transport between sites in less than one day and eliminates the cost of a dedicated high-speed network.
Implementing the Solution
The circumstances surrounding how much data exists in the world have raised concerns among business leaders and scientists for several years. Without viable ways to process and move large quantities of big data, humanity can’t reap the benefits of available information.
Tsecond’s BRYCK® is the solution to current and future challenges posed by big data, because it enables industries such as the prime generators listed above to capture, store, and transport data in a fraction of the time. This rugged, hand-held solution offers high-quality data capture at moving or remote edges, blazing fast data uploads, and analytics and machine learning to process data rapidly and immediately.
Secure, cost-effective, and fast physical transport of the device between two points reduces the risk of the data falling into the wrong hands, getting compromised, or incurring the high costs associated with ransomware or social engineering. It’s a new day and a new way. It’s time to explore BRYCK®.