Big Data for Small Business
Posted by Timothy Platt on Jul 8, 2017
Big Data for Small Business
By now you’ve probably heard about “Big Data” and the benefits companies are deriving. Big businesses such as Amazon, Google, Microsoft, and others gather key operational and business insights by being able to quickly and efficiently filter through the mountains of data their customers generate every single day (or hour!). Trends can be identified and turned into actionable business information in record time. Historically, Big Data has been difficult to process for the small business. Why? Because the expertise and technical infrastructure required upfront made for a daunting hurdle. The initial upfront investment – in terms of both personnel and infrastructure – was simply too large.
Cloud services, such as Microsoft’s Azure Cloud Computing Platform have become the great equalizer. It’s no longer necessary to make large capital expense (capex) investments for mass computing and storage power. Cloud provides the power, in an easy to engage operating expense (opex) fashion. And further, cloud services reduce the technical complexity and offer a surprisingly easy “as a Service” experience – minimizing the technical skill sets needed.
For a complete explanation of the benefits of cloud computing, please see here
Big Data – What is it?
Firstly, let’s define Big Data. Big Data was originally defined by the 3 V’s – Volume, Velocity, and Variety. Let’s talk Volume first. This is simple – big data is big. This simply means there’s a lot of it. Millions of rows or records consuming gigabytes, terabytes, or petabytes of storage space. The sheer volume of this data challenged traditional approaches to data warehousing – there was simply too much of it, and too much expense to be able to deal with it in the timeframe needed. And that leads us to the next V – Velocity. The data is generated and consumed at a rapid pace – outpacing what could be done with traditional technology. Lastly – Variety – the masses of data available now can exist in a form not reminiscent of traditional database rows and structures. Just think about the pervasive influence of digital in our lives – text updates on social media, photos, images, etc.
To be complete – 2 V’s have been added since the formulation of the original definition – Value and Veracity. If we’ve got a lot of data to deal with – it needs to be valuable and accurate. Otherwise all this big data processing is simply a wasted expense.
Examples of Big Data
How about some practical examples? Here’s a few.
- Customer sentiment – A company can extract “sentiment” from product review comments, Twitter, Facebook, or other social media updates. This gives the business an agile way to respond to rapidly changing situations – and to get ahead of customer service issues or concerns. This is done from a combination of Big Data infrastructure and Natural Language Processing (NLP) – which is the ability for a program to analyze human written text and figure out if it is positive or negative feedback.
- Internet of Things (IoT) – IoT enabled devices can generate mountains of data – and Big Data processing gives the ability to handle that data – and gather the insights from it.
Do these situations apply to your business? Maybe they do, or maybe they don’t. But another way to think of Big Data capabilities is that they give you the ability to perform analysis you couldn’t otherwise – due to the Volume, Variety, or Velocity of the incoming data. Have you ever had a data source, but no practical way to process it? Big Data processing power may be the solution to your problem.
Big Data Processing in Microsoft Azure
Earlier, we called cloud computing the great equalizer. And we’ll now go into detail using Microsoft Azure as a specific example.
Microsoft’s Azure offers an impressive breadth of Big Data capability – delivered in an end-to-end architecture.
- Azure Storage – Because big data is big – we need a cost effective way to store it – and Azure delivers that in multiple ways – but the first is using Azure Storage. Azure Storage is cloud storage – affordable and infinitely scalable. It’s cheap enough to hold that big data, in its entirety. It’s also a “first class citizen” when it comes time to load that data into the other Azure services.
- Azure Data Factory – How do we get our big data into the storage? Azure provides Data Factory specifically for that purpose. Think of this component as the Extract-Transform-Load (ETL) component of our solution – it’s on-ramp for the data – no matter where it’s coming from.
- HDInsight – HDInsight is Microsoft’s PaaS implementation of Hadoop – the de-facto standard for parallel processing of Big Data. By providing a parallelized computing capability, coupled with distributed storage (HDFS delivered via Azure Storage) you can process all that big data in parallel. This is a key capability necessary for big data. It’s simply too slow to run the data through a single node. For more details on Hadoop, see this article
- Data Lake Store – Data Lake Store is an HDFS storage option with a lot of bells and whistles – and still infinitely scalable and compatible with the rest of the tech stack. It also provides for integrated security via Azure Active Direction (AAD).
- Data Lake Analytics – Microsoft’s made great strides in offering a complete end-to-end experience – from loading the data, to storing it, to analyzing it efficiently. And Azure Data Lake Analytics provides the ability to interactively query and work with the data – without the hassle (or ongoing expense) of managing a Hadoop cluster.
Lastly, Azure offers all the traditional relational database and business intelligence (BI) capabilities you would expect via Azure SQL Database and Azure SQL Data Warehouse.
NOTE: We didn’t discuss them here, but Amazon Web Services (AWS) and Google Cloud Platform (GCP) offer similar functions, using slightly different specific technologies. But the core set of capabilities these alternatives deliver is the same. We like Azure because it’s got an impressive breadth of services and is easy for any business to weave into their infrastructure and process.
How to get started with Microsoft Azure
Is your head spinning yet? That’s OK, we’re here to help. We’re passionate and dedicated professionals that work on these technologies all day, every day. We’ve implemented all of this in various client situations – for companies of ALL sizes. We’ve seen the benefits and we know how to avoid the drawbacks. Please reach out if we can be of help to your company.
The Orlando IT Company that Cares About Your Small Business
Virtual Operations is the Orlando IT company that cares about your small business. The computer support partner you choose can make all the difference. Our Managed IT Service offering provides the expertise and proactive care required to ensure your technology works for you. We are Orlando’s top small business tech support, and that is the advantage that Virtual Operations provides.