They become believers, and want to go running out to get as much data as fast as they can, or they become skeptics, because it sounds too hype-y or too much trouble for not enough value. It’s right to be excited about the opportunities associated with big data. There is high potential insight locked in emerging data sources like text, location and digital. Using more traditional data faster is also a great opportunity. But hospitality managers are also right to be cautious. There are many pitfalls in this big data world. Privacy concerns are getting a lot of press, harnessing big data requires investment in people and technology, and it’s really easy to get distracted by big data, going down rabbit holes, finding meaningless patterns, and making erroneous conclusions.
 
This is where the concept of responsible use of big data comes in. Starting with a disciplined approach to big data will ensure that you can get the insight without the distraction. Even if a vendor will be analyzing big data for you, you still need to ensure the data will add value to the solution. 

It’s tempting to think that you can just shove all of that new, giant data into one of those new “big data” databases and you’ll be good to go. Regardless of how inexpensive data storage has become, managing and analyzing data still requires people and technology support. This can get expensive quickly.  Do your due diligence to make sure the data has the potential to justify the investment.

Identifying the Business Problem

Once you define the potential business value, the second step is to understand the characteristics of the data source.  If you get to ask for the data, the better you define what you want the easier it will be for your partners to deliver it to you in a usable format.  Even if it just is what it is, ask the following five sets of questions:

1. Define the data: Where does it come from? How is it collected? What does it look like? Who is generating the data?  Can you count on it to be collected and transmitted at appropriate intervals? How big is it? How are the fields calculated? Is it closely related to any other data? The more familiar you become with the data, the better you will be able to answer the rest of these questions. 

2. Store the data: How will the data fit within your database architecture? Will you need to integrate it with other data? Will you need to link it to other data tables somehow? Will you need to set up permissions? Is there a hierarchy or other relationships that you need to define? Will you require specialized technology to store it properly?

3. Add additional data: How often is the data updated and how? Can your systems execute the ETL (extract, transform and load) process fast enough such that the data is updated at the speed of business?  Who is authorized to add or update data? How will changes to the data structure or contents be communicated and accounted for?

4. Analyze the data: What tools/algorithms will you need to analyze the data? Report on the data? How will analytics results be stored? How will reports be stored?  Do you need to define any commonly used, calculated fields? How much flexibility or structure is required?

Forming a cross-functional team within the organization can help to assess data needs, define key metrics, and brainstorm new opportunities. You will likely find that other departments could benefit from access to a new (or existing) data source, but they might require expanded information or additional parameters. Knowing this upfront saves work later. This group could also provide data governance – establishing a set of agreed upon rules for defining, accessing, and storing data across the organization.

How Much Data is Enough Data?

Sometimes adding more data to an analysis does not result in a better answer, i.e., adding additional data sources, as opposed to more of the same data. More of the same adds confidence and statistical rigor to an analysis.

Additional data might not result in a better answer if:

The data does not have a significant causal relationship with what you are trying to predict. If there are no predictive relationships, the data source adds overhead without adding insight.

The data does not change significantly over time. Adding it to an algorithm would not result in a different answer. For example, weather data might do a good job of explaining what happened to hotel demand in the past, but how do you incorporate weather forecast into a forecast for future dates? If it’s March now, and I’m setting prices for November, I only have the “average” forecast for that time period.
 
The data is extremely volatile or “dirty.”  If the data does not conform to some sort of pattern, it adds noise and uncertainty to the analysis. Regrets and denial data fall into this category for revenue management. Theoretically this information would help in demand unconstraining, but this type of data is very unstable. Collected by a call center agent, it is subject to user error. Collected via a website, it is subject to noise from shopping, inability to identify unique visitors, or inability to identify exactly what the user was focused on in a page with multiple products.

The points above are all also important considerations when it comes to deciding what data belongs in an algorithm versus a report or strategic view. 

If your system provider is collecting this data on your behalf, challenge them in the same way you would challenge yourselves internally. Make sure you understand all of the data inputs they will use and the inputs’ potential influence on the outputs. After all, you are the one who will ultimately have to implement and defend them.

These same principles apply to existing data, particularly as you strive to pull together an enterprisewide data management strategy.  Even with data you’ve already been storing and accessing, it is important to have clear definitions of what the fields mean, how metrics are derived from them, where the source data is and who is able add to or change the data. The cross-functional team I mentioned earlier is the one that takes the lead on decisions like this as well. This level of data governance will be crucial to an effective long-term data management strategy.