Hotel room rates: human work or algorithmic plaything?
You would like to book a hotel room and browse the internet for which rooms and rates are on offer. The rates provided depend on forecasted demand and come about through the use of computer algorithms. However, the rates are often manually adjusted by hotel personnel. What are the consequences and how can those consequences be measured? PhD defence on 12 November 2019.
According to PhD candidate Larissa Koupriouchina, the field of hotel revenue management – the 'art and science' of forecasting demand while simultaneously adjusting price and availability of inventory to meet demand – has developed rapidly over the past decade. Nowadays, automated systems forecast future demand and revenue managers have to decide whether to agree with these estimates. 'As increasing numbers of algorithms are currently used to support human decisions in a variety of fields, an even greater need to successfully combine computer output with human knowledge and intuition arises. Can humans improve on decisions suggested by these data-intensive and sophisticated algorithms?'
The subject of Koupriouchina's research stems from her passion for technology. 'After receiving my master's degree, I gravitated towards any project related to technology. I found a job in hospitality management higher education and started searching for interesting topics to challenge myself further. By following what revenue management professionals were discussing online and analysing the literature, as well as all the available content of specialized discussion groups on LinkedIn, I discovered that forecasting was a highly discussed issue.'
Algorithms are used for this kind of forecasting. 'Every day we see examples of algorithms around us, like potential partners suggested by a dating site, and self-driving cars. All these traditionally human areas have been 'invaded' by the growing power of computer algorithms. They come with fancy names and high price tags, but should we blindly follow them? How do we know if they are right? And how can we evaluate the influence of our own interventions?'
Contradictory answers
Koupriouchina researched anonymized hotel forecasting data obtained from thousands of hotels worldwide through a collaboration with a global hotel Revenue Management System (RMS) provider with more than 10 thousands clients in 124 countries. 'I studied detailed forecasting data and evaluated whether human interventions improved these forecasts. Various statistical techniques to analyse the data were used, including multilevel regression analysis, also referred to in the literature as hierarchical linear modelling, mixed linear modelling and growth-curve modelling.'
Firstly, Koupriouchina examined the accuracy of forecasting accuracy measures. With seventeen different measures, she calculated the accuracy of more than 2000 automated forecasts, which were to be compared to judgmental adjustments introduced by the revenue managers. 'Different error measures generate contradictory answers and the forecasting accuracy can be misjudged and, as a consequence, potentially undermine decision-making in other important hotel management areas, such as pricing, inventory control, operational planning, distribution, and strategy. The findings can be used to further educate revenue managers about the pitfalls and biases of each accuracy measure, so that they are in a position to carefully select the forecasting accuracy measures applicable for their conditions.'
Furthermore, the researcher demonstrates that the accuracy of forecasts improves considerably when forecast horizons are relatively small, that frequent manual adjustments are more favourable for the accuracy of forecasts for group bookings than for individual bookings, and that manual adjustments at a late stage have a more favourable effect on accuracy than adjustments at an early stage.
Recommendations
Given the importance of forecasting in the hotel revenue management optimization cycle, a more fruitful approach may be to expand the common set of narrow accuracy measures approach with a more structured, comprehensive and consistent framework of forecasting quality assessment. One of the important elements of this framework could be the collaborative development and implementation of a set of automated or semi-automated forecast quality monitoring procedures, including feedback mechanisms that allow users to learn from their past decisions and actions. To make these feedback mechanisms meaningful, additional efforts will be required from the hotels and their revenue managers. For example, it would be extremely helpful if revenue managers kept a consistent log in RMS of the reasons for overrides, especially for large and frequent overrides, making it possible to systematically assess these reasons and incorporate the outcomes of this analysis in the feedback loop.
RMS providers, on the other hand, could add automated procedures to continuously monitor judgmental adjustments introduced by the revenue managers, and analyse and report on various important aspects such as size, direction, frequency, timing, segments to which they are applied, type of override, and so forth. This analysis could include measurements of override effectiveness and the impact on forecasting performance, categorized by type of override and by reason. Moreover, by leveraging the immense number of exchanges with the thousands of hotels operating under different conditions, RMS software providers could further shape their software with a deeper understanding of the users' behaviour.
The findings of this study have a number of implications for academia, the hotel industry, and RMS software providers. The findings have been published in the International Journal of Hospitality Management, International Journal of Contemporary Hospitality Management, and as a book chapter in the textbook Management Science in Hospitality and Tourism: Theory, Practice, and Applications. Several academic authors have already incorporated the recommendations in their research. To attract the attention of the hotel industry, as well as to illustrate the need to incorporate these findings in practice, the intermediate results have been shared at various educational sessions, conferences and meetings of Revenue Management associations in the Netherlands and abroad (USA, UK, Germany, France, Croatia, China, Russia, etc.). Moreover, a pilot online training course for hotel managers was created and the research outcomes will be further incorporated in the Revenue Management course taught at Hotelschool The Hague, which prepares hotel managers for the hospitality industry worldwide.
Dissertation: Human-computer interaction between revenue management professionals and forecasting support systems (NWO 023.002.090)
Supervisor Professor J.I. van der Rest on the research of Larissa Koupriouchina:
'Our society increasingly relies on technologies based on deep learning, machine learning and artificial intelligence. Larissa’s PhD work deals with the interesting and important topic of the role human decision making plays in a business field that is increasingly supported by technology. What sets her research apart is that she does not focus on the improvements that these technologies can bring, but rather looks into the results of human interventions with complex algorithms to evaluate whether humans can still add value and improve the final decisions. This research contributes to a very under-researched area and potentially opens up a new research agenda.
During her PhD studies Larissa substantially developed her research skills and did a marvellous job in dealing with the numerous challenges related to her undertaking. For example, very little had been published on how revenue managers use their revenue management systems. In addition, the extensive real-life research data comprised complex structures and hundreds of different, but related, variables. It was paramount for her to understand how the Revenue Management System worked and to apply this knowledge in the difficult task of cleaning up the data and preparing it for analysis. Larissa was also not afraid to explore various analytical approaches and did this with the highest scientific rigor. She showed creativity and determination in solving the numerous challenges that arose during the project. Overall, in my view this dissertation has a high practical value and potential scientific significance.'
Text: Floris van den Driesche
Contact