Deploy Hive metastore listeners outside Hive’s context to improve metastore reliability — We, the Expedia Groupᵀᴹ Data Platform Team, are building the next-gen petabyte-scale data lake. This next stage in the evolution of our data lake is based on our Apiary data lake pattern and utilizes a number of our open-source components like Waggle-Dance, Circus-Train, etc. A Hive metastore (HMS) proxied by…

Story of a 4 month long investigation into data loss issues with Hive — The short answer is no, but there could be other things wrong with your query! Read on to find out what they turned out to be in our case. As software engineers, it is quite usual for us to hit bugs in our applications when we run them in a…

How to run a Hadoop Application in an ECS Cluster — Recently, to create infrastructure for our new product, Shunting Yard, we, the Hotels.com Big Data Platform Team, had the opportunity to work on Terraform, Docker and AWS Elastic Container Service. The task was to amalgamate these three tools to automate infrastructure creation. …

A step by step guide to migrate Spring applications to Micrometer — In Hotels.com’s Big Data Platform team we have a few products which use Spring for the amazing capabilities it provides. We had been importing the spring-io-platform Platform BOM to enable Spring in our products until recently when we found that it’s nearing end of life in April, 2019. …