Rakuten frees itself of Hadoop investment in two years

Based in San Mateo, California, Rakuten Rewards is a shopping rewards company that makes money through affiliate marketing links across the web. In return, members earn reward points every time they make a purchase through a partner retailer and get cash back rewards.

Naturally this drives a lot of user insight data – hundreds of terabytes on active recall with more in cold storage, to be exact.

Also on InfoWorld: Snowflake review: A data warehouse made better in the cloud ]

In 2018 the business started to get serious about giving more users access to this insight – without having Python or Scala coding chops – while also reducing its capital expenditure on hardware, and started looking to the cloud.

‘SQL server machines don’t scale elegantly’

Formerly known as Ebates, the business was acquired in 2014 by the Japanese e-commerce giant Rakuten, and has been growing fast since, forcing a drive to modernize its technology stack and become more data-driven in the way it attracts and retains customers.

This starts with the architecture. In the past three years Rakuten Rewards has moved its big data estate from largely on-prem SQL to on-prem Hadoop to, today, a cloud data warehouse courtesy of Snowflake.

“SQL server machines don’t scale elegantly, so we went on-premises Hadoop with Cloudera, using Spark and Python to run ETL, and got some performance out of that,” VP for analytics at Rakuten Rewards, Mark Stange-Tregear, told InfoWorld.

Copyright © 2020 IDG Communications, Inc.

Source link