Hope springs everlasting within the database enterprise. Whereas we’re nonetheless listening to about knowledge warehouses (quick evaluation databases, sometimes that includes in-memory columnar storage) and instruments that enhance the ETL step (extract, rework, and cargo), we’re additionally listening to about enhancements in knowledge lakes (which retailer knowledge in its native format) and knowledge federation (on-demand knowledge integration of heterogeneous knowledge shops).
Presto retains developing as a quick technique to carry out SQL queries on huge knowledge that resides in knowledge lake information. Presto is an open supply distributed SQL question engine for operating interactive analytic queries in opposition to knowledge sources of all sizes. Presto permits querying knowledge the place it lives, together with Hive, Cassandra, relational databases, and proprietary knowledge shops. A single Presto question can mix knowledge from a number of sources. Fb makes use of Presto for interactive queries in opposition to a number of inside knowledge shops, together with their 300PB knowledge warehouse.
The Presto Basis is the group that oversees the event of the Presto open supply undertaking. Fb, Uber, Twitter, and Alibaba based the Presto Basis. Extra members now embody Alluxio, Ahana, Upsolver, and Intel.
Ahana Cloud for Presto, the topic of this assessment, is a managed service that simplifies Presto for the cloud. As we’ll see, Ahana Cloud for Presto runs on Amazon, has a reasonably easy consumer interface, and has end-to-end cluster lifecycle administration. It runs in Kubernetes and is very scalable. It has a built-in catalog and straightforward integration with knowledge sources, catalogs, and dashboarding instruments.
Opponents to Ahana Cloud for Presto embody Databricks Delta Lake, Qubole, and BlazingSQL. I’ll draw comparisons on the finish of the article.
Presto and Ahana structure
Presto is not a general-purpose relational database. Relatively, it’s a instrument designed to effectively question huge quantities of information utilizing distributed SQL queries. Whereas it could exchange instruments that question HDFS utilizing pipelines of MapReduce jobs corresponding to Hive or Pig, Presto has been prolonged to function over completely different sorts of information sources together with conventional relational databases and different knowledge sources corresponding to Cassandra.
Briefly, Presto just isn’t designed for on-line transaction processing (OLTP), however for on-line analytical processing (OLAP) together with knowledge evaluation, aggregating massive quantities of information, and producing stories. It may well question all kinds of information sources, from information to databases, and return outcomes to plenty of BI and evaluation environments.
Presto is an open source project that operated under the auspices of Facebook. It was invented at Facebook and the project continues to be developed by both Facebook internal developers and a number of third-party developers under the supervision of the Presto Foundation.
Presto’s scalable, clustered architecture uses a coordinator for SQL parsing, planning, and scheduling, and a number of worker nodes for query execution. Result sets from the workers flow back to the client through the coordinator.
Ahana Cloud packages managed Presto, a Hive metadata catalog, a data lake hosted on Amazon S3, cluster management, and access to Amazon databases into what is effectively a cloud data warehouse in an open, disaggregated stack, as shown in the architecture diagram below. The Presto Hive connector manages access to ORC, Parquet, CSV, and other data files.
The Ahana control plane takes care of cluster orchestration, logging, security and access control, billing, and support. The Presto clusters and the storage live inside the customer’s VPC.
Using Ahana Cloud for Presto
Ahana provided me with a hands-on lab that allowed me to create a cluster, connect it to sources in Amazon S3 and Amazon RDS MySQL, and exercise Presto using SQL from Apache Superset. Superset is a modern data exploration and visualization platform. I didn’t really exercise the visualization portion of Superset, as the point of the exercise was to look at SQL performance using Presto.
I found the process of adding data sources a bit annoying because it required me to edit URI strings and JSON configuration strings. It would have been easier if the strings had been assembled from pieces in separate text boxes, especially if the text boxes were populated automatically.
How to integrate Ahana Presto with machine learning and deep learning
How do people integrate Ahana Presto with machine learning and deep learning? Typically, rather than using Superset as a client, they use a notebook, either Jupyter or Zeppelin. To perform the SQL query, they use a JDBC link to the Ahana Presto query engine. Then the output from the SQL query populates the appropriate structure or data frame for use in machine learning, depending on the framework used.
New features of Ahana Cloud for Presto
The version of Ahana Cloud I tested incorporated the improvements announced on March 24, 2021. These included performance improvements such as data lake I/O caching and tuned query optimization, and ease of use improvements such as automated and versioned upgrades of Ahana Compute Plane.
I didn’t use all of them myself. For example, I didn’t enable data lake I/O caching because the data lake table I was using was too small, and I didn’t spend long enough with Ahana to see a version upgrade.
Ahana Cloud for Presto vs. competitors
Overall, Ahana Cloud for Presto is a good way to turn a data lake on Amazon S3 into what is effectively a data warehouse, without moving any data. Using Ahana Cloud avoids most of the work required to set up and tune Presto and Apache Superset. SQL queries run quickly on Ahana Cloud for Presto, even when they are joining multiple heterogeneous data sources.
Databricks Delta Lake uses different technologies to accomplish some of the same things as Ahana Cloud for Presto. All the files in Databricks Delta Lake are in Apache Parquet format, and Delta Lake uses Apache Spark for SQL queries. Like Ahana Cloud for Presto, Databricks Delta Lake can speed up SQL queries with an integrated cache. Delta Lake can’t perform federated queries, however.
Qubole, a cloud-native data platform for analytics and machine learning, helps you to ingest datasets from a data lake, build schemas with Hive, query the data with Hive, Presto, Quantum, and/or Spark, and continue to your data engineering and data science. You can use Zeppelin or Jupyter notebooks, and Airflow workflows. In addition, Qubole helps you manage your cloud spending in a platform-independent way. Unlike Ahana, Qubole can run on AWS, Microsoft Azure, Google Cloud Platform, and Oracle Cloud.
BlazingSQL is an even faster way of running SQL queries, using Nvidia GPUs and running SQL on data loaded into GPU memory. BlazingSQL lets you ETL raw data directly into GPU memory as GPU DataFrames. Once you have GPU DataFrames in GPU memory, you can use RAPIDS cuML for machine learning, or convert the DataFrames to DLPack or NVTabular for in-GPU deep learning with PyTorch or TensorFlow.
Ahana Cloud for Presto is a worthwhile alternative to its competitors, and is easier to set up and maintain than an open source Presto deployment. It’s certainly worth the effort of a free trial.
Cost: $0.25/Ahana Cloud Credit (ACC) hour. See pricing calculator and table of instance charges. Example: Presto Cluster of 10 x r5.xlarge running every workday costs $256/month.
Platform: Runs on Amazon Elastic Kubernetes Service.
Copyright © 2021 IDG Communications, Inc.