Apache Spark SQL
The SQL engine of Apache Spark: query files and tables in a data lake with SQL or DataFrames across a cluster. See how developers rate it month by month on DTDb.
- Category
- database engine
- Developer
- Apache Software Foundation
- Created by
- Michael Armbrust, Reynold Xin
- First release
- 2014
- License
- Apache-2.0
- Open source
- Yes
- Pricing
- Free
- Website
- https://spark.apache.org/sql/
- Written in
- Scala
- Data model
- Relational
- Storage layout
- Column-oriented
- Workload
- Analytics (OLAP)
- Analytics architecture
- Data lake
- Query language
- SQL, DataFrame API
- ACID transactions
- No
- Schema
- Strict
- Deployment
- Distributed cluster, Managed cloud
- Scaling
- Natively distributed
- Create, read, update, delete
- Create: fast (Parallel batch and streaming writes to files); Read: fast (Distributed scans over columnar files); Update: slow (Only through a table format such as Delta or Iceberg); Delete: slow (Only through a table format such as Delta or Iceberg)