Tungsten execution engine
E705280
Tungsten execution engine is a low-level, memory- and CPU-optimized execution backend in Apache Spark designed to significantly improve performance of data processing workloads.
Statements (47)
| Predicate | Object |
|---|---|
| instanceOf |
Spark execution backend
ⓘ
query execution engine ⓘ |
| basedOn | project Tungsten NERFINISHED ⓘ |
| benefits |
better hardware utilization
ⓘ
higher throughput ⓘ lower latency ⓘ |
| componentOf | Spark SQL Catalyst optimizer pipeline NERFINISHED ⓘ |
| designedBy |
Apache Spark community
NERFINISHED
ⓘ
Databricks engineers ⓘ |
| developedFor |
Apache Spark Core
NERFINISHED
ⓘ
Apache Spark SQL NERFINISHED ⓘ |
| goal | improve performance of data processing workloads ⓘ |
| implements | whole-stage Java code generation for query plans ⓘ |
| improves |
CPU utilization
ⓘ
memory utilization ⓘ query execution speed ⓘ |
| introducedIn | Apache Spark 1.4 NERFINISHED ⓘ |
| language |
Java
NERFINISHED
ⓘ
Scala NERFINISHED ⓘ |
| manages | off-heap memory pages ⓘ |
| nameOrigin | named after metal tungsten ⓘ |
| nameReason | emphasizes performance and efficiency ⓘ |
| optimizationFocus |
CPU efficiency
ⓘ
cache locality ⓘ memory efficiency ⓘ |
| partOf | Apache Spark NERFINISHED ⓘ |
| reduces |
Java object allocation
ⓘ
garbage collection overhead ⓘ interpretive query execution overhead ⓘ |
| relatedTo |
Spark SQL Catalyst optimizer
NERFINISHED
ⓘ
Spark physical execution layer ⓘ |
| replaces | row-based interpreted execution in Spark SQL ⓘ |
| runsOn | Java Virtual Machine NERFINISHED ⓘ |
| storesDataAs | compact binary rows ⓘ |
| supports |
DataFrame API
ⓘ
Dataset API NERFINISHED ⓘ Spark SQL queries ⓘ |
| targetEnvironment | distributed data processing ⓘ |
| targetWorkloads |
ETL pipelines
ⓘ
batch processing ⓘ interactive SQL queries ⓘ |
| uses |
binary row format
ⓘ
cache-friendly data structures ⓘ expression code generation ⓘ off-heap memory management ⓘ runtime code generation ⓘ whole-stage code generation ⓘ |
Referenced by (1)
Full triples — surface form annotated when it differs from this entity's canonical label.