Overview
1 Overview
As a powerful computing middleware, Linkis can easily interface with different computing engines. By shielding the usage details of different computing engines, Linkis provides a set of unified user interface upwards. The operation and maintenance cost of deploying and applying Linkis' big data platform is greatly reduced. At present, Linkis has connected several mainstream computing engines, which basically cover the data requirements of production. In order to provide better scalability, Linkis also provides relevant interfaces for accessing new engines, which can be used to access new computing engines.
The engine is a component that provides users with data processing and analysis capabilities. Currently, it has been connected to the Linkis engine, including mainstream big data computing engines such as Spark, Hive, and Presto, as well as engines with scripting capabilities such as python and Shell. DataSphereStudio is a one-stop data operation platform connected to Linkis. Users can easily use the engine supported by Linkis in DataSphereStudio to complete interactive data analysis tasks and workflow tasks.
Supported engines and version information are as follows:
| Engine | Default Engine | Default Version |
|---|---|---|
| Spark | Yes | 3.2.1 |
| Hive | yes | 3.1.3 |
| Python | yes | python2 |
| Shell | Yes | 1 |
| JDBC | No | 4 |
| Flink | No | 1.12.2 |
| openLooKeng | No | 1.5.0 |
| Pipeline | No | 1 |
| Presto | No | 0.234 |
| Sqoop | No | 1.4.6 |
| Elasticsearch | No | 7.6.2 |