What is Sparksession?
When spark runs, spark Driver creates a SparkSession which is an entry point to start programming with RDD, DataFrames, and Dataset to connect with Spark Cluster. The Sparksession was introduced in the Spark 2.0 version. It provides an interface for working with structured data processing. Before SparkSession, SparkContext used to be the entry point to run Spark. We must know that SparkSession doesn’t completely replace SparkContext, because SparkSession creates SparkConfig, SparkContext. The APIs that we used earlier in SparkContext like SQLContext, and HiveContext were now used with SparkSession.
SparkSession includes the following APIs:
- SparkContext
- StreamingContext
- SQLContext
- HiveContext
How to Import SparkSession in Scala?
This article focuses on discussing how to import SparkSession in Scala.
Table of Content
- What is Sparksession?
- Prerequisites
- Approach to Import SparkSession in Scala
- Implementation
- Create a DataFrame Using SparkSession
- Conclusion