Methods to get Pyspark Random Sample

• PySpark SQL Sample
  1. Using sample function
  2. Using sampleBy function
• PySpark RDD Sample
  1. Using sample function
  2. Using takeSample function

Are you in the field of job where you need to handle a lot of data on the daily basis? Then, you might have surely felt the need to extract a random sample from the data set. There are numerous ways to get rid of this problem. Don’t know all the ways? Continue reading the article further to know more about the random sample extraction in the Pyspark data set using Python.

Prerequisite

• Install Java
• Install Python
• Install Apache Pyspark

Note: In the article about installing Pyspark we have to install python instead of scala rest of the steps are the same.

Modules Required:

Pyspark: An open source, distributed computing framework and set of libraries for real-time, large-scale data processing API primarily developed for Apache Spark, is known as Pyspark.  This module can be installed through the following command in Python.

pip install pyspark

Student_data.csv file: