Understanding Of Primary Terminologies
- Azure HDInsight: An azure service that provide managed clusters for big data processing and analytics.
- Hadoop: A Distributed open source processing framework helps in processing large datasets across the clusters using simple programming models.
- Apache Spark: It is an open source distributed computer system that used for providing an interface programming entire all clusters with implict data parallelism and fault tolerance.
- Apache Hive: It is built on top of Hadoop provides data summarization, query and analysis acting as a data ware house infrastructure.
- Cluster: A group of inter connection devices that performing the work as single unit.
- Blob Storage: A type of Azure storage service that is used of storing a large amount of unstructured data such as text or binary data.
- Data Lake Storage: A scalable and secure storage service from Azure provider that is used for big data analytic workloads.
Create and Configure Azure HDInsight
In our chapter about the amazing Poly Base thingy, we presented this super cool SQL Server 2024 feature to query CSV files stored in Azure Storage accounts. We mentioned that in PolyBase, hey, you can query data in Hadoop (HDInsight) using SQL Server. HDInsight is like, totally a very popular system in Azure that eventually you will, like, need to interact with if you use SQL Server. That is why we will, like, give an explanation for all the newbies out there about it, you know?