DPMM

The DPMM uses the Dirichlet process as a prior. Formally we can define DPMM as an extension of a finite mixture model that allows for an infinite number of components that uses the Dirichlet Process (DP) as a prior distribution for the mixture model, enabling the model to automatically determine the number of components or clusters needed to represent the data.

Let us now understand how we can solve the original problem at hand of learning the cluster assignments of a given data.

We are interested in finding the cluster assignments of our data group. For this, we use a technique called gibbs sampling. The Gibbs sampling process makes use of Dirichlet process as prior.

• Initialize: We start with a random initial assignment of data points to clusters and an initial set of parameters for each cluster.
• Iteration
  • We pick a data point. We fix the cluster assignments of all the other data point except the chosen point.
  • We now want to assign the chosen point a new cluster . This new cluster can be from existing cluster or a totally new cluster
  • The assignment of a point depends on the prior probability of the Dirichlet process multiplied by the likelihood of the data coming from that cluster
  • This can be mathematically expressed as
    • Probability of assignment to an existing cluster is
    • The probability of assignment to the new cluster is
    • Here we have assumed based distribution is normal with mean zero and and unit variance
    • Mathematically what we are doing for each data point we calculate the probability of the data belonging to that cluster(this is equivalent to prior probability) multiplied by the likelihood of the data point coming from that point (using our Gaussian normal distribution pdf)
  • We calculate above proabilites and assign the chooses data point to the cluster with the highest probabilites
• Repeat the above process till we reach convergence. that are no more changes in cluster assignment

Advantages over traditional methods

• One of the primary advantages of DPMMs is their ability to automatically determine the number of clusters in the data. Traditional methods often require the pre-specification of the number of clusters (e.g., in k-means), which can be challenging in real-world applications.
• DPMMs operate within a probabilistic framework, allowing for the quantification of uncertainty. Traditional methods often provide “hard” assignments of data points to clusters, while DPMMs give probabilistic cluster assignments, capturing the uncertainty inherent in the data
• DPMMs find applications in a wide range of fields, including natural language processing, computer vision, bioinformatics, and finance. Their flexibility makes them applicable to diverse datasets and problem domains.

Clustering is the process of grouping similar data points. The objective is to discover natural grouping within a dataset in such a way that data points within the same cluster are more similar to each other than they are to data points in another cluster. It’s unsupervised learning where we do not have a predefined target or label.

Key features of clustering are

• Similarity Measure: Clustering algorithms typically rely on a measure of similarity or dissimilarity between data points. Common measures include Euclidean distance, cosine similarity, or other distance metrics
• Grouping Criteria: Clusters are formed based on a grouping criterion that determines how data points should be combined. This criterion is often defined by the chosen clustering algorithm
• Unsupervised: The algorithm explores the data structure without prior knowledge of class labels or categories.