How to manipulate sparse tensors?
You can manipulate sparse tensors in multiple ways using different methods. It is completely at you discretion what operation you want to do with your tensor. We will see different ways we can manipulate the sparse tensors in TensorFlow.
There is a package called tf.sparse package which consist of all the tools and methods that allow us to perform arithmetic manipulations, matrix multiplications, concatenation, and more. Let us discuss few of them.
Arithmetic Manipulations
Traditional arithmetic manipulations are not directly applicable on sparse tensors. Therefore, to add sparse tensors of same shape we use tf.sparse.add function. You can study the code give below. It shows two sparse tensors which are added and the result is stored in the result_sparse_tensor variable.
Python3
import tensorflow as tf # Create two sparse tensors sparse_tensor1 = tf.sparse.SparseTensor(indices = [[ 0 , 3 ], [ 2 , 4 ]], values = [ 10 , 20 ], dense_shape = [ 5 , 5 ]) sparse_tensor2 = tf.sparse.SparseTensor(indices = [[ 1 , 2 ], [ 4 , 2 ]], values = [ 30 , 40 ], dense_shape = [ 5 , 5 ]) # Add sparse tensors result_sparse_tensor = tf.sparse.add(sparse_tensor1, sparse_tensor2) # Print the result print (result_sparse_tensor) |
Output:
SparseTensor(indices=tf.Tensor(
[[0 3]
[1 2]
[2 4]
[4 2]], shape=(4, 2), dtype=int64), values=tf.Tensor([10 30 20 40], shape=(4,), dtype=int32), dense_shape=tf.Tensor([5 5], shape=(2,), dtype=int64))
Sparse-Dense Matrix Multiplication
If you want to multiply a sparse tensor and a dense tensor you can use tf.sparse.sparse_dense_matmul method as given below.
Python3
import tensorflow as tf # Define a sparse tensor sparse_indices = tf.constant([[ 0 , 2 ], [ 1 , 0 ], [ 1 , 2 ], [ 2 , 1 ]], dtype = tf.int64) sparse_values = tf.constant([ 1.0 , 2.0 , 3.0 , 4.0 ], dtype = tf.float32) sparse_shape = tf.constant([ 3 , 3 ], dtype = tf.int64) sparse_tensor = tf.sparse.SparseTensor(indices = sparse_indices, values = sparse_values, dense_shape = sparse_shape) # Define a dense matrix dense_matrix = tf.constant([[ 1.0 , 2.0 , 3.0 ], [ 4.0 , 5.0 , 6.0 ], [ 7.0 , 8.0 , 9.0 ]], dtype = tf.float32) # Perform sparse-dense matrix multiplication result_dense_matrix = tf.sparse.sparse_dense_matmul(sparse_tensor, dense_matrix) # Print the result print ( "Sparse Tensor:" ) print (sparse_tensor) print ( "\nDense Matrix:" ) print (dense_matrix) print ( "\nResult of Sparse-Dense Matrix Multiplication:" ) print (result_dense_matrix) |
Output:
Sparse Tensor:
SparseTensor(indices=tf.Tensor(
[[0 2]
[1 0]
[1 2]
[2 1]], shape=(4, 2), dtype=int64), values=tf.Tensor([1. 2. 3. 4.], shape=(4,), dtype=float32), dense_shape=tf.Tensor([3 3], shape=(2,), dtype=int64))
Dense Matrix:
tf.Tensor(
[[1. 2. 3.]
[4. 5. 6.]
[7. 8. 9.]], shape=(3, 3), dtype=float32)
Result of Sparse-Dense Matrix Multiplication:
tf.Tensor(
[[ 7. 8. 9.]
[23. 28. 33.]
[16. 20. 24.]], shape=(3, 3), dtype=float32)
Concatenation
In order to concatenate two sparse tensors, you can use tf.sparse.concat method. Check the example given below that concatenates two sparse tensors.
Python3
import tensorflow as tf # Create two sparse tensors sparse_indices1 = tf.constant([[ 0 , 1 ], [ 1 , 0 ], [ 2 , 2 ]], dtype = tf.int64) sparse_values1 = tf.constant([ 9 , 2 , 5 ], dtype = tf.int32) sparse_shape1 = tf.constant([ 3 , 3 ], dtype = tf.int64) sparse_tensor1 = tf.sparse.SparseTensor(indices = sparse_indices1, values = sparse_values1, dense_shape = sparse_shape1) sparse_indices2 = tf.constant([[ 3 , 1 ], [ 4 , 0 ]], dtype = tf.int64) sparse_values2 = tf.constant([ 6 , 8 ], dtype = tf.int32) sparse_shape2 = tf.constant([ 5 , 3 ], dtype = tf.int64) sparse_tensor2 = tf.sparse.SparseTensor(indices = sparse_indices2, values = sparse_values2, dense_shape = sparse_shape2) # Concatenate sparse tensors vertically result_sparse_tensor = tf.sparse.concat(axis = 0 , sp_inputs = [sparse_tensor1, sparse_tensor2]) # Print the result print ( "Sparse Tensor 1:" ) print (sparse_tensor1) print ( "\nSparse Tensor 2:" ) print (sparse_tensor2) print ( "\nResult of Concatenation:" ) print (result_sparse_tensor) |
Output:
Sparse Tensor 1:
SparseTensor(indices=tf.Tensor(
[[0 1]
[1 0]
[2 2]], shape=(3, 2), dtype=int64), values=tf.Tensor([9 2 5], shape=(3,), dtype=int32), dense_shape=tf.Tensor([3 3], shape=(2,), dtype=int64))
Sparse Tensor 2:
SparseTensor(indices=tf.Tensor(
[[3 1]
[4 0]], shape=(2, 2), dtype=int64), values=tf.Tensor([6 8], shape=(2,), dtype=int32), dense_shape=tf.Tensor([5 3], shape=(2,), dtype=int64))
Result of Concatenation:
SparseTensor(indices=tf.Tensor(
[[0 1]
[1 0]
[2 2]
[6 1]
[7 0]], shape=(5, 2), dtype=int64), values=tf.Tensor([9 2 5 6 8], shape=(5,), dtype=int32), dense_shape=tf.Tensor([8 3], shape=(2,), dtype=int64))
Slicing
It is very easy to slice a sparse tensor. You simply need to use tf.sparse.slice function like give below in the code.
Python3
import tensorflow as tf # Create a sparse tensor sparse_indices = tf.constant([[ 0 , 1 ], [ 1 , 0 ], [ 2 , 2 ], [ 2 , 3 ]], dtype = tf.int64) sparse_values = tf.constant([ 7 , 14 , 12 , 11 ], dtype = tf.int32) sparse_shape = tf.constant([ 3 , 4 ], dtype = tf.int64) sparse_tensor = tf.sparse.SparseTensor(indices = sparse_indices, values = sparse_values, dense_shape = sparse_shape) # Print the original sparse tensor print ( "Original Sparse Tensor:" ) print (sparse_tensor) # Slice the sparse tensor sliced_sparse_tensor = tf.sparse. slice (sparse_tensor, start = [ 1 , 0 ], size = [ 2 , 3 ]) # Print the sliced sparse tensor print ( "\nSliced Sparse Tensor:" ) print (sliced_sparse_tensor) |
Output:
Original Sparse Tensor:
SparseTensor(indices=tf.Tensor(
[[0 1]
[1 0]
[2 2]
[2 3]], shape=(4, 2), dtype=int64), values=tf.Tensor([ 7 14 12 11], shape=(4,), dtype=int32), dense_shape=tf.Tensor([3 4], shape=(2,), dtype=int64))
Sliced Sparse Tensor:
SparseTensor(indices=tf.Tensor(
[[0 0]
[1 2]], shape=(2, 2), dtype=int64), values=tf.Tensor([14 12], shape=(2,), dtype=int32), dense_shape=tf.Tensor([2 3], shape=(2,), dtype=int64))
Elementwise Operations
If you want to perform elementwise operations on nonzero values in sparse tensors (TensorFlow 2.4 and higher), you can use tf.sparse.map_values to do the same.
Python3
import tensorflow as tf # Create a sparse tensor sparse_indices = tf.constant([[ 0 , 1 ], [ 1 , 0 ], [ 2 , 2 ], [ 2 , 3 ]], dtype = tf.int64) sparse_values = tf.constant([ 21 , 11 , 13 , 8 ], dtype = tf.int32) sparse_shape = tf.constant([ 3 , 4 ], dtype = tf.int64) sparse_tensor = tf.sparse.SparseTensor(indices = sparse_indices, values = sparse_values, dense_shape = sparse_shape) # Print the original sparse tensor print ( "Original Sparse Tensor:" ) print (sparse_tensor) # Define an elementwise operation function def elementwise_operation(value): return value * 2 # Apply elementwise operation to nonzero values result_sparse_tensor = tf.sparse.map_values(elementwise_operation, sparse_tensor) # Print the result print ( "\nResult of Elementwise Operation:" ) print (result_sparse_tensor) |
Output:
Original Sparse Tensor:
SparseTensor(indices=tf.Tensor(
[[0 1]
[1 0]
[2 2]
[2 3]], shape=(4, 2), dtype=int64), values=tf.Tensor([21 11 13 8], shape=(4,), dtype=int32), dense_shape=tf.Tensor([3 4], shape=(2,), dtype=int64))
Result of Elementwise Operation:
SparseTensor(indices=tf.Tensor(
[[0 1]
[1 0]
[2 2]
[2 3]], shape=(4, 2), dtype=int64), values=tf.Tensor([42 22 26 16], shape=(4,), dtype=int32), dense_shape=tf.Tensor([3 4], shape=(2,), dtype=int64))
Sparse tensors in Tensorflow
Imagine you are working with a massive dataset which is represented by multi-dimensional arrays called tensors. In simple terms, tensors are the building blocks of mathematical operations on the data. However, sometimes, tensors can have majority of values as zero. Such a tensor with a lot of zero values is called as sparse tensor.
Sparse tensors are mostly encountered in the fields of computer vision and natural language processing. These can be pretty overwhelming at times. Therefore, in this article we will be discussing various aspect related to sparse tensors. You will have the following concepts cleared when you read this article:
Table of Content
- What are Sparse Tensors?
- How to create Sparse Tensors in TensorFlow?
- How to manipulate sparse tensors?
- Handling Sparse Tensors: Distinguishing Zero vs Missing Values