Printing the price

We are almost done. We only need to print the price, as we have stored the price in the file, we need to store it in the variable to print it according to our needs. We will use a while loop until the file reaches the end of the file and extract the only line into a variable. Finally, we use the echo command to print the price. 

while read price
  do 
      val=$price
  done <$output
  echo "The price of $coin is = $val" 

For a layer of readable code, we’ll create functions for each task, i.e. for striping or scraping the webpage and for printing the price. 

function strip_html(){
  grep -oP '(?<=<span class="no-wrap" data-price-btc).*?(?=</span>)' $output >temp.txt 
  sed -i 's/[^>]*>//g' temp.txt >$output
  sed -ni '1p' temp.txt >$output
  cp temp.txt $output
  rm temp.txt
}
function print(){
  while read price
  do 
      val=$price
  done <$output
  echo "The price of $coin is = $val" 
}

Below is the complete implementation.

Making several modifications, such as clearing the output of cURL to /dev/null, will flush the output and make it loop a bit cleaner. If it shows any error, please remove those commands. We also took the user input and stored the value in the coin variable.

#!/bin/bash

function strip_html(){
  grep -oP '(?<=<span class="no-wrap" data-price-btc).*?(?=</span>)' $output >temp.txt 
  
  sed -i 's/[^>]*>//g' temp.txt >$output
  sed -ni '1p' temp.txt >$output
  cp temp.txt $output
  rm temp.txt
}
function print(){
  while read price
  do  
      val=$price
  done <$output
  echo "The price of $coin is = $val" 
}

read -p "enter the coin code : " coin
url='https://www.coingecko.com/en/coins/'$coin''
output=price.txt
touch $output temp.txt
curl -o $output $url 
strip_html  
print



Shell Script to Scrape prices of Cryptocurrency coins in Linux

Cryptocurrency is such a hype that everyone wants to be a part of it. Even nerds and programmers want to dive into this amazing field because it is quite versatile and interesting in every aspect.  So how about scraping the price of cryptocurrency coins such as bitcoin, ethereum, dogecoin, and a ton of other coins using shell script and coingecko.com

There are tons of websites that show the pieces of various Cryptocurrencies but many of them have bots and security systems to avoid recursively accessing a website. So the safest option here is to use coingecko which is a great and massive platform for analyzing cryptocurrency and other aspects as well.

Similar Reads

Inspecting the site

This is the most crucial part of web scraping to inspect and analyze various aspects of the website. This allows us to get familiar with the structure and components of the website. If you are using chrome, developers have a great option of Dev tools, such as inspecting the webpage and see the tags and elements associated with it....

Finding the target element or tag

Now move on to the actual site’s content and find our target, which in this case is the current price of the cryptocurrency. We can use the “Select the element from the page to inspect” option on the top left side of the inspect/ developer window. This allows us to see the elements/tags by clicking or hovering over them. This will make sure we are selecting the correct tag from the source of the webpage. Also, it becomes quite easy to find our target element as we can visually see the element....

Using cURL to access the website and store it in a file

Now we move on to the actual scrapping and accessing the website page from the terminal. cURL command is a great option for this as it is available by default in many Linux/Unix systems. We also have an alternative cURL called wget, but is not widely available in many systems. We can access a website URL using the following command....

Scraping the webpage file using grep and sed

We now have the source of the webpage from which we need the price of the cryptocurrency coins. We will start by using grep to tick out the span tag with the class no wrap and data-price-BTC, which’s value changes every time, so we will check it here. We won’t be able to scrap the content if we hard code the value, it may change every second, so just keep it till there....

Printing the price

We are almost done. We only need to print the price, as we have stored the price in the file, we need to store it in the variable to print it according to our needs. We will use a while loop until the file reaches the end of the file and extract the only line into a variable. Finally, we use the echo command to print the price....