Python Scraper to Automatically Find Internal Link Opportunities
Internal links can make your website more user-friendly and have positive SEO benefits.
In this article I will share a script which will scrape your website and find internal linking opportunities for you based on keyword match.
First create a sheet based on following template: https://docs.google.com/spreadsheets/d/1xu3N9JlYPTo37J5uXdmRdP0u5cRAfHzeN_J205PJE0U/edit#gid=0
In the first tab enter your keywords and the URL which is ranking for that keywords.
In the second tab you can enter all urls of your blog or website that you would like to scrape for finding internal link opportunities.
Once you completed this, you can make a copy of following Colab to your own drive.
You can execute the fields pressing shift and the enter button.
What happens in the script is
- The sheet will be read (you need to give access to Google Sheets API by entering a code)
- The data in the sheet will be converted to dataframes and lists, so it can be used by the script.
- The script will scrape all urls and look if keywords are present on the page
- In case a certain keyword is present, the script will count the number of links that are present to the page the keyword needs to rank for.
- The script will write the internal link possibilities to a new worksheet “Internal Links” within your Google Spreadsheet
- You can start optimizing internal links.