Tools or Extension For Scraping Headlines

Published on: 14-May 04:41am

Q: Hi all, Is there a tool/ chrome extension you would suggest to scrape top headlines from Search Engines; Youtube, Niche articles... based on a set of seed keywords?

 

 

 

 

 

 

 

 

 

 

A: Not exactly what you are asking for but pretty close. You add the links you want to scrape. I search google for keywords, use LinkClump to get URLs, add to sheet and that"s it.

 

This is a basic scraper in sheets I made a while ago.Not perfect but is simple.  Add URLs to the URL column, that"s it. Make sure to delete what is there first.  

Scrapes Title-Description-H1-Text on-page.  The bad thing is it scrapes all the text on the page usually, so has to be cleaned up.  Also sometimes throws errors, when that happens you usually just have to delete urls and add again. It will pull the title and only a part of the description from YT videos.

https://docs.google.com/spreadsheets/d/1fYb7iv_9kYiCvY_nKGBnusdXCQiONbzaWepWiT9hz2M/edit?usp=sharing 

 

A chrome extension I use all the time to get URLs off of google, YT, etc. is LinkClump.  Here is a video that shows what it does and how to set it up (not mine).

 

 

 

Another thing I do is search for other countries with my keywords in that language (i.e. dentista at google.it), take articles, and translate in sheets.  Can get some decent content that passes Copyscape.  (The Translate to English tab in the sheet.)

 

Again, not exactly what you want but maybe someone will find it useful.

 

Wanted to mention also that sometimes the errors are simply because something is not on the page.

Unable to find an answer?

Looking for anything specific article which resides in general queries? Just browse the various relevant folders and categories and then you will find the desired article.

Contact Us

Confirm Action

Are you sure? You want to perform this action.