Replace Pioneer Home   All Examples   Free Download

 New request --free  RSS: Replace Pioneer Examples

1311.Batch download -- How to extract all http links from multiple web sites in one time?

User: slimbo -- 2015-07-16          << 1310  1312 >>
Hits: 1444
Type: Batch download   
Search all Batch download examples
Description:
How to extract all http links from multiple web sites (DEPTH LEVEL 2) in one
time?
Input Sample:
www.site1.com
www.site2.com
Output Sample:
step 1 (Depth Level 1):
www.site1.com/1/index.html
www.site1.com/1/test.html
www.site2.com/1/index.html
www.site2.com/1/test.html

Step 2 final (Depth Level 2):
www.site1.com/2/url1.html
www.site1.com/3/urlx.html
www.site2.com/2/url1.html
www.site2.com/3/urlx.html
Answer:
Hint: You need to Download and install "Replace Pioneer" on windows platform to finish following steps.
1. open "Tools->Batch Runner" menu
2. click "Extract links" button
* set "file type filter" to the link type you want to extract
* check option of "Extract 1 level deeper", and set link type you want to extract deeper.
* enter the web page(up to 9) you want to extract, and click "extract" button.
3. all links will be in the list of "Batch Runner" window, you can use "Export List" to export to file.

If you want to extract more than 9 webpages, you need to do it multiple times.

Similar Examples:
How to extract all http links from multiple web sites in one time? (100%)
How to extract all image links from multiple html files?  (70%)
How to extract last 100 lines from multiple files? (67%)
How to extract first line from multiple files and generate a new file? (66%)
How to extract all links from a web page?   (66%)
How to extract all image links from a list of webpages? (63%)
How to extract all text enclosed by H1 tag from multiple html files? (63%)
How to extract all c++ comments from multiple cpp files? (63%)

Check Demo of Batch download
Keywords:
level deeper  level  file type filter  webpages  how to extract all http links  extract all http links  export  export to file  site  more than  download webpages from list  download multiple webpages  list of all webpages  extract multiple webpages  download webpages  download multiple  extract http links  extract links from web page file