Replace Pioneer Home   All Examples   Free Download

 New request --free  RSS: Replace Pioneer Examples

1012.Batch download -- How to extract all links of a webpage periodically and save to file?

User: Paul Maguire -- 2012-10-06          << 1011  1013 >>
Hits: 2177
Type: Batch download   
Search all Batch download examples
Description:
Is it possible to set up an automated process to extract all links from the same website every day and then export the list of links to a text file. The text file should have the date in the title eg ExtractList20121001.Currently I use Batch Runner to extract and export the list. I am hoping to use Windows Scheduled Tasks to automate the daily process. Many thanks in advance.
Answer:
Hint: You need to Download and install "Replace Pioneer" on windows platform to finish following steps.
We can provide a workaround that may meet your request.
For testing purpose, we provide a sample to extract links in http://www.mind-pioneer.com, for every 10 seconds, and repeat for 6 times.
1. ctrl-h open 'replace' dialogue
2. click 'advanced' tab, set 'run following for each matched unit' to:

3. click "replace", the ExtractList2012XXXXXXXX.txt will be generated every 10 seconds.

Note: 
1. The file is generated under d:\, please change accordingly
2. The file is generated every 10 seconds and totally 6 times, please change to 86400 seconds and totally 100 days or so.
3. Please remove %H%M%S part in case you don't need hour/minutes/seconds as part of filename.
4. The GUI may looks like blocked, please ignore it.

Screenshot 1:  Replace_Advanced_Window


Similar Examples:
How to extract all links from a web page?   (70%)
How to extract links from a web page and add the page title at the end? (67%)
How to extract all the links and images from many web page files? (67%)
How to batch check all the links of a website and save to a file? (66%)
How to extract all lines with numbers from many text files? (65%)
How to extract all image links from a list of webpages? (63%)
How to extract all specific links from webpage? (62%)
How to extract all image links from a html file? (62%)

Check Demo of Batch download
Keywords:
testing  schedule  scheduled  task  640  210  minut  xxxxx  round  seconds  periodically download  extract from website periodically  remove day  download a file 100 times  save website  date extract from text file  text file of all links from website  extract date from text file