Replace Pioneer Home   All Examples   Free Download

 New request --free  RSS: Replace Pioneer Examples

1012.Batch download -- How to extract all links of a webpage periodically and save to file?

User: Paul Maguire -- 2012-10-06          << 1011  1013 >>
Hits: 2177
Type: Batch download   
Search all Batch download examples
Is it possible to set up an automated process to extract all links from the same website every day and then export the list of links to a text file. The text file should have the date in the title eg ExtractList20121001.Currently I use Batch Runner to extract and export the list. I am hoping to use Windows Scheduled Tasks to automate the daily process. Many thanks in advance.
Hint: You need to Download and install "Replace Pioneer" on windows platform to finish following steps.
We can provide a workaround that may meet your request.
For testing purpose, we provide a sample to extract links in, for every 10 seconds, and repeat for 6 times.
1. ctrl-h open 'replace' dialogue
2. click 'advanced' tab, set 'run following for each matched unit' to:

3. click "replace", the ExtractList2012XXXXXXXX.txt will be generated every 10 seconds.

1. The file is generated under d:\, please change accordingly
2. The file is generated every 10 seconds and totally 6 times, please change to 86400 seconds and totally 100 days or so.
3. Please remove %H%M%S part in case you don't need hour/minutes/seconds as part of filename.
4. The GUI may looks like blocked, please ignore it.

Screenshot 1:  Replace_Advanced_Window

Similar Examples:
How to extract all links from a web page?   (70%)
How to extract links from a web page and add the page title at the end? (67%)
How to extract all the links and images from many web page files? (67%)
How to batch check all the links of a website and save to a file? (66%)
How to extract all lines with numbers from many text files? (65%)
How to extract all image links from a list of webpages? (63%)
How to extract all specific links from webpage? (62%)
How to extract all image links from a html file? (62%)

Check Demo of Batch download
testing  schedule  scheduled  task  640  210  minut  xxxxx  round  seconds  periodically download  extract from website periodically  remove day  download a file 100 times  save website  date extract from text file  text file of all links from website  extract date from text file