TIP: Click on subject to list as thread! ANSI
echo: os2rexx
to: All
from: Eddy Thilleman
date: 1999-09-22 16:36:26
subject: hstart wget?

Hoi!

I've it working. :) It turned out to be the result of a limitation of wget:
wget cannot 'timestamp' and 'not clobber' files at the same time.

I had timestamping = on in wget.cfg and used the -nc (no clobber) parameter at 
the same time.

I had for each wget a logfile specified, it surprised me that wget doesn't
create the logfile because of this error (so I didn't see this error because
the sessions were closed).

------------------- begin of wget.cfg -------------------
continue = on
convert_links = on
dirstruct = on
quota = 1m
tries = 5
follow_ftp = on
timestamping = on
-------------------- end of wget.cfg --------------------


--------------------- begin of wgetmir.cmd ---------------------
/* start wget mirroring each website/file listed in wget.mir */
arg InFile

if InFile = '' then
  InFile = 'wget.mir'

dir = directory( 'c:\www' )
'set wgetrc=c:\www\wget.cfg'

cmd1 = 'wget.exe -m -c -np --tries=1 -k -w1 '
cmd2 = ' -a wgetmir'
cmd3 = '.log'

signal on notready

LineNr = 0

LineIn( InFile, 1, 0 )
  /* No read; opens inputfile */
  /* (if file is already open, sets the read position to the first line). */

do while Lines( InFile ) > 0
  Line = LineIn( InFile )
  LineNr = LineNr + 1
  say LineNr': 'Line
  cmd = cmd1||Line||cmd2||LineNr||cmd3
  "@hstart /win /c /b "cmd">nul"
end

exit

NotReady:
  if LineNr = 0 then
    say 'could not open 'InFile

exit
--------------------- end of wgetmir.cmd ---------------------

For each line in wget.mir a separate wget-task is started, all running at the
same time. This way, next lines won't have to wait until files on previous
lines are complete.

I stopped using RxFTP because wget is much more capable: wget can pick up
where a previous download of a file was not complete, mirroring, etc.

  Greetings   -=Eddy=-        email: eddy.thilleman@net.hcc.nl

... "Windows NT Performance", on the next "In Search Of"
--- GoldED/2 3.0.1
* Origin: Windows98 is a graphic DOS extender (2:500/143.7)

SOURCE: echoes via The OS/2 BBS

Email questions or comments to sysop@ipingthereforeiam.com
All parts of this website painstakingly hand-crafted in the U.S.A.!
IPTIA BBS/MUD/Terminal/Game Server List, © 2025 IPTIA Consulting™.