Hello All,

I've revised my wgetmir.cmd REXX file again.

both underlined comparisons  if Line = ''  and  if strip( Line ) = ''
                             ------------       ---------------------

always return 0 (false), so the previous script always failed.

but  if strip( Length( Line ) ) > 0  (or without strip()) works well.
     ------------------------------

I've made some more modifications, if the text file with the listed url's
couldn't be opened, an errormessage is written in the file with the same name
as the file with the url's with '.error' appended, this file is only created
if the inputfile couldn't be opened. So if the inputfile is 'this.site' then
the accompanying errorfile is 'this.site.error' (without the quotes ofcourse). 
For each line in the inputfile a separate session with wget is started and
with its own logfile: the filename of the logfile consists of the filename of
the inputfile, a dot, number (counter = linenumber without counting empty
lines) and '.log'. So for each line in 'this.site' the logfiles are
'this.site.1.log', 'this.site.2.log', etc.

so:
inputfile      logfile           errorfile
-----------    ---------------   ---------------
wget.mir       wget.mir.1.log    wget.mir.error
               wget.mir.2.log
               etc.
wget.mir1      wget.mir1.1.log   wget.mir1.error
               wget.mir1.2.log
               etc.
wget1.mir      wget1.mir.1.log   wget.mir.error
               wget1.mir.2.log
               etc.
this.site      this.site.1.log   this.site.error
               this.site.2.log
               etc.

You can give the inputfile any name, that makes it easier to manage. This also 
makes it easy to stuff the lines in different files and call this REXX file
with a command line like (in a batch file)
  for %%a in (wget*mir) do wgetmir %%a
for inputfiles like wget.mir, wget.byte.mir, wget.site.mir, etc.

Remember, you can use any filename for the inputfiles, as long as the file
exists. If you don't supply a filename, the inputfile defaults to 'wget.mir'.

The possibilities are (almost) endless and the flexibility is (almost)
endless.

Caution: I believe it's not a good idea to download one and the same file in
two (or more) separate sessions at the same time, this applies also to
different links (url's) that gets resolved (points)
to the same file, at best this only halves the throughput of your internet
connection, or with bad luck this locks both sessions (but I don't intend to
have experience on this).

If more than one session are downloading different files at the same time,
these sessions ofcourse shares your internet connection, and the throughput of 
your internet connection. Separate sessions which are downloading at the same
time is most useful with mirroring (parts of) sites which are already partly
mirrored before, because then that session has to skip the files which are
already local and for that the demand for throughput is low.

------------------- begin of wgetmir.cmd -------------------
/* start wget mirroring each website/file listed in wget.mir */
arg InFile

if InFile = '' then
  InFile = 'wget.mir'

dir = directory( 'c:\www' )
'@set wgetrc=c:\www\wget.cfg'

cmd1      = 'wget.exe -m -c -np --tries=1 -k '
LogFile   = InFile||".log"
ErrorFile = InFile||".error"
filler    = copies('-',30)

LineNr = 0

check = stream( InFile, 'c', 'open read' )
if pos( 'READY', check ) = 1 then        /* if we opened file */
  do while pos( 'READY', stream( InFile, 'state' ) ) = 1  /* while stream is
ready */
    Line = LineIn( InFile )              /* read new line */
    if strip( Length( Line ) ) > 0 then  /* if Line not empty */
    do
      LineNr = LineNr + 1                /* keep count */
      LogFile = InFile"."LineNr".log"    /* logfile with numbered filename */
      say LineNr': 'Line                 /* report number and line */
      cmd = cmd1 || Line || ' -a ' || LogFile           /* assemble
command-line */
      check   = stream( LogFile, 'C', 'open write' )    /* open logfile */
      check   = stream( LogFile, 'C', 'seek <' 0 )      /* seek to end of
logfile */
      written = LineOut( LogFile, filler date('W') date() Time() filler )  /*
append day date time to logfile */
      check   = stream( LogFile, 'C', 'close' )         /* close logfile so
wget can use it */
      "@start /win /c /b" cmd                           /* start new session
with wget */
    end  /* if Line not empty */
  end  /* while stream is ready */
else /* couldn't open the file */
  do
    errmsg  = stream( InFile, 'D' )     /* ask description about possible
error */
    check   = stream( ErrorFile, 'C', 'open write' )    /* open errorfile */
    check   = stream( ErrorFile, 'C', 'seek <' 0 )      /* seek to end of
errorfile */
    written = LineOut( ErrorFile, date('W') date() Time() "Cannot open file"
InFile", it is" errmsg )
    check   = stream( ErrorFile, 'C', 'close' )
  end

check = stream( InFile, 'C', 'close' )

return
-------------------- end of wgetmir.cmd --------------------

(some lines in this REXX file might get wrapped, especially on a 80-columns
display, the wider the display the less lines are wrapped, I run my message
reader/editor in a 105-columns OS/2-session)


Injoy autostarts the 1x.cmd batch file:

------------------- begin of 1x.cmd -------------------
@echo off
c:
cd \www
call already c:\www\1x.cmd
if errorlevel 1 goto NoTime

TIME868
if exist TIME868.LOG type TIME868.LOG>>TIME868.NOG
if exist TIME868.LOG echo.>>TIME868.NOG
if exist TIME868.LOG del  TIME868.LOG

:NoTime
cd \mail\internet\pmmail
start /n pmmail.exe
call cheap
if not errorlevel 1 exit

cd \www
if exist wget.ftp    call wgetftp
for %%a in (wget*mirror) do start /win /c wgetmirror %%a > nul
if exist wget.mir    start /win /c wgetmir > nul
if exist wget.news   start /win /c wgetnews> nul
if exist wget.www    start /win /c wgetwww > nul
if exist wget.x      start /win /c wgetx   > nul
exit
-------------------- end of 1x.cmd --------------------

The textfiles wget.ftp, wget.mir, wget.news, wget.www, wget.x and files with
filenames which conform to the 'wget*mirror' filenamemask are inputfiles
(textfiles with the files/url's to get) for wget / the wgetmir.cmd REXX file.
For inputfiles which don't exist, no action is taken.

I've plain batch files that gives wget this file as input, so the url's in
this file are done one at a time.

------------------- begin of wgetftp.cmd -------------------
@echo off
setlocal
c:
cd \www
rem mode 100,45

set wgetrc=c:\www\wget.cfg
call datetime>>wget.ftp.log
if exist wget.ftp wget -k -i wget.ftp -a wget.ftp.log
endlocal
-------------------- end of wgetftp.cmd --------------------


------------------- begin of wgetmirror.cmd -------------------
@echo off
setlocal
c:
cd \www
rem mode 100,45

set wgetrc=c:\www\wget.cfg
if not (%1)==() set get=%1
if     (%1)==() set get=wget.mirror

call DayDateTime >> %get%.log
if exist %get% wget.exe -m -c -np --tries=1 -Q1m -k -i %get% -a %get%.log
endlocal
exit
-------------------- end of wgetmirror.cmd --------------------

You can supply wgetmirror.cmd too with different inputfiles and you can also
use for...in...do with one or more filemasks (separated by a space) between
the '(' and the ')', for example:
   for %%a in () do wgetmirror %%a
(I used this with the start command to run wgetmirror.cmd in separate
sessions)


------------------- begin of wgetnews.cmd -------------------
@echo off
setlocal
c:
cd \www
rem mode 100,45

set wgetrc=c:\www\wget.cfg
call datetime >> wget.news.log
if exist wget.news wget.exe -c -np --tries=5 -k -i wget.news -a wget.news.log
endlocal
exit
-------------------- end of wgetnews.cmd --------------------


------------------- begin of wgetwww.cmd -------------------
@echo off
setlocal
c:
cd \www
rem mode 100,45

set wgetrc=c:\www\wget.cfg
call datetime>>wget.www.log
if exist wget.www wget -k -i wget.www -a wget.www.log
endlocal
exit
-------------------- end of wgetwww.cmd --------------------


------------------- begin of wgetx.cmd -------------------
@echo off
setlocal
c:
cd \www
rem mode 100,45

set wgetrc=c:\www\wget.cfg
if exist wget.x wget.exe -m -t1 -np -Q1m -w1 -c -k -x -N --follow-ftp -i
wget.x -a wget.x.log
endlocal
exit
-------------------- end of wgetx.cmd --------------------

All this takes place in the c:\www directory on my system.

Already.cmd and cheap.cmd are in my c:\os2\rexx directory which is in my path.

------------------- begin of already.cmd -------------------
/* already done today? */
arg filename
if filename = '' then
  do
    parse source . . filename
  end

datum = ''
lastdate = ''
rc = 0
eat. = ''
eat.ascii = 'FFFD'x

datum = date( 'S' )         /* yyyymmdd */

if SysGetEA( filename, '.DONE', 'done' ) = 0 then
do
  parse var done 3 leng 5
  leng = c2d( reverse( leng ) )
  parse var done 5 lastdate +(leng)
end

rc = (lastdate = datum)     /* TRUE = 1, FALSE = 0 */
if (lastdate  datum) then
do
  leng = right( d2c( length( datum ) ), 2, '00'x )
  datumea = reverse( eat.ascii ) || reverse( leng ) || datum
  call SysPutEA filename, '.DONE', datumea
end
return rc
-------------------- end of already.cmd --------------------

I load all the REXX sysutil functions in my startup.cmd.

------------------- begin of cheap.cmd -------------------
/* can we call cheap (telephone/modem)? */
result = 0
DayOfWeek = date('w')

if DayOfWeek = 'Saturday' then result = 1
if DayOfWeek = 'Sunday'   then result = 1

if Time() < '07:45:00' then result = 1
if Time() > '20:00:00' then result = 1

say DayOfWeek Date() Time() 'result:' result

return result
-------------------- end of cheap.cmd --------------------

I hope someone might find this useful, so I hope I haven't posted this for
nothing.

  Greetings   -=Eddy=-        email: eddy.thilleman@net.hcc.nl

... SET RESTARTFOLDERS=DONTRESTARTTHEONETHATCRASHEDOKAY?
--- GoldED/2 3.0.1
2433/225