Hi, Frank.

 > NP>  > I've just coded the 32-bit CRC which PKZIP & ARJ use
in (very) fast
 > asm.
 > NP>  > It takes 6 minutes to calc the CRC of 5400 files,
220meg, on one of
 > my
 > NP>  > hard disks. Also did the 16-bit CRC which all the other archivers
 > use,
 > NP>  > and the 16-bit XModem one but didn't bother timing those.

 > NP> Is the 32 bit code available in an EXE file ??????

 > Not today, but tell me what you want and it'll be easy to make it so -
 > like, what do you want me to do with the generated CRC?

 > Write it on the screen?

If the program is as fast as you say, then what's the use :-)
It would only slow the program down.

> Append it to the .EXE file unless there's already one there in
> which case report changes? I can do that, in fact I'd like to.

Not really an option as there are quite a few programs that will object
violently to this sort of tampering. Nortons being one example.

I have a program that does a CRC and stores it in the time field of the
files directory entry, but this causes problems as well.



 > FYI I'm trying to build something which I've wanted for a long time, to
 > report *any* changes to files (fuck the archive flag, useless), to
 > maintain a complete catalogue of all my files (spread across several
 > computers physically separated (by miles)), mirror certain directories
 > between those physically separate machines,

Two of the networked machines I mirror certain directories on, and have
done that using BAT files. The BAT files on each machine are identical
(they even mirror themsleves) thus allowing them to initiated from either
machine and mirror in both directions in the one operation.

 > include the contents of
 > archives such as ZIP files in all of the above,

 Looking inside the ZIPs is not a consideration for me. Just so long as i
know that the contents of the zip have altered.

 > etc, etc. It's a most
 > interesting exercise

Certainly is!
I am only partway down the path, but it became obvious I _had_ to do
something when the drive letters in use reached 24 :-)

 > and means I have to develop *fast* disk I/O,
 > *fast* CRC routines, *fast* directory sorting, *fast* archive
 > extraction, etc - and then put it all together. :-)

As far as the fast routines go, I will upload to TML with this message a
winprogram called DIGSIG.ZIP
It contains the complete description and formulas for 3 different methods
of generating a digital fingerprint of a file that you should find
extremely useful.

 > And I'd quite like the opportunity to put some of what I've done to the
 > test by other users - but I need to know what you want.

I'll test it if you write it, no problems !!!

What I want.......

1.
Have the program search the whole drive (including all subdirectories to
all depths) but only generate CRC's for the files that have an extension
that
matches one in a list of extensions held in (SCAN.LST) a text editable file.
 Like...
EXE
COM
BIN
OVL
etc
(the program would default to *.* if SCAN.LST does not exists.)


2.
Have the program take command line parameters of...
Drive to scan      (1 to 10 bytes)
Drive and directory where the applicable LST, DAT & LOG files exist.
Name of DAT file to use.
Name of LOG file to use.

3
Check for existence of the DAT file specified in 2
                    in the location specified in 2
If the DAT file does not exists, then create the file.
If the DAT file exists then the data it contains is to be used as the
benchmark for the current scan

4.
Store in the DAT file the following information.
Full Drive, Path and filename
File size in bytes
CRC or digital fingerprint
(repeated for each file scanned)


5
Check for the existence of the LOG file specified in 2 in the location specified in 2
If the LOG file does not exist then create it.
If the LOG file does exist then _append_ to the LOG file.


6
Store in the LOG file the following information...

Date and time of scan                                *****
Volume label of drive scanned                        *****
Number of files processed
Any differences in size or CRC since the last scan
Path and filename, size and CRC of any files added since the last scan
Seperator line ------------------------------         *****

I can handle from the BAT the items marked with ***** if you don't wish to
include them

7.
When there is a CRC or filesize difference the program should halt, display
on screen the full path and filename, the old and new sizes / the old and
new CRC's and request a reply as to whether the NEW size or CRC should be
written to the DAT file or to allow the old data to remain.
A simple "Update the DAT file with the new data ? Y/N " should suffice.
(It should NOT halt simply because of a new file existing on the drive)

8.
Indication on screen as to which drive is being scanned and some sort of
indication of what position the process is in towards completion.


The above is what I am already doing with my EXE.
I have the DAT & LOG files stored as hidden/system in the root
directory of the drive scanned, and I am using different named DAT &
LOG files on any given drive depending on whether the scan is being
performed on a local drive or a networked drive. This is essential as the
Pathname to any file is different when it is accessed across the LAN  (Even
the LAN path name could be different depending on which machine the scan is
initiated from)

For speed gain, and less HD head movement,(very significant) I place the
DAT & LOG files in a working directory on a RAMDRIVE and copy them back
to the correct location on completion.

It is the _same_ BAT file on all machines and it is aware of which machine
it is being called from. It will alter the Ram drive letter and also the
parameters it feeds the EXE accordingly


Other things that I will handle from within the calling BAT file...

The DAT & LOG files are System/Hidden but will not be by the time your
program has to access them

The viewing of the log file at the completion of a scan

Trimming of the LOG file if it is above a given limit.


What else do you need apart from time to write it ????? :-)

Cheers
Niels Petersen

--- FMail/386 0.98