The progress file consists of three entries on each row of the file.
- The hash of the file contents.
- The message ID of the last message that was archived successfully.
- The file name that was scanned.
The progress file allows the worker to easily detect when new rows are added to the log files and avoids re-sending entries in the log files multiple times across multiple jobs. If you have a job that runs two times in a single day. The first job runs and archives an entire log file. The second job runs the later, but there have been new log entries added between the two job runs. In this case the hash would change, indicating that there are new entries, and we skip all the messages up to the message ID in the file since they have already been archived. If the hash doesn't change, we skip the file completely.
MMS, SMS, and Phone will each get their own row in the file. The third entry in the row should be the file name.
The .worker file can be deleted, to re-archive all messages that day. Or, the progress file can be edited to re-archive all messages for a specific form of messaging.