findmissing is a python script that prints out missing numbers in an incrementing or decrementing sequence of numbers.

findmissing has been developed and tested with Python 2.7 and 3.6.

I had some really big files I needed to upload to Google Drive, so split them into 1 GB files. An annoyance with the Google Web interface is that it will upload files that are queued by random rather than in order. So if you queue up the following files:

SubFile.001
SubFile.002
SubFile.003
SubFile.004
SubFile.005

It may start with SubFile.004 and then Subfile.002 and so on. Allowing for potential gaps in what files are uploaded, especially when the upload fails for whatever reason.

Initially, when this happened I had to resort to manually determining which files were missing and reupload those. More often than not I would inadvertently upload duplicates (Google Drive allows for files with the same name to exist at the same path), which would waste time in uploading files I didn't need to upload and additional effort in removing duplicated files.

I use gdrive to query the files I have uploaded to Google Drive with the command gdrive list --query "name contains 'DSC003'" | sort -k2:

Id                                  Name         Type   Size     Created
1vBpXEM3_Jal1yUcHHghLdrG3rXrxQ3p0   DSC003.002   bin    1.1 GB   2018-06-12 13:55:37
1RknmQFhtzNDorO4Lp8AGpr4S_zmBqwKc   DSC003.008   bin    1.1 GB   2018-06-12 14:26:43
1Dgt5ZFKT3zilwPZdiPyPmhM0Y3Fz_xo8   DSC003.010   bin    1.1 GB   2018-06-12 16:57:14
16QPadXBXPGIwiRhzMFRuY7tZkdIlJ1DI   DSC003.011   bin    1.1 GB   2018-06-12 16:54:57
1zD492J3F42Azeoct82DMlXRRuWA8AUiY   DSC003.015   bin    1.1 GB   2018-06-12 17:32:12
1bah8OM_F49BNeoct8M2DlXRRuOD8AUiY   DSC003.019   bin    1.1 GB   2018-06-12 17:24:41
1eZfY4qn3Ol6w1BFafQn6YDy1uOU82NNA   DSC003.020   bin    1.1 GB   2018-06-12 17:21:46
1x-P84Jq1cyuJCW2Opvy4m_qOjOHvAKtA   DSC003.021   bin    1.1 GB   2018-06-12 17:27:16
1O4TjR8Cl31A5qxHkyVmhmukp4dofCO3t   DSC003.025   bin    1.1 GB   2018-06-12 17:36:00

Note: The above sample can be found in readme_sample.txt.

The above output shows nine 1.1 GB files in google drive. If we look at the extensions of the files it is evident that there are several files missing.

findmissing can be used to determine the missing files like so:

$ python findmissing.py -f readme_sample.txt -p "DSC003\.(\d+)"
3
4
5
6
7
9
12
13
14
16
17
18
22
23
24
$ 

The -p flag is required and takes in a regex expression with a group (specified with parenthesis) denoting which portion of the string contains the number.

Also, the script is capable of reading from stdin so redirecting output to it is possible. Here is the aformentioned example but reading from stdin instead of from a file:

$ cat readme_sample.txt | python findmissing.py -p "DSC003\.(\d+)"
3
4
5
6
7
9
12
13
14
16
17
18
22
23
24
$ 

In the example above, files climbed up to DSC003.029. We can inform the script of this with the --last (-l) flag like so:

$ python findmissing.py -f readme_sample.txt -p "DSC003\.(\d+)" --last DSC003.029
3
4
5
6
7
9
12
13
14
16
17
18
22
23
24
26
27
28
29

Conversely, files started at DSC003.000, which we can inform the script of as well (with --first [-i]):

$ python findmissing.py -f readme_sample.txt -p "DSC003\.(\d+)" --first DSC003.0 --last DSC003.29
0
1
3
4
5
6
7
9
12
13
14
16
17
18
22
23
24
26
27
28
29

These two options can also be used together to hone in on a specific sub-range of missing values. For instance let's say we wanted to only view missing values from 10 to 20 (inclusive):

$ python findmissing.py -f readme_sample.txt -p "DSC003\.(\d+)" -i DSC003.10 -l DSC003.20
12
13
14
16
17
18