I've searched for this function, and there are lots of answers, but I haven't been able to find anything that works for me.

I have a folder with about 30,000 files, and I want a terminal command or bash script that will move these files into different folders. About 3000 files for each folder. Some of the filenames have underscores, spaces and hyphens, so the command should allow me to move those files as well as the files without underscores, spaces, and hyphens. The files are .jpgs, but I'd like a command that I can customize for any file type and file amount, in case I need to use it in the future for different formats and quantities.

UPDATE:

I found a bash script that answers my needs. It will move files in a folder into subfolders. It works with any type of file. The number "3001" represents the number of files you want moved to each subfolder. This number can be changed. So if you have a folder with 20000 files and you want the 20000 files divided into batches of 500 and moved to subfolders then you would replace "3001" with "500". You can also modify the script to only move particular file types, e.g.: to move only .jpg files, change "for file in *" to "for jpg_file in *.jpg". Also change "$file" to "$jpg_file".

Please clarify your specific problem or add additional details to highlight exactly what you need. As it's currently written, it’s hard to tell exactly what you're asking. See the How to Ask page for help clarifying this question.
If this question can be reworded to fit the rules in the help center, please edit the question.

4

Are there any criteria you'd want to use to determine in which folder the files should go (e.g. alphabetical, year from the EXIF data), or simply 10 folders of 3000 files each?
–
ph0t0nixAug 13 '14 at 17:57

2

I don't get what you really want. What distincts one file from another. Why do you need this?
–
BernhardAug 13 '14 at 18:07

@ph0t0nix: Alphabetical would be best, because not all the files I will want to move have the necessary metadata.
–
user8547Aug 13 '14 at 18:11

@Bernhard: I don't like manually going through a folder with 30000 files and sorting them into folders because it's tedious an can be slow due to system limitations.
–
user8547Aug 13 '14 at 18:17

4

@user8547 It is unclear to me based on what you want to divide them. IF it does not matter at all, but you just want them to be 3000 per subdirectory at random, mention that at least. I also don't see the purpose, but that is a different topic.
–
BernhardAug 13 '14 at 19:44

Could you modify the script to move any files, regardless of having numerical or alphabetical filenames, uppercase or lowercase, and regardless of whether there are spaces, underscores, and hyphens in the filenames?
–
user8547Aug 13 '14 at 19:10

You seem to have a problem with quotes. In the first code block, you're quoting the *, which strips it of its meaning as a wildcard. In the second code example, you have only one (unbalanced) quote.
–
G-ManAug 13 '14 at 22:21

@user8547: ph0t0nix's answer can trivially be extended to create a directory for each digit. And then you could simply do mkdir /path/to/sorted/directories/special; mv *.jpg /path/to/sorted/directories/special after you've moved all the files whose names begin with a letter (or digit).
–
G-ManAug 13 '14 at 22:25

#!/bin/bash
shopt -s nullglob
for i in <directory>; do
mv *.jpg "$dir"
done

This will move all jpegs to $dir which you will need to set or you will need to make possibly create an array with the directory name(s). You can easily customize this code by switching *.jpg to any different format. You will need to tweak around with this snippet to customize it to how you want it.

@ryekayo: I just tried the updated script and it moved over 9000 files into a folder, which is too much. I'd like the script to automatically move 3000 files into separate directories, and create the directories on the fly. The name of the directories isn't important. As far as sorting, I'd like them to be moved in numerical-alphabetical order.
–
user8547Aug 13 '14 at 19:08

@user8547: this snippet is serving as an example for moving files to a given directory. In the moveFile function, you will need to create a counter that will count files to 3000 and then break out of the function. I will look up what can go in there and update it shortly.
–
ryekayoAug 13 '14 at 19:11

u may alter parameters, read man ls for sorting options.
this is just an example and not optimal solution (calling ls every time, give so many params to mv is not a good idea, it's better to use xargs instead if total length is too large) but I think it will help you to understand what do you really need.

Tx! But it only partially works. It created 10 folders, but only moved files into some of the folders. And the files that were moved into some of the folders didn't amount to 3000, but varying quantities. The filenames are not uniform. Some have spaces, letters, numbers, underscores. I wonder if that's why it's not working.
–
user8547Aug 13 '14 at 20:36

@Leben ls will as used above will not capture file names with special characters as described by the author.
–
Simply_MeAug 13 '14 at 20:59

It worked well and fast. Now my only problem is getting it to move any file into folders, not just ".jpg". It's also case sensitive and will not move ".JPG".Any ideas on how to tweak it to move any file, or at the very least any image file?

In order to move any file (regardless of extension), simply change the for loop: for file in * (and then also rename any occurence of $jpg_file with $file.
–
ph0t0nixAug 13 '14 at 22:17

@ph0t0nix: Tx! That worked like a charm!
–
user8547Aug 13 '14 at 22:45

@user8547 make sure it deals with files that have space in them.
–
Simply_MeAug 13 '14 at 22:47

@Simply_Me: What do you mean? The file names should have spaces in them? I tested it out with files that have no spaces, but do have hyphens and underscores, and it worked well.
–
user8547Aug 13 '14 at 22:51

The script below finds all of the files that match the searching criteria defined by file_screen, then executes a while loop to go move the selected files.

New directories are created based on modulo criteria: loop iteration mod files_in_each. If mod returns zero, new directory is created and files will be moved to it.

Parameters to tweak in your runs:

files_in_each=3000 # controls how many files are placed in each directory
directory_to_move="/home/shadowe/test1/test2" # where are the files located
file_screen="jpg" # only move files that match this criteria

Tx! But it doesn't work right. It created a bunch of folders, but only moved files into some of them. I changed "find test2/*" to "*jpg". Before I changed that it didn't work at all.
–
user8547Aug 13 '14 at 20:51

1

@user8547 no, your change is wrong. test2 should be changed to the directory name where you store the files.
–
Simply_MeAug 13 '14 at 20:52