Describing Queue Folders

What does Queue Folders do--a more complete description

Queue Folders is a file system watcher for Windows platforms which associates a local folder with a queue in RPM Remote Print Manager® ("RPM") Elite. We arrange to monitor this folder for new files. Files are automatically submitted to RPM as a print job, then removed.

What happens when you create a job

When you create a new file either by copying or moving an existing file or an output of a program, Queue Folders detects the new file and sends information about it to RPM. We do this by getting file events from Windows.

It turns out that Windows can send file events much faster than we can create jobs. On my development machine, I routinely count 40 or more jobs per millisecond. It takes roughly five thousand times as long to actually create the job in the database, copy the file, look up the username, etc. For that reason, RPM keeps the jobs in a work queue internally until they are all created.

You can see this in action if you want. Run the RPM UI and select a queue with a watch folder. You’ll notice the queue list at the left has “Name”, “Jobs” and “Total”. The total column is the cumulative number of jobs you’ve sent to this queue. 

Now, copy several hundred files to that watch folder. The number in the total column will continue to spin up for a time after the files are copied. On my system, it takes anywhere from 10 to 40 ms to create a job, which works out to roughly 25 to 100 jobs per second, compared to 40,000 per second that Windows can potentially send us.

Access

When RPM creates the job, it needs to have unrestricted access to the file, because it’s going to look up some information on the file; then copy it to a folder where jobs are spooled for that queue, and finally remove the file. We realize that the other program might not be done with the file so we wait for about 1 second to begin this process.

RPM will look up a username associated with the file. For this, it needs to temporarily open the file. If another process still has the file open, RPM will retry in five seconds.

On my system, Windows returns the result “Administrator” rather than “Dave”, which is what I would expect. Windows Explorer shows me “Dave”, although I’m part of the network admin group so perhaps “Administrator” makes sense. 

RPM will use the filename part of the path as the job name. 

Jobs are created in RPM in the order they are sent to the folder.

How Queue Folders handles files already in the folder

If Queue Folders is assigned to watch a folder that already has jobs in it, it will get all the filenames, then sort in order of the “last write time”. This would normally be the last time the file was updated. Then the files are sent to RPM to be processed into jobs.

It’s worth noting that the “copy” and “move” commands in the command prompt might create a file in your watch folder with the same “last write time” as the file had in its original folder. This means, you might not get the files to process in the order you expect, and there’s nothing we can do to change that since “last write time” seemed more important than any other metric.

However, if you are generating files from program output, you can rely on the “last write time” to be the same as when the program finishes 

There are two times when you would run into this:

  1. When you launch RPM, and Queue Folders is licensed, and you have files in one or more of the watch folders you’ve configured Queue Folders to monitor
  2. When you manually configure a queue with the UI, while Queue Folders is already running, to watch a folder which already has files

Where did my files go? And when?

Queue Folders notifies RPM about the files but it doesn’t do anything with the files on its own.

Remember that RPM receives the file info from Queue Folders faster than it can create jobs. Since we don’t want to present any delays to Queue Folders, so that it can monitor files uninterrupted, RPM caches the files and creates jobs when it can.

RPM will remove the file from the watch folder as one of the last steps in creating the job. This is why, if you send a large number of files to the watch folder at a time, they don’t disappear all at once; the number reduces over time.

And remember that RPM is attempting to process jobs in the order they are created on your hard drive, which means that it may take some from one watch folder, then from another. Again, we felt the priority was “in the order created” which may not always come out alphabetically.