Home / Technology / Improving Efficiencies with Batch Processing

Improving Efficiencies with Batch Processing

EDITOR’S NOTE: The following was written by Linda Straub Jones, Director, Collections Compliance, LexisNexis Risk Solutions

I attended 10 industry conferences in 2017. One common theme I heard, is that many companies still do not understand what a batch process is, or realize the efficiencies that can be gained by using batch processes.

When working with your various data vendors, many of them will have a process called “batch processing.” This is where you send in a group of accounts (called a batch file), the data vendor processes those accounts past whatever dataset you are looking for, and then returns the results to you via another batch file. There are generally two types of batch processing: scrub and monitoring.  

A scrub is a one-time look up of data for the accounts you are sending. Monitoring is an ongoing process, where the accounts are looked at each day to see if there are changes or new data available.  

Let’s first talk about the best practices for engaging with your data vendor, and then dive into the efficiencies gained by using a batch process.



How to Start

Many of your data vendors will have similar processes when sending/receiving batch files and even though some may have slight differences, the overall procedure should be fairly uniform.  

First, setup the batch process with your vendor. This may seem fairly obvious, but many companies do not realize that having the correct person on this call with your data vendor can help reduce the setup time and more importantly make sure you have the process correct right from the start. Going forward, you’ll want to have a technical contact at your data vendor, working with them and asking questions as they arise.  

From your company, make sure someone from your IT department is involved. This will be the person or people who will be creating, sending, receiving back and uploading the file to your system. If you only have one person – make sure they have a backup because batch processes generally run every weekday, and sometimes on Saturdays. Additionally, you will want to have a business person from the business unit that the data is flowing to – for example if you are setting up a bankruptcy scrub, include the manager of the bankruptcy team, if a phone scrub – include the manager from the dialer team as well as a collection manager, etc.

Then, consider your options – will you be sending in one file to run past several different data sets?  Will that be the same each time? If not, how is that communicated to your data vendor? Are there filtering options that may impact the return file (for example date filters or account-level filters)? Will you be setting up a waterfall of various products? Find out all options available, and make sure you walk through everything on your setup call and set expectations accordingly. Get the setup specifications in writing, and sign off on them before the process begins.

Will this be a one-time scrub process or monitoring? Does your data vendor monitor? Monitoring is a great option for high-risk pieces of data – for example bankruptcy or deceased. If you are aware, right away, each morning of the new bankruptcy or deceased hits on your portfolio, you can remove those accounts from the calling queue immediately.

Importantly, you must test the process – most data companies will offer a test period to ensure the process is working correctly before you go live. Find out what this process is, and make sure to test the sending and receiving of files with your vendor. Also test that the fields are populating correctly in your collection software.

Handling the data

While its crucial to setup your program efficiently, and effectively, you then must prepare for handling the files, both sending and receiving.

If you have collection software, check with your software vendor to see if they have existing processes with data vendors. Most already have several data vendors programmed in, and this will streamline the process for you. However, if your data vendor of choice is not already programmed into your collection software system, there are usually work-arounds.

This may sound basic, but make sure you are pulling and sending the correct accounts. I can’t tell you how many times I’ve been involved in a situation where a customer is asking for a large credit because they inadvertently sent their entire portfolio, instead of just the subset they are looking for new data on.  

You’ll need to ascertain how the data will be moved from your company to the data vendor. Make sure you both have the proper security processes in place to protect the data as it moves between your companies. Do not send any data via email, especially if it contains Personally Identifiable Information (PII). Email is not secure, and moving PII this way is too risky. SFTP or FTP with PGP encryption is generally the baseline most companies work with.

When using new data, make sure you have a home for the new data you receive. Whether you are using a collection software vendor or have your own proprietary software, make sure you know which field the data will go into. Try not to put everything into a “notes” section – that can be very confusing and hard to retrieve.

Lastly, if you are incorporating the data into workflows, make sure you have tested the process and are ready for your new data to be incorporated. You don’t want to buy data and then have it sitting there not being used. If you are using the new data in a score – again, make sure you have tested and know your expectations for the data.



The most obvious efficiency is that you no longer need to look up each account individually. This is a huge time saver. Batch can process millions of files daily, much more than manually looking up information.

Monitoring offers the additional bonus of only having to send a file to the data vendor once. The vendor will look daily for changes in that file. If you are monitoring you can generally add and delete files as needed.  This saves time in having to pull large files daily and send them back and forth. Usually batch files can be processed at your vendor overnight and be available to you first thing in the morning.

Batch processes are also less costly than manually looking up data. It’s not just the data cost you need to look at but also the cost of your employee’s time. Batch processes allow you to re-allocate those employees to more revenue generating activities.


If you are not currently using batch processes, this is something you will want to consider going forward.  Batch scrubs and monitoring will not only save you time and money, but will give you the peace of mind that all of your accounts are being attended to on a daily basis.  


About mikegibb

Check Also

FCC Report on Robocalls Short on Solutions to Fix Problem

The Federal Communications Commission this week released its first ever report on illegal robocalls, and …

Leave a Reply

Your email address will not be published.

Skip to toolbar