Scrubbing Data

Keeping your data neat and clean is always a good idea, whether importing updates or building a database. If you bring junk into your Education Management database, you will end up with junk data. Scrubbing data ensures you are working with accurate and concise data. It may take a little extra time, but it will pay off in the long run.

  • Remove duplicate records from the file so that duplicates are not created in the system.

  • Remove redundant ethnicities if necessary - i.e. Caucasian & White or Black & African American. Filters in Excel can help you identify redundant entries.

  • Remove redundant religions if necessary - i.e. Christian (Non-Denominational) & Non-Denominational Christian. Filters in Excel can help you identify redundant entries.

  • Check to make sure phone numbers are in the format you want them to be in the system - i.e. 555-555-5555 or 555.555.5555 vs 5555555555. Controlling the way this data is returned in Excel can help standardize phone number values.

  • Ensure that the column headers match the column headers outlined in the file layout guide. If column headers do not match the layouts supplied in the file layout guide the data will not be imported. Do not supply additional columns like Role or Username in the General User import.

  • Ensure all users have a Host ID. Filters in Excel can help you identify entries without a Host ID.

  • Ensure all users have a unique Host ID. Filtering for duplicate values in the Host ID column can help you identify entries without a unique Host ID.

  • Check to make sure dates are in MM/DD/YYYY format for birthday, hire date, or enrollment date. Excel has a tool to format dates.

  • Check to make sure Grad Year is in YYYY format. Filters in Excel can help you quickly identify if any grad years are not in the correct 4-digit format.

  • Ensure that the salutations are for the address record and not the individual people within the household.

  • Ensure that the school levels and grade levels match what is in the system under Core, Settings, School information, School & grade levels. Filters in Excel can help you identify entries that do not match what is in Core.

  • Look for data shifts in your file(s) for both the fields and records.

    • Examples may include:

      • Data swapped in two fields

      • Data shifted left or right one or more positions to the wrong fields; this may be in all records or in individual records.

      • Data shifted up or down in records (e.g. host ids associated with the wrong users)