Batch Attachment Uploader

[Grant]

This guide explains how to use the Batch Attachment Uploader to bulk upload attachments (files) to existing records. It covers the two available matching modes, Match by Filename and Match by Mapping File, the permissions required, the fields filenames can be matched to, and how to upload multiple attachments to a single record.

For more information about what attachments are, please read:

Attachments in Specify

This guide explains the functionality of the Attachments system, which allows users to link digital media to database records. It details how the Web Asset Server manages these files externally and outlines methods for adding attachments. Specify provides a robust system for attaching digital media and documents to your collection data records. These attachments are managed through a dedicated service called the Web Asset Server / Attachment Server. An attachment is any digital f…

Features of the Batch Attachment Uploader

• Two Matching Modes: Choose between filename-based matching (parse record identifiers from each filename) or mapping file matching (use an explicit CSV file to define which file belongs to which record).
• Validation and Upload: Validate, upload, and roll back attachments as needed.
• Record Disambiguation: Automatically disambiguate records when a filename matches multiple records.
• Upload Control: Ability to stop uploads or rollbacks at any point.
• Network Resilience: Upload and rollback processes are tolerant of network issues and client-side crashes.
• Metadata Attachment: Attach remarks to the metadata of the dataset.
• Numeric Field Matching: When using numeric fields (e.g., Catalog Number) for matching, filenames like 000000001.jpg, 1.jpg, and 0001.jpg are treated as equivalent. However, text fields will differentiate between these formats. If you upload multiple attachments for a numeric field, you will need to include all leading zeros for the format, see below.

Permissions

To use the Batch Attachment Uploader, non-admin users must be assigned the “Bulk Attachment Import” role by an admin in the “Security and Accounts” section. This role grants full access to the uploader and can be customized as needed.

For more information on Security & Accounts and user roles, refer to this guide.

Working with Data Sets

Choosing a Matching Mode

When creating a new attachment dataset, you must choose how attachment files will be matched to database records. Two modes are available:

Matching Mode	Description	Best For
Match by Filename	Parse the record identifier directly from each filename using a field formatter.	Structured numeric fields like Catalog Numbers where filenames contain predictable identifiers (e.g., 000271806.jpg).
Match by Mapping File	Use an explicit CSV file that lists each filename alongside its corresponding record identifier.	Non-standard filenames, multiple files per record, or when you want to pre-define an exact import manifest.

[!NOTE]
The matching mode is locked once the dataset is saved. If you need to change the mode later, you must create a new dataset.

1. Navigate to the Attachments gallery by clicking the side menu icon and then click on Import in the top-right corner.

2. A dialog displaying existing attachment datasets will appear. Click on New to create a new dataset.

3. The Matching Mode dialog will appear, offering two choices:

   • Match by filename — The traditional method. Parse the record identifier from each filename.
   • Match by mapping file — Use a CSV mapping file (described in detail below).

4. Select your preferred mode and click Proceed.

   • If you chose Match by Filename, continue to Selecting and Validating Files (Filename Mode).
   • If you chose Match by Mapping File, continue to Using a Mapping File.

Selecting and Validating Files (Filename Mode)

1. You can select files in two ways:

   • Click Choose files or drag them here.
   • Drag files onto the blank area in the center.

2. By default, no field is selected for matching files to records. Choose the desired matching field from the picklist. Available options include:

   • Collection Object / Catalog Number
   • Collection Object / Alt Cat Number
   • Taxon / Full Name
   • Storage / Full Name
   • Collecting Information / GUID
   • Collecting Information / Collector Number
   • Loan / Loan Number
   • Accession / Accession Number
   • Gift / Gift Number
   • Preparation / Bar Code
   • Preparation / GUID
   • Borrow / Invoice Number
   • Treatment Event / Field Number
   • Exchange In / Exchange In #
   • Exchange Out / Exchange Out #
   • Reference Work / GUID
   • Reference Work / Title
   • Locality / Unique Identifier
   • Locality / GUID
   • Permit / Permit Number
   • Agent / GUID
   • Collecting Trip / Trip Name
   • Deaccession / Deaccession Number

3. After selecting the path, the system will mark files as correctly or incorrectly formatted. For instance, 000000001.txt may be marked as correctly formatted, while Not Correctly Formatted.txt is marked incorrectly if it does not match the numeric field requirements.

   

4. Click on Validate to test matches against records in the database:

   • If a match is found, a formatted version of the record will be displayed with a link to open it in a new tab.
   • If no match is found, the message No match will be displayed.
   • If multiple matches occur, the message Multiple Matches. Click to Disambiguate will appear.
   • Incorrectly formatted records will retain the label ‘Incorrectly Formatted’.
   

Using a Mapping File

The mapping file mode lets you use an explicit CSV file to define which attachment file belongs to which database record. This is especially useful when filenames do not contain recognizable record identifiers, or when multiple files need to be associated with the same record.

Preparing Your Mapping CSV

Create a CSV file with at minimum two columns:

• Match Value Column: Contains the record identifiers (e.g., catalog numbers). The column header should match the schema field name (e.g., catalogNumber) or its localized caption (e.g., Catalog Number).
• File Name Column: Contains the attachment filenames. The column header should be something like File Name, Filename, Attachment, or Attachment Name.

Supported file formats: .csv, .tsv, and .txt with comma, tab, or pipe delimiters.

Example CSV content:

catalogNumber	File Name
000271806	271806.jpg
000601146	601146.jpg
000601146	601146_2.jpg
000273074	273074.jpg

Loading the Mapping File

1. After choosing Match by mapping file in the Matching Mode dialog, the mapping file setup area appears.

2. Upload your CSV by clicking the file picker or dragging a CSV file into the area.

3. The system automatically scans the column headers and attempts to detect:

   • Which column contains the match values (by comparing headers against data model field names and their localized captions).
   • Which column contains the file names (by recognizing common header patterns like “File Name”, “Filename”, etc.).

4. If the auto-detection is incorrect, use the dropdown selectors to manually choose the correct Match Value Column and File Name Column.

5. Once both columns are selected, the system creates placeholder rows in the main attachment table — one for each entry in the CSV. Each placeholder shows:

   • The expected filename
   • The match value from the CSV

   Placeholders are initially marked as File Missing.

6. Click Proceed to confirm.

Uploading Files Against the Mapping

1. Select your attachment files using the file picker or drag-and-drop. As files are added, the system cross-references them against the mapping CSV:

   • Files matching a CSV row receive the correct match value and replace the placeholder.
   • Files not listed in the CSV are flagged as Not in Mapping File and appear at the bottom of the table.
   • Files with duplicate filenames in the CSV are flagged as Duplicate in Mapping File.

2. Choose the matching field (e.g., Catalog Number) from the field picklist in the toolbar, just as in filename mode.

3. Click Validate to verify that your match values correspond to existing database records.

4. Click Upload to attach the files to their respective records.

[!TIP]
You do not need to upload all files at once. The dataset saves your progress — including which placeholders have been filled and which remain missing — so you can resume later.

[!NOTE]
The CSV is the source of truth. If you reload the dataset or re-select files, the match values from the CSV are always reapplied, ensuring consistency.

Add Multiple Attachments to a Single Record

In many collections, it is common to have multiple attachments associated to a single record. For example, you may need to document different orientations of a specimen, such as ‘dorsal’ and ‘frontal’ views, or include multiple photographs of the same specimen to capture various details.

Add Multiple Attachments

To add additional attachments to a single record in your data set, you can simply add an underscore after the field value you are matching and input any value in the file name.

When matching multiple attachments, the selected path field must have a format defined in the UIFormatters file in App Resources. Additionally, this format must not have regex parts.

When matching a numeric field, such as a 9-digit Catalog Number, you must include all leading zeroes in the filename to ensure it matches the format exactly when adding attachments to a single record.

This should follow the format you see on the data entry form for Collection Object (e.g. if the catalog number field shows 000493839, this should be the file name).

Example:
You are uploading 3 attachments to a Collection Object with the catalog number 000493839.

Filename	Status
000493839.jpg	Correctly Formatted
000493839_dorsal.jpg	Correctly Formatted
000493839_frontal.jpg	Correctly Formatted

If you upload a single attachment, Specify will treat filenames such as 0000000493839.jpg, 0493839.jpg, and 493839.jpg as equivalents. If you upload multiple attachments, you will need to follow the format exactly.

Filename	Status
493839.jpg	Correctly Formatted
493839_dorsal.jpg	Incorrectly Formatted
493839_frontal.jpg	Incorrectly Formatted

Uploading Data Set

1. If an upload is possible, the Upload button will become visible. Click on it to start the upload process.

   

2. While the upload is in progress, a progress bar will display, along with an option to stop the upload.

   • Clicking Stop will halt the upload. At least one file will always be uploaded unless the stop action is executed immediately. The table will indicate which files were uploaded and which were not.
   • For remaining files after stopping, it will show Cancelled: Stopped By User.
   

3. If you refresh or open the link in a new tab, all previously selected files will display a warning, and you will need to reselect them before uploading. The system will match new selections against previously selected files based on filename, size, and type. If a match is found, the previous file will have (No File) removed.

   

4. Click Upload again. Files that were previously uploaded will not be uploaded again and will show as Skipped: Already Uploaded. Files not correctly matched will display Skipped: No Match or Skipped: Multiple Matches. Incorrectly formatted files will show Skipped: Incorrectly Formatted.

   

Rolling Back Uploads

If you made a mistake when uploading attachments, you can reverse this upload and remove the attachments from the database. This is called a rollback.

1. If a rollback is possible (i.e., at least one file has been uploaded), the Roll Back button will become visible. Click on it to delete the uploaded files from the asset server and corresponding records from the database.
2. Similar to the upload process, if you stop the rollback, the system will indicate which files were deleted and which were not.
3. For files that were not uploaded, after the rollback, the message Skipped: No Attachments Found will appear.
4. You can stop the rollback at any point.

[QuinnE]

Hello! We just uploaded our first batch of image - it was magical and went very smoothly.

We did have a question though. When uploading, we can only add in the linked field. However, there are a few fields - such as copyright - that we would like to update for all the records (ours is blank). I assume the metadata is currently pulled from what is embedded in the attachment. Is there any way for us to bulk edit the attachment fields, such as copyright?

Thank you for your help!
Quinn

[SPalpurina]

Hello there,

I am an admin but I cannot see the button Import in the top-right corner of the Attachments gallery. Is there some setting I should turn on to make it visible?

[robinsp]

I’m used to attaching multiple images to each of multiple Collection Objects using an Import Mapping File in Specify 6.8.03. I have not been able to work out a similar process in Specify 7. Is this option currently available?

[Grant]

Hi @robinsp,

Currently, there is no support for attachment mapping files, but we have a feature request on GitHub for this:

Add Attachment Mapping File support

I’ve added you as a requestor!

[Grant]

Hi @SPalpurina,

Can you verify you are on the correct version of Specify 7? Please share a copy of your Specify 7 System Information with us to review!

---

Hi @QuinnE,

That’s great to hear! This tool currently does not support batch update operations for individual fields (or field-level metadata being included in the upload), but we are developing a batch edit utility that will be available in the coming months. This will enable the kind of operation you described!

[SPalpurina]

Apologies. Here it comes: v7.8.13
Specify 7 System Information - 2024-08-28T08_14_21.855Z.txt (664.3 KB)

btw, @cicadina has mentioned that he plans to install soon a new version I am keeping him in the loop just to be informed.

[SPalpurina]

Oh, yes, now I see we need version 7.9.3, right?
Thanks for your time, @Grant. Next time I will keep in mind to check for which release the post is associated with.

[Grant]

Hi @SPalpurina,

Yes, once you update to version 7.9.3 or newer, this tool will be available.

[markp]

To prevent others from the same false assumptions I made today, the following statement

Files that were previously uploaded will not be uploaded again and will show as Skipped: Already Uploaded

Will only apply to files within that same bulk upload. If an attachment already exists in the system, and then an identical file is added to a fresh bulk upload for the same record, it will not be picked up as already uploaded, and will upload as normal.

In essence, the Already Uploaded functionality shouldn’t be used as a shortcut to avoid uploading duplicates overall.