Skip to main content

Hunting Invasive Species with HTCondor: High Throughput Computing for Big Data and Next Generation Sequencing

Summary

Large amounts of data are being generated that require hours, days, or even weeks to analyze using traditional computing resources. Innovative solutions must be implemented to analyze the data in a reasonable timeframe. The program HTCondor (https://research.cs.wisc.edu/htcondor/) takes advantage of the processing capacity of individual desktop computers and dedicated computing resources as a single, unified pool. This unified pool of computing resources allows HTCondor to quickly process large amounts of data by breaking the data into smaller tasks distributed across many computers. This project team implemented HTCondor at the USGS Upper Midwest Environmental Sciences Center (UMESC) to leverage existing computing capabilities for [...]

Contacts

Attached Files

Click on title to download individual files attached to this item.

6-McCalla-CDI_project_20170308.pdf
“CDI Monthly Meeting Presentation”
5.55 MB
Figure16_condor_flocking.png
“Schematic of flocking with HTCondor between USGS centers”
thumbnail 164.09 KB

Project Extension

projectStatusCompleted

Schematic of flocking with HTCondor between USGS centers
Schematic of flocking with HTCondor between USGS centers

Communities

  • Community for Data Integration (CDI)

Tags

Provenance

Data source
Input directly

Item Actions

View Item as ...

Save Item as ...

View Item...