ScienceBase

ScienceBase Geospatial Services

ScienceBase allows users to catalog, manage, share, and interact with geospatial information records. ScienceBase geospatial services are compatible with various client and online GIS software.

You can get the specific details of these capabilities by reading the following sections. You can also see a quick video tour:

ArcGIS Services
Video shows how to upload ESRI Service Definition (.sd) files and create ArcGIS REST services, which may be consumed in software applications.

Shapefiles and Rasters
Video shows how to upload shapefile and raster formats and consume the generated open source GIS formats.
 

Shapefiles

A shapefile is a set of multiple files that may be uploaded as a single zipfile or as individually selected files. An uploaded shapefile in .zip format may be unzipped during the file upload process - an option to unzip the shapefile is presented to the user. A projection (.prj) file must be included in the shapefile bundle to render and create GIS Services. ScienceBase automatically generates WMS and WFS services for uploaded, unzipped shapefiles. The WMS and WFS service URLs are located in the "Spatial Services" section under the preview map on a ScienceBase item (the item needs to be 'public' for these GIS Services to be accessible).

If a map display and GIS services are not needed, the uploaded shapefiles may remain zipped during upload process. Symbology consisting of a solid blue outline with a transparent blue fill is the default symbology.

Rasters

Any raster format may be uploaded to ScienceBase. However, only GeoTiff (.tif) files are rendered and given GIS Service capabilities. The generated WMS and WCS service URLs are located in the "Spatial Services" section under the preview map on a ScienceBase item (the item needs to be 'public' for these GIS Services to be accessible). A blue color ramp is the default symbology.

Shapefiles and Rasters Video - shows how to upload shapefile and raster formats and consume the generated open source GIS formats. 

Geospatial Data Symbology

For stand-alone shapefiles and rasters, ScienceBase will default to blue symbology. To customize your symbology you can use one of the following methods:

1) Upload an ESRI Service Definition to ScienceBase. This process is explained below in the 'ArcGIS Mapping Services' section.

2) Add open-source symbology to a stand-alone shapefile or raster.

Shapefile:

Install and open QGIS Desktop.
Add data to your map and style as appropriate.
Open your vector layer's properties.
Click the Style drop-down in the lower-left > Save Style > SLD File...
In your ScienceBase item, delete any existing .sld file and then upload your newly created one. The filename for your .sld should match the filename of your shapefile.

Raster (there are two options for stand-alone rasters):

1) Install and open QGIS Desktop.
Add a raster to your map and style as appropriate.
Click Pugins > Manage and Install Plugins...
In the search box, type 'sld'. Click on SLD4raster > Install plugin > Close.
Click the SLD plugin icon () in your toolbar > Generate SLD > Export as SLD
In your ScienceBase item, delete any existing .sld file and then upload your newly created one. The filename for your .sld should match the filename of your raster.

2) Add a colormap to a .tif (for single-band rasters that are stored as 8 or 16-bit unsigned integer). If the .tif file contains a colormap, ScienceBase will render the raster accordingly. To create a colormap in ArcMap:

In ArcMap, open the Properties of the GeoTiff layer and click on the Symbology tab. Click Unique Values to symbolize the raster. Then, click the Colormap drop-down arrow and choose Export Colormap. After saving the colormap file, open ArcToolbox > Data Management Tools > Raster > Raster Properties > Add Colormap.  Select the input GeoTif in the first box, and select the colormap file in the 3rd box (the 2nd box may remain empty).  After the process runs, the raster may be uploaded to ScienceBase and displayed showing the personalized symbology assigned in the ArcMap session.

ArcGIS GeoDatabase (file and personal)

A GeoDatabase should be zipped prior to uploading into ScienceBase. During the Import File process, DO NOT select the unzip option. Geodatabases cannot be read/displayed on a map by ScienceBase. Users may download the zipped GeoDatabase file from ScienceBase and subsequently, unzip the file for use on local systems. Additionally, a GeoDatabase can be read/rendered as part of a GIS Service when incorporated in an ArcGIS Service Definition file (instructions are in the section 'ArcGIS Mapping Services' below).

Map Packages

Previously, ScienceBase generated an ArcGIS Service when a Map Package was uploaded. ScienceBase no longer starts an ArcGIS Service when a Map Package is uploaded. If an ArcGIS Service is needed, a Service Definition file should be uploaded (instructions are in the section 'ArcGIS Mapping Services' below).

ArcGIS Mapping Services

A Service Definition (.sd) file must be uploaded to generate an ArcGIS Service (uploading a Map Package will NOT start an ArcGIS Service). If a GIS Service is not necessary, consider uploading a Map Package (.mpk) instead of a Service Definition. Service Definition uploads automatically create ArcGIS Services for items that are publicly readable (as set in ScienceBase's permissions). For items that are not publicly readable, you must take an extra step to create your ArcGIS Service. In the 'Item Actions' section at the bottom of the page, go to the 'Manage ArcGIS Service Definition' section and click on 'Create ArcGIS Services'. ScienceBase permissions secure the ScienceBase item, but the ArcGIS Service will be publicly accessible.

To Create a Service Definition File (use ArcMap 10.1 or higher):

  1. Create a new, or open an existing ArcMap document (refer to the next section titled 'Service Definition Best Practices' for tips on creating a speedy ArcMap document/ArcGIS Service).
  2. Once the ArcMap document is finalized, save it and choose File > Share As > Service from the main menu.
  3. Click the radio button for Save a service definition file. Click Next.
  4. In the Save a Service Definition window, choose the radio button for No available connection and check the box for Include data in Service definition when publishing. The Service Name box may be left unaltered - ScienceBase will generate a service name. Click Next.
  5. Choose a local location to save the Service Definition file. Click Continue.
  6. The Service Editor appears. Click on Capabilities in the left-hand pane. Then, check the box for WMS (SB enables this by default). Optionally, you can check the WFS box, if you want that capability. If you don't need WFS, leave it unchecked because it will enforce stricter rules. ScienceBase uses a standard template, therefore, all other options in the Service Editor (other than WFS) will be ignored. Click Analyze to have ArcMap examine the document and determine if it can be saved as a Service Definition. Fix any 'errors' in the Prepare window - this must be done before the Service Definition may be staged locally on disk. There's no need to correct 'warnings' stating Map is being published with data copied to the server using data frame full extent.
  7. When done fixing the 'errors' and, optionally, any 'warnings' and 'messages', click Stage. The Service Definition is saved to your specified location and may be uploaded to ScienceBase at any time.

ArcGIS Services Video - shows how to upload ESRI Service Definition (.sd) files and create ArcGIS REST services, which may be consumed in software applications.

Service Definition Best Practices

General rules for creating a usable Service Definition/ArcGIS Service:

  • If you need to make a change to an existing ArcGIS Mapping Service, please delete the old .sd 'extension' first and then add in the new one. You cannot add two Service Definition files to one ScienceBase item.
  • Creating a .sd file will copy the ENTIRE dataset your ArcMap document references, not just what is seen in the current data frame extent. This can produce very large, unmanageable files.
  • Make an attempt to fix 'warnings' and 'informational messages' in the 'Analyze' step to further improve performance.
  • If the service will be combined with publicly available basemaps (ArcGIS Online, Google, Bing), it's strongly recommended to project the data to match the standard coordinate system of these basemaps: Projected Coordinate Systems > World > WGS 1984 Web Mercator (auxiliary sphere). Failure to do so can result in a Service slowed by on-the-fly projection.
  • Make sure all datasets and data frame use the same coordinate system. Failure to do so can result in a Service slowed by on-the-fly projection.
  • When authoring your Map Document, set large, complex datasets to only display when zoomed-in really close.
  • Avoid using highly complex, large datasets that contain too many vertices. Examples of datasets that can cause a slow Map Service include:   - Raster datasets that have been converted to vector (the original raster dataset should be used instead) and complex multipolygons with many vertices.
  • Use the 'Simplify' tool to help overcome the problem of a slow service with too many vertices: ArcToolbox > Cartography Tools > Generalization > Simplify Line/Polygon. Use the 'point remove' method and experiment to find the right balance of speedy performance versus accurate representation of the data.
  • Click File > Map Document Properties and fill in the form to the best of your abilities. This will be used as metadata for your Service.
  • Make sure to generate pyramids for all raster datasets (this is very important).
  • Large raster datasets should use compression. We recommend converting all rasters to .tif format using  'LZW' or 'LZ77' compression. Failure to do so can lead to unfavorable file size, slower performance, and problems in the unpackaging/unzipping process. To compress rasters using ArcCatalog: right-click on raster >  Export > Raster to Different Format... > Environments (button at bottom) > Raster Storage > Compression > LZW or LZ77. Make sure your output raster is saved as TIFF format (.tif on the end of the file name).
  • ESRI Service Definition Size Limits -- Single file uploads should not exceed 1-2GB. ScienceBase can process uploaded .sd files if the unzipped content does not exceed 4GB. 

Downloading a Service Definition

A Service Definition file contains the Map Document (.mxd) and the original GIS data. To use these data, download the .sd file from ScienceBase and use unzipping software to unzip the .sd file. Then, drill down to find the specific files of interest. The Map Document (.mxd) resides in the 'v10x' folder. 

Connect to ScienceBase Geospatial Services using ArcGIS Desktop

Map Service Migration from ScienceBase ArcGIS Server to ArcGIS Online (AGOL) 

The ScienceBase team has a newly developed workflow to pull an existing ESRI service definition (.sd) file from on-premise storage in Denver and make a small modification to the file to work with the USGS ArcGIS Online (AGOL) instance. An automated process can then move the file to ScienceBase cloud storage, push the file to AGOL to start an ESRI map service in AGOL, and update the ScienceBase item JSON with the new map service URL. This will result in an updated map service URL being listed in the item, and a change in the file location in the JSON, but otherwise the item will remain unchanged. This workflow is intended for public/finalized data that benefit from being additionally accessible via a public ESRI map service. The intent is to support better uptime and service resiliency for these resources, and the process is also consistent with Bureau goals to move resources into the cloud. The team is currently testing this process with authors, with plans to migrate all ScienceBase ArcGIS services to the cloud at a future point. We will communicate additional plans as they evolve. Users interested in testing this new process, or with a need for maximum uptime in map services, can provide a list of item IDs in ScienceBase with associated AGS services and the ScienceBase team will perform the migration on their behalf. If you are interested in trying out this functionality for existing ScienceBase items (or know others who might be) please contact sciencebase@usgs.gov. 


Site Team