Official US Government Icon

Official websites use .gov
A .gov website belongs to an official government organization in the United States.

Secure Site Icon

Secure .gov websites use HTTPS
A lock ( ) or https:// means you’ve safely connected to the .gov website. Share sensitive information only on official, secure websites.

Secure Data Commons

In This Section

Secure Data Commons - UseCaseCrossProjectAnalysis

Cross Project Analysis  

Identify Technologies and Systems to Enable Intra-Project Analysis and Scale to National Level 

A research program has identified internal DOT Researchers and external Partners (academia and other federal agencies) to share and analyze data from various grantees. SDC services helped support Researchers to conduct multiple data set analysis and develop general purpose code for data access across multiple projects. Research  Programs are enabled to focus on gaining strategic value and policy insights, as well increase ROI future large DOT grant deployments.

Why Should Your Research Program use SDC Services?

  • Set up for collaboration internally (DOT) and externally (academia and other federal agencies)
  • Scalable for data size and computational needs
  • Includes data science tools that can be extended and shared with partners
Last updated August 2022

Secure Data Commons - ProjSpotLightBTSWaze

BTS Partnership Plan on Incorporating SDC Waze Data 

 

Utilize SDC to build analytical weather-transportation modeling tools

The Bureau of Transportation Statistics (BTS) program has a need to identify congestion hotpots, flood impacts, and severe storm impacts to support the Emergency Planning Transportation Data Initiative and national highway traffic cluster analysis (Waze “Hot Spot” Analysis).  BTS utilized the Secure Data Commons (SDC) to build proofs-of-concept of analytical weather-transportation impacts.

Emergency Planning Transportation Data Initiative

The Emergency Planning Transportation Data Initiative is supported by a partnership between BTS and National Center for Atmospheric Research (NCAR). SDC provided support to Researchers conducting data integration on severe weather with the national transportation system, historical analysis and forecasting components, and post-op evaluation of dataset limitations/opportunities. This initiative was able to make discoveries useful for public safety and emergency evacuation preparedness at the multi-state scale. 

Waze “Hot Spot” Analysis

The Waze “Hot Spot” Analysis builds upon research Volpe has already conducted for the Safety Data Initiative. It includes the research evaluation of SDC Waze data, developing a proof of concept on a subset of the nation’s highway system data, and an experiment on predictive analysis on traffic forecasts. BTS Researchers were able work on machine learning and cluster analysis to evaluate options for creating a daily dashboard of “hot spots” of operational characteristics on the nation’s highway system: Congestion/traffic jam locations, Bottleneck sites, and Weather-related Road closures. 

Sponsor: Bureau of Transportation Statistics (BTS)
Last updated December 2022

Secure Data Commons - ProjectSpotLightsSummaary

Project Spotlights

Project Spotlights are designed to bring awareness to the how SDC was utilized to support the research program needs. 

 

 
 
 
 
 
Last updated December 2022

Secure Data Commons - Data Security

Data Security Overview

 

The Secure Data Commons (SDC) platform stores and analyzes sensitive data with fine-grained secured access controls and secured data export processes.

Security Posture

SDC inherits DOT Enterprise Cloud security controls, monitoring services, and polices related to network, encryption, access, etc. The platform is classified at the Moderate sensitivity level, which requires safeguarding data that may include personally identifiable information (PII) and confidential business information (CBI). It is configured with comprehensive audit logging to monitor actions SDC cloud resources to reduce security and operational risk. SDC offers authorized and controlled access to individual datasets and the metadata associated with those datasets.

Security Services and Controls

SDC is configured to encrypt data in transit (HTTPS, SSH) and at rest, using 256-bit Advanced Encryption Standard (AES-256). User access is constrained by Identity Access Management (IAM) roles assigned to users’ workstation and implementation of database user permissions. Data cannot leave the platform without approval from the Research Project’s Data Steward.

How We Protect Your Research Data

Each Data Provider signs a Data Agreement that provides rules for sharing metadata and handling data. The SDC team also works with each project team to document and secure PII and CBI data within the platform. SDC has implemented processes to export secure data out of the SDC so that Researchers can export data based on the compliance and data usage policies set forth and approved by the Data Provider.  Once an export request has been submitted, the request will be sent to the appropriate Data Steward. Data Stewards are responsible for accepting or rejecting the export requests for researchers so that they are able to get their data products out of the SDC system.

For information on Exporting Data Out of SDC, please see:

Data Provider Guide: Chapter 5, Exporting Data Out of SDC

Research Team Guide: Chapter 4, Exporting Datasets from the SDC

For more information on SDC, please see SDC Capabilities (pdf).

For more information on  SDC Security Capabilities, please see SDC Data Security (pdf).

Last updated November 2022

Secure Data Commons - ProjSpotLightCVP

Connected Vehicle Pilot (CVP) Deployment Program

 

Facilitating independent evaluation of the safety impacts of connected vehicle technology 

The CV Pilots Deployment Program sought to combine Connected Vehicle technologies in innovative and cost-effective ways to improve traveler mobility and system productivity, while reducing environmental impacts and enhancing safety. The USDOT selected three pilot deployers: New York City Department of Transportation (NYCDOT), Tampa-Hillsborough Expressway Authority (THEA), and Wyoming Department of Transportation (WYDOT).  All three deployments provided post-processed data to the Secure Data Commons (SDC).  The SDC provided support to Independent Evaluators conducting safety-related assessments analyzing vehicle paths, driver alerts, forward collision warnings, and imminent collision warning data. These safety evaluators were able to make discoveries from CV Pilot data in a variety of formats including: KML Files, CSV Files, Auto Generated Reports, and Data Histograms within SDC platform. Per privacy-related agreements with the deployment sites (documented in their Data Management Plans), these data have been retired. Data that have been scrubbed for sensitive and personally-identifiable information, along with associated analytic tools, may be found on the ITS Data Hub.

For more information, please visit Connected Vehicle Pilots

Sponsor: Intelligent Transportation Systems Joint Program Office (ITS JPO)
Last updated July 2022

Secure Data Commons - Capabilities

About This Document

Description of multiple capabilities and services provided by the USDOT OCIO Secure Data Commons (SDC). These capabilities and services support data & visualizations, security, open-source collaboration, obtaining your feature enhancements requests, and dedicated SDC...

Secure Data Commons - ProjSpotlightFraARDS

Agile Railroad Data Science Programs (ARDS)

 

Integrate railroad data with external datasets, from other DOT modes and other government agencies

Agile Railroad Data Science (ARDS) project integration into SDC continues to help accomplish Federal Railroad Administration (FRA) Safety Data objectives using the SDC to modernize FRA’s analytical capabilities, and to establish a reliable “hub.” The SDC supports FRA’s vision to integrate railroad data with external datasets to facilitate greater understanding and improvement of railroad safety. The FRA has already made Railroad Accident and Incident, Highway Rail Grade Crossing Incident, Activation Failure and False Proceed and Positive Train Control data available in SDC and is currently working to ingest external datasets, such as population data from the 2020 Decennial Census. Additionally, the SDC is working on developing a data visualization service to make it easier for users to export data into a data visualization tool.
 

Sponsor: Federal Railroad Administration

Last updated April 2022

Secure Data Commons - ProjSpotlightOSS4ITS

Open-Source Software for Intelligent Transportation Systems (OSS4ITS)

 

OSS4ITS advances the deployment of interoperable transportation systems

USDOT has several development projects underway that are using Agile Development practices to create open-source software with robust Communities of Practice. These include, but are not limited to, the Secure Data Commons (SDC), the Operational Data Environment (ODE), CARMA, and the V2X Hub. Each of these projects support different parts of an overall US DOT Intelligent Transportation Systems (ITS) deployment architecture and are managed separately, with individual development teams. OSS4ITS is an ecosystem that promotes the deployment of interoperable transportation systems.

Architecture Reference for Cooperative and Intelligent Transportation (ARC-IT)

OSS4ITS ecosystem promotes development of open-source code that ensures code reuse and shared architecture. The software suite provides a common framework for planners and engineers to design and implement interoperable systems to support research teams in DOT, software companies, and device manufactures.

Upcoming OSS4ITS Reference Implementation

Research teams in DOT, academics, and the private sector are looking for the latest Work Zone data technologies in progress. The full functionality of OSS4ITS software suite (including SDC) enabled data exchanges for the Work Zone use case. The team prepared and informed transportation agencies and companies on how OSS4ITS software tools work seamlessly for this use case. Also the TFHRC cloud solution established the connection between CARMA analytics and SDC so that CARMA data can be transferred via AWS.

Last updated December 2022

Secure Data Commons - ProjSpotLightWazeAlerts

Highlights from the Waze Alerts User Community

 

Compiled SDC Crowdsourced Traffic Data on Traffic Jams, Hazardous Roadside Parking, Crashes, and Reported Road Closures

Volpe developed a COVID-19 Waze Traffic Alert Dashboard in March 2020 to track relative changes in weekly traffic jam alerts for all U.S. metropolitan areas. SDC enabled the team to leverage existing crowdsourced traffic incident activity, and rapidly developed dashboard using existing code. The team has continued to provide weekly updates through 2022. The Waze dashboard provides a rapid indicator of traffic jams covering all U.S. metropolitan areas, increasing accessibility to state, metropolitan, and county-level time trends. This ongoing project aims to enhance the dashboards for broader use by U.S. DOT.

Adirondack hot spots and daily trends

Waze alerts showed how traffic jams, crashes, and hazardous parking fluctuated through the Adirondack park’s main road during hiking days. These informed options for a shuttle route, planned to start in summer 2022.

Regional crash monitoring

Waze alerts are providing crash analyses to 31 parks in the Inter-Mountain Region. Waze can supplement current NPS crash data, or in cases where NPS crash data is not available, Waze can act as the major source of crash trends.

Mount Rainier entrance congestion

Waze alerts are giving information about how popular park entrances fill up and create traffic jams. These can be related to the gateway communities and roadways used to enter the park from visitor home locations (e.g. Seattle, Portland, Spokane).

Last updated July 2022