Judging Criteria

These criteria are weighted equally. Applicants should understand that their submittals will be reviewed by technical experts in transportation safety analysis, data, IT and visualization.  If the level of detail is deemed insufficient by the experts, the submission will not be regarded as feasible. The evaluation panels will consider each proposal’s alignment with each of these criteria and make recommendations to the selecting official.

Stage I – Ideation Judging Criteria

Criteria Applying To All Tools:

• Benefits.  Defines a target user or demonstrates the potential for users, should the tool be developed.  Takes into consideration inputs from potential users.  Details appeal and relevance to target user and describes method to measure benefits.  Describes extent to which the tool expands upon existing safety understanding, generates actionable insights for its proposed target audience, and has the potential for dramatic impacts on transportation safety.  Addresses the challenges facing broad adoption, and how could they be overcome.
• Data.  Identifies data requirements and appropriately handles uncertainty.  Describes a new, original idea that integrates data in an unprecedented and novel way or enhances understanding of the data.  Explains what risks or challenges exist for accessing and/or using the data, and how could they be overcome.  Proposes using cost-effective data.  Details an appropriate use of the data for the analytical visualization tool in supporting insight discovery and for the message of the visualization.
• Technology.  Discusses existing technologies and describes how the proposed tool differs from current technologies.  Details the features of the tool and how they can be accessed and understood by the maximum possible number of users.  Describes a wide range of accessibility and uptake for target users by using technology that can easily be deployed and maintained.  Proposes using cost-effective technologies.
• Cost to Implement.  Provides a clear schedule for project implementation, monitoring, and evaluation.  Includes initial estimates of data input and operating costs and includes five-year maintenance estimates for implementing and potentially scaling the tool.  Describes how the benefits of the proposed tool outweigh the costs of end-users obtaining the data, technology, skills and resources necessary to implement and sustain the tool.

Criteria Applying To Only Discover Insights Tool:

• Insights.  Provides an innovative plan and project narrative that describes how the proposed tool will reveal insights to a solution.  Details the tool’s unique differentiating factor for reducing serious crashes.  Identifies and addresses one or more of the priority safety focus areas.  Exemplifies potential for raising awareness about transportation safety and for possible solutions to transportation safety challenges.

Criteria Applying To Only Simulation Tool:

• Simulation.  Provides an innovative plan and project narrative that describes how the proposed tool will allow users to simulate different conditions from models.  Describes the capacity to customize the tool based on different scenarios from models.  Details the accuracy and precision the tool will perform at in visualizing a variety of scenarios from models.  Identifies and addresses one or more of the priority safety focus areas.  Exemplifies potential for raising awareness about transportation safety and for possible solutions to transportation safety challenges.

Stage II – Concepts Judging Criteria

Criteria Applying to All Tools:

• Technical Approach.  Demonstrates significant evolution and improvement of the initial Ideation through additional specifics and refinement of concept.  Demonstrates a high-level of technical merit for the proposed approach to accessing and analyzing the data and designing the tool.
• Design and Desirability.  Demonstrates how the interface and visualization meet the needs of users.  Shows how it could simplify their work or inform their decisions and how will it appeal to them based on intuitive design and ease of use.
• Analytical Depth.  Considers a variety of data sources and application of an appropriate analysis technique.
• Technology Transfer Readiness Level and Feasibility.  Demonstrates a reasonable path for implementation and production, and a clear method for validating data with a high degree of confidence backed by credible supporting evidence.  Uses technology that USDOT or the public sector can easily deploy.  Provides a clear breakdown of data input costs and operating and maintenance costs.
• Testing and Deployment Approach.  Provides a plan for how the system would be tested in a real-world setting and deployed, and describes how the system could be expanded to other geographic areas, to different safety data, or to other scenarios of concern for transportation safety.  Demonstrates feasibility of implementation and scalability of tool and data.
• Team.  Outlines the costs of development and a plan for funding these costs. Demonstrates the commitment and expertise to bring the Full Working Analytical Visualization Tool design to fruition.

Stage III –Tools Judging Criteria

• Sustainability. The extent to which the submission illustrates a plan to maintain consistent long-term access to the tool and ongoing technical support for the tool users, and adequately addresses potential constraints and possible unintended consequences of the tool’s use.
• Deployment Approach. The feasibility of the approach for deploying the tool in real-world settings and expanding usage to multiple geographic areas and/or nationwide.
• Functionality and Technical Effectiveness. The ability of the final product to provide significant insights or visualize scenario analyses. The level of speed, sensitivity and precision (completeness and usefulness) of the results based on the safety problems posed in the Challenge. The intuitive design and ease of use for different user groups and skill levels. The flexibility to adapt to varying user needs.
• Expected Return on Investment. The benefits of the tool as compared to the data, operating and maintenance costs for implementing and potentially scaling the tool.