# Information for Advanced Users

The following sections provide additional information for users who already have some experience with the CMF Clearinghouse and searching for CMFs. These sections provide more advanced information about:

- Referencing individual CMFs
- Downloading data from the CMF Clearinghouse
- Understanding standard error and confidence interval
- Applying multiple CMFs
- Understanding the relationship of the CMF Clearinghouse to the HSM
- Developing high quality CMFs

## How do I Reference Individual CMFs?

Users should always reference the exact CMF they have used when writing reports or communicating with others about their analysis. To facilitate this, the CMF Clearinghouse assigns each CMF a unique ID number (i.e., 3127). This CMF ID is listed at the top of the CMF details page (the page that provides all the details about a particular CMF). Additionally, the CMF details page for each CMF has a unique URL (internet address). For example, the link to the details page for CMF #3127 would be http://www.cmfclearinghouse.org/detail.php?facid=3127. The CMF ID is noticeable as the final characters at the end of that URL.

There are two methods to locate a specific CMF using the CMF ID number. First, the Clearinghouse search mechanism provides an option for searching for a "Single CMF ID" via a checkbox under the search box. A user must simply type in the CMF ID number, check the box for "Single CMF ID", and hit submit. Second, a user can modify the URL of any CMF details page by replacing the final numbers with the ID number of the CMF of interest.

## How do I Download Data from the CMF Clearinghouse?

Some users may prefer to download the results of a CMF search into Excel in order to filter and sort the data on their own. At the bottom of the search results screen is a button to allow users to export the results of their search as an Excel file. The Excel output contains 47 attribute fields for each CMF, including such fields as the star rating, study methodology, roadway type, and the geographic area where the CMF was developed.

## What is the standard Error and Confidence Interval?

It is important to understand that CMFs are developed as *estimates* of the effect on crashes, and each CMF has a range which may contain the true value, referred to as the confidence interval. A larger confidence reflects more uncertainty about the true value of the CMF. This could be due to the fact that the CMF was developed using only a small sample of sites or a set of sites whose data varied widely. A small confidence interval reflects more certainty about the true value and would reflect a CMF that was developed using a large dataset that had more consistent results.

The confidence interval is used to determine whether the CMF is statistically significant and is based on the standard error of the CMF, a measurement of the potential variability in the CMF value. A CMF is determined to be statistically significant if the specified confidence interval of the CMF does not include 1.0, since a value of 1.0 indicates no effect from the countermeasure. For a given CMF and standard error, the confidence interval will depend on the significance level that is used. The most common significance levels are 0.05 (corresponds to 95% confidence interval), 0.10 (corresponds to 90% confidence interval), and 0.15 (corresponds to 85% confidence interval).

For the 95% confidence level, the confidence interval is equal to the CMF ± 1.96 * (standard error).

For the 90% confidence level, the confidence interval is equal to the CMF ± 1.64 * (standard error).

For the 85% confidence level, the confidence interval is equal to the CMF ± 1.44 * (standard error).

**Example 1:**

The CMF for countermeasure A is 0.80 with a standard error of 0.15. The lower and upper limits of the 95% confidence interval are the following:

Lower limit: 0.80 – 1.96*0.15 = 0.80 – 0.294 = **0.51**

Upper limit: 0.80 + 1.96*0.15 = 0.80 + 0.294 = **1.10
**Since the 95% confidence interval (0.51, 1.10) includes 1.0, this CMF is not statistically different from 1.0 (at the significance level 0.05, i.e., confidence level 0.95).

**Example 2:**

On the other hand, if the same CMF had a standard error or 0.09, then the lower and upper limits of the 95% confidence interval will be the following:

Lower limit: 0.80 – 1.96*0.09 = 0.80 – 0.1764 = **0.62**

Upper limit: 0.80 + 1.96*0.09 = 0.80 + 0.1764 = **0.98
**Since the 95% confidence interval (0.62, 0.98) does not include 1.0, this CMF is statistically different from 1.0 (at the significance level 0.05, i.e., confidence level 0.95).

If a CMF is not statistically significant, a user should be cautious and use engineering judgment when applying the CMF to a particular situation. Users should also know that the standard error is used as part of the star rating criteria, so CMFs that are not statistically significant will receive fewer points on the rating scale.

## How do I Apply Multiple CMFs?

It is often the case that an agency will implement more than one countermeasure in a location. If multiple countermeasures are implemented at one location, the common practice is to multiply the CMFs to estimate the combined effect of the countermeasures. However, there is limited research documenting the combined effect of multiple countermeasures. Although implementing several countermeasures might be more effective than just one, it is unlikely the full effect of each countermeasure would be realized when they are implemented concurrently, particularly if the countermeasures are targeting the same crash type.

For example, shoulder rumble strips and enhanced edgeline retroreflectivity would both target roadway departure crashes, so the CMFs for these treatments would be highly related. Other examples of related CMFs would be the use of increased lighting and installation of pavement reflectors, both of which would target nighttime crashes; and chevrons and advanced curve warning signs, both of which would target curve-related crashes.

Countermeasures that would be considered independent are those that target different crash types. For example, the installation of a pedestrian signal would be relatively independent of the installation of a left turn phase at an adjacent intersection, since the one addresses pedestrian-vehicle crashes while the other addresses left-turn opposite-direction crashes. Likewise, the conversion of a left turn phase from permissive to protected along with the installation of an exclusive right turn lane would be fairly independent in that they target different crash types.

Therefore, unless the countermeasures act completely independently, multiplying several CMFs is likely to overestimate the combined effect. The likelihood of overestimation increases with the number of CMFs that are multiplied. Therefore, much caution and engineering judgment should be exercised especially when estimating the combined effect of more than three countermeasures at a given location. The “Using CMFs” page provides links to training videos on how to select and apply methods for analyzing multiple CMFs.

This topic was also presented by Frank Gross at the 2019 CMF Clearinghouse Annual Webinars. For more detailed, see NCHRP Project 17-63 “Guidance for the Development and Application of Crash Modification Factors”.

## What is the Relationship of the CMF Clearinghouse to the HSM?

The first edition of the AASHTO Highway Safety Manual was published in 2010 and serves as a major source of information and guidance on many aspects of road safety, including CMFs. The CMF Clearinghouse incorporates all CMFs from the first edition of the HSM. This includes CMFs used to adjust crash predictions in safety performance functions in Part C and CMFs used to estimate safety effects of various countermeasures in Part D.

Although both the CMF Clearinghouse and the HSM Part D provide CMF information for countermeasures, there are some notable differences:

- The HSM and the CMF Clearinghouse use different methods for determining and indicating the reliability of a CMF. The HSM uses a system of notations (e.g., bold, italics, etc.) to indicate a reliability based primarily on an adjusted standard error, whereas the Clearinghouse uses a star rating as described previously.
- The HSM and the CMF Clearinghouse are different in scope. The HSM presents a single CMF for each countermeasure, whereas the Clearinghouse presents all published CMFs.
- The HSM adjusts both the CMF and the standard error to account for biases, whereas the CMF CH presents both as they are reported by the study author (with a star rating to indicate reliability).

When the Clearinghouse added CMFs from the HSM, star quality ratings were assigned to those CMFs based on the adjusted standard error as listed in the HSM. Go here to see more information about that process and how the HSM and the CMF Clearinghouse compare to each other.

## How do I Develop High Quality CMFs?

Researchers who develop CMFs play a critical role in providing valuable information to the transportation community. It is important that CMF development studies are conducted in such a way as to produce high-quality CMFs that are based on solid data and good methodology and free of biases. Common characteristics of a high-quality CMF are:

- statistically rigorous study design with reference group or randomized experiment and control
- large sample size that covers multiple years with a diversity of sites
- small standard error compared to the value of the CMF
- controlled for all sources of known potential bias
- based on a diverse data source, including states representing different geographies

The Resources section of the Clearinghouse offers a page with several resources on developing CMFs .

- Better CMFs , safer roadways: Tips for building high-quality CMFs . This two-page flyer provides a basic overview on how to develop high-quality CMFs, with information on questions such as, “What does a quality CMF study look like?” and “Why is documentation important?”
- A Guide to Developing Quality Crash Modification Factors. The purpose of this guide is to provide direction to agencies interested in developing crash modification factors (CMFs). Specifically, this guide discusses the process for selecting an appropriate evaluation methodology and the many issues and data considerations related to various methodologies.
- Recommended Protocols for Developing Crash Modification Factors. The CMF Protocols provide guidance for the development and documentation of research studies that develop CMFs. The major goal of these protocols is to describe what pieces of the research study should be documented by the study authors and how various potential biases should be addressed.

The Clearinghouse also provides ideas and inspiration for future CMF research with the CMF Most Wanted List. This list represents areas or specific countermeasures for which the CMF Clearinghouse does not have much good quality information. These areas have been shown to be of interest to users of the Clearinghouse based on an analysis of searches conducted. Essentially, the question posed was, “what are people searching for but not finding?” Examples include realignment of road segments, curb extensions (also called bulb-outs or bump-outs), rectangular rapid flashing beacons, and dynamic feedback speed signs.

# Conclusion

The CMF Clearinghouse serves as a valuable resource to the transportation safety community. The CMFs made available through the Clearinghouse assist safety practitioners in estimating the safety effectiveness of many different countermeasures and support the safety investment decision-making process. CMFs can be a valuable tool if used and applied correctly. The CMF Clearinghouse user guide provides information for users to locate the CMFs they need and gain the knowledge to apply them appropriately and best address critical safety issues.